vitals

0.3.0

Large Language Model Evaluation

0packages depend

3.9Kdownloads / year

78.3%test coverage

13/13checks pass

Overview

About

Maintained by Simon CouchFirst published 2025-06-243 releasesCRAN page ↗GitHub ↗

A port of 'Inspect', a widely adopted 'Python' framework for large language model evaluation. Specifically aimed at 'ellmer' users who want to measure the effectiveness of their large language model-based products, the package supports prompt engineering, tool usage, multi-turn dialog, and model graded evaluations.

Install

R (CRAN)pak (dev)Ubuntu (r2u)openSUSE (OBS)

install.packages("vitals", repos = "https://cloud.r-project.org/")

Installs the current CRAN release. Prebuilt binaries on Windows and macOS.

if (!requireNamespace("pak", quietly = TRUE))
  install.packages("pak", repos = "https://cloud.r-project.org/")
pak::pak("tidyverse/vitals")

Development version from tidyverse/vitals. Installs pak first if it is not already available.

Release

# enable cranapt (r2u) if it is not already set up, then install
grep -rqs r2u.stat.illinois.edu /etc/apt/sources.list /etc/apt/sources.list.d/ \
  || curl -fsSL https://raw.githubusercontent.com/eddelbuettel/r2u/master/inst/scripts/add_cranapt_noble.sh | sudo bash
sudo apt install -y r-cran-vitals

Fast Ubuntu binaries via r2u (no compilation). The cranapt setup runs once and is skipped when it is already enabled.

bspm (used by r2u) also needs make: sudo apt install make

# add the autoCRAN OBS repo (Tumbleweed) if it is missing, then install
zypper -q lr | grep -qi autoCRAN \
  || sudo zypper addrepo https://download.opensuse.org/repositories/devel:/languages:/R:/autoCRAN/openSUSE_Tumbleweed/devel:languages:R:autoCRAN.repo
sudo zypper install -y R-vitals

RPMs auto-built from CRAN on openSUSE OBS (Tumbleweed).

Health

CRAN checks

13OK

Slowest check: 2.9 min · r-devel-linux-x86_64-fedora-clang

Code health

Yes

Tests · ratio 2.49

78.3%

Coverage · measured lines

100%

Documentation · exports

Dependencies · direct

Check history

OK2026-06-09
13 OK · 0 NOTE · 0 WARNING · 0 ERROR · 0 FAILURE
ERROR2026-06-07
12 OK · 0 NOTE · 0 WARNING · 1 ERROR · 0 FAILURE
OK2026-04-25
12 OK · 0 NOTE · 0 WARNING · 0 ERROR · 0 FAILURE
NOTE2026-03-10
11 OK · 3 NOTE · 0 WARNING · 0 ERROR · 0 FAILURE

Documentation

READMEYes · 369 wordsVignettesNopkgdown siteNoNEWSYes · 100% structuredCode of conductNoContributing guideNo

Examples that run: 100%
Documented parameters: 100%
Return-value docs: 89%
References docs: 0%

Downloads

3.9K

CRAN downloads in the past year

Rank #6,699 · ~11/day · ~328/mo

Releases R versions

Daily download trend is not available in this view yet.

35130 days

1.5K90 days

3.9K1 year

Compare downloads with other packages →

Also on681 r2u42 autocran

Repository

GitHub ↗ tidyverse/vitals

58Stars

13Forks

8Open issues

0Open PRs

3Releases

464Commits

12Contributors

464 commits · Last activity 2026-07-12 · +1.8% stars, 30d

Stars over time

2026-07-14 · 572026-07-19 · 58

Repository practices

Upstream repositoryBeta

8 development-tooling and community-health practices detected across 6 families in the upstream repository

Checks run against github.com/tidyverse/vitals on 2026-07-19.

Continuous integration (1)

GitHub Actions

Reproducibility and dev environment (1)

data-raw/

CRAN release process (1)

cran-comments.md

Docs source (1)

README.Rmd

Show all practices

Lint, format, editor (3)

AirVS CodeRStudio project

Coverage (1)

Codecov

How this is detected·Detection ruleset v1 (2026-07-18)

Dependencies

Declared dependencies

23 external dependencies (excludes base and recommended)

Depends (1)

R >= 4.1

Imports (14)

cli dplyr ellmer glue httpuv httr2 jsonlite purrr R6 rlang S7 tibble tidyr withr

LinkingTo (0)

none

Suggests (9)

ggplot2 here htmltools knitr magick ordinal rmarkdown testthat vcr

Enhances (0)

none

Dependency license breakdown →

Reverse dependencies

0direct

0indirect

Nothing depends on this yet.

Code & Tests

People & History

People (5)

Maintainer (1)

Simon Couch

Author, Maintainer

Authors (1)

Simon Couch

Author, Maintainer

Contributors (3)

Mine Cetinkaya-Rundel

Contributor

Contributor

Contributor

Funders (1)

Package Timeline

3 releases. Pick two to compare their code metrics. R releases are shown for context.

0.3.0Latest
2026-05-16 · current release · diff ↗
R
R 4.6.0 released · 2026-04-24
0.2.0
2025-12-01 · diff ↗
0.1.0
2025-06-24
R
R 4.5.0 released · 2025-04-11

Package metadata

First published: 2025-06-24
Total releases: 3 / 1 yrs
License: MIT + file LICENSE OSI
Minimum R: ≥ 4.1
Bundled data: 14 KB / 1 file
Download size: 2.7 MB
Installed size: not tracked yet
With dependencies: not tracked yet

Report a problem with this page →