quallmer.app
Interactive Validation App for 'quallmer'
Description
Companion package to 'quallmer' providing an interactive 'shiny' application for manual coding, reviewing large language model (LLM) generated annotations, and computing inter-rater reliability metrics. Supports three modes: blind manual coding, LLM output validation, and agreement calculation. Computes standard reliability metrics including Krippendorff's alpha (Krippendorff 2019 <doi:10.4135/9781071878781>), Cohen's kappa, Fleiss' kappa (Fleiss 1971 <doi:10.1037/h0031619>), intraclass correlation coefficient (ICC), and percent agreement for nominal, ordinal, interval, and ratio data. Also computes gold-standard validation metrics including accuracy, precision, recall, and F1 scores following Sokolova and Lapalme (2009 <doi:10.1016/j.ipm.2009.03.002>).
Downloads
3
Last 30 days
23338th
3
Last 90 days
3
Last year
CRAN Check Status
Show all 6 flavors
| Flavor | Status |
|---|---|
| r-devel-linux-x86_64-fedora-clang | OK |
| r-devel-linux-x86_64-fedora-gcc | OK |
| r-devel-macos-arm64 | OK |
| r-oldrel-macos-arm64 | OK |
| r-oldrel-windows-x86_64 | OK |
| r-release-macos-arm64 | OK |
Check details (6 non-OK)
*
*
*
*
*
*