Skip to content

corpustools

Managing, Querying and Analyzing Tokenized Text

v0.5.2 · Jul 7, 2025 · GPL-3

Description

Provides text analysis in R, focusing on the use of a tokenized text format. In this format, the positions of tokens are maintained, and each token can be annotated (e.g., part-of-speech tags, dependency relations). Prominent features include advanced Lucene-like querying for specific tokens or contexts (e.g., documents, sentences), similarity statistics for words and documents, exporting to DTM for compatibility with many text analysis packages, and the possibility to reconstruct original text from tokens to facilitate interpretation.

Downloads

562

Last 30 days

6758th

1.2K

Last 90 days

1.2K

Last year

Trend: -12.9% (30d vs prior 30d)

CRAN Check Status

14 OK
Show all 14 flavors
Flavor Status
r-devel-linux-x86_64-debian-clang OK
r-devel-linux-x86_64-debian-gcc OK
r-devel-linux-x86_64-fedora-clang OK
r-devel-linux-x86_64-fedora-gcc OK
r-devel-macos-arm64 OK
r-devel-windows-x86_64 OK
r-oldrel-macos-arm64 OK
r-oldrel-macos-x86_64 OK
r-oldrel-windows-x86_64 OK
r-patched-linux-x86_64 OK
r-release-linux-x86_64 OK
r-release-macos-arm64 OK
r-release-macos-x86_64 OK
r-release-windows-x86_64 OK
Check details (14 non-OK)
OK r-devel-linux-x86_64-debian-clang

*


            
OK r-devel-linux-x86_64-debian-gcc

*


            
OK r-devel-linux-x86_64-fedora-clang

*


            
OK r-devel-linux-x86_64-fedora-gcc

*


            
OK r-devel-macos-arm64

*


            
OK r-devel-windows-x86_64

*


            
OK r-oldrel-macos-arm64

*


            
OK r-oldrel-macos-x86_64

*


            
OK r-oldrel-windows-x86_64

*


            
OK r-patched-linux-x86_64

*


            
OK r-release-linux-x86_64

*


            
OK r-release-macos-arm64

*


            
OK r-release-macos-x86_64

*


            
OK r-release-windows-x86_64

*


            

Check History

OK 14 OK · 0 NOTE · 0 WARNING · 0 ERROR · 0 FAILURE Mar 9, 2026

Reverse Dependencies (2)

imports

suggests

Dependency Network

Dependencies Reverse dependencies wordcloud stringi Rcpp R6 udpipe digest data.table quanteda (>= 1.5.1) igraph tokenbrowser RNewsflow Matrix pbapply rsyntax text2sdg LexisNexisTools corpustools

Version History

new 0.5.2 Mar 9, 2026