piecemaker
Tools for Preparing Text for Tokenizers
v1.0.2
·
Jun 2, 2023
·
Apache License (>= 2)
Description
Tokenizers break text into pieces that are more usable by machine learning models. Many tokenizers share some preparation steps. This package provides those shared steps, along with a simple tokenizer.
Downloads
268
Last 30 days
11840th
268
Last 90 days
268
Last year
CRAN Check Status
14
OK
Show all 14 flavors
| Flavor | Status |
|---|---|
| r-devel-linux-x86_64-debian-clang | OK |
| r-devel-linux-x86_64-debian-gcc | OK |
| r-devel-linux-x86_64-fedora-clang | OK |
| r-devel-linux-x86_64-fedora-gcc | OK |
| r-devel-macos-arm64 | OK |
| r-devel-windows-x86_64 | OK |
| r-oldrel-macos-arm64 | OK |
| r-oldrel-macos-x86_64 | OK |
| r-oldrel-windows-x86_64 | OK |
| r-patched-linux-x86_64 | OK |
| r-release-linux-x86_64 | OK |
| r-release-macos-arm64 | OK |
| r-release-macos-x86_64 | OK |
| r-release-windows-x86_64 | OK |
Check details (14 non-OK)
OK
r-devel-linux-x86_64-debian-clang
*
OK
r-devel-linux-x86_64-debian-gcc
*
OK
r-devel-linux-x86_64-fedora-clang
*
OK
r-devel-linux-x86_64-fedora-gcc
*
OK
r-devel-macos-arm64
*
OK
r-devel-windows-x86_64
*
OK
r-oldrel-macos-arm64
*
OK
r-oldrel-macos-x86_64
*
OK
r-oldrel-windows-x86_64
*
OK
r-patched-linux-x86_64
*
OK
r-release-linux-x86_64
*
OK
r-release-macos-arm64
*
OK
r-release-macos-x86_64
*
OK
r-release-windows-x86_64
*
Check History
OK 14 OK · 0 NOTE · 0 WARNING · 0 ERROR · 0 FAILURE Mar 9, 2026
Reverse Dependencies (2)
imports
Dependency Network
Version History
new
1.0.2
Mar 9, 2026