cohere-ai/tokenizers v0.9.0
cohere-ai/tokenizers
Captured source
source ↗published Aug 15, 2024seen 5dcaptured 13hhttp 200method plain
v0.9.0
Repository: cohere-ai/tokenizers
Tag: v0.9.0
Published: 2024-08-15T19:45:41Z
Prerelease: no
Release notes: Following: https://github.com/daulet/tokenizers/releases/tag/v0.9.0
> # What's Changed > * feat: add option to retrieve offsets from tokenizer > * Update to huggingface/tokenizers v0.20.0
Notability
notability 3.0/10Routine version release of existing library