ReleaseCohereCoherepublished Aug 15, 2024seen 5d

cohere-ai/tokenizers v0.9.0

cohere-ai/tokenizers

Open original ↗

Captured source

source ↗
published Aug 15, 2024seen 5dcaptured 13hhttp 200method plain

v0.9.0

Repository: cohere-ai/tokenizers

Tag: v0.9.0

Published: 2024-08-15T19:45:41Z

Prerelease: no

Release notes: Following: https://github.com/daulet/tokenizers/releases/tag/v0.9.0

> # What's Changed > * feat: add option to retrieve offsets from tokenizer > * Update to huggingface/tokenizers v0.20.0

Notability

notability 3.0/10

Routine version release of existing library