sambanova/tokenizers

sambanova/tokenizers

Stars: 0

Forks: 0

Open issues: 0

Created: 2025-06-24T19:56:26Z

Pushed: 2025-07-03T21:06:50Z

Default branch: main

Fork: no

Archived: no

README:

Tokenizers

This repo contains pre-loaded tokenizers files from Hugging Face for the following models:

• Meta-Llama-3-8B (with extended vocab)
• Meta-Llama-3-8B-Instruct (with extended vocab)
• Meta-Llama-3.1-8B
• Meta-Llama-3.1-8B-Instruct
• Meta-Llama-3.1-70B
• Meta-Llama-3.1-70B-Instruct
• Meta-Llama-3.2-1B
• Meta-Llama-3.2-1B-Instruct
• Meta-Llama-3.2-3B
• Meta-Llama-3.2-3B-Instruct
• Mistral-7B-v0.3:
• Mistral-7B-Instruct-v0.3
• GPT2