mistralai/mistral-inference v1.2.0

v1.2.0 Add Mamba

Repository: mistralai/mistral-inference

Tag: v1.2.0

Published: 2024-07-16T12:11:27Z

Prerelease: no

Release notes:

Welcome 🐍 Codestral-Mamba and 🔢 Mathstral

pip install mistral-inference>=1.2.0

Codestral-Mamba

pip install packaging mamba-ssm causal-conv1d transformers

1. Download

export MAMBA_CODE=$HOME/7B_MAMBA_CODE
wget https://models.mistralcdn.com/codestral-mamba-7b-v0-1/codestral-mamba-7B-v0.1.tar
mkdir -p $MAMBA_CODE
tar -xf codestral-mamba-7B-v0.1.tar -C $MAMBA_CODE

2. Chat

mistral-chat $HOME/7B_MAMBA_CODE --instruct --max_tokens 256

Mathstral

1. Download

export MATHSTRAL=$HOME/7B_MATH
wget https://models.mistralcdn.com/mathstral-7b-v0-1/mathstral-7B-v0.1.tar
mkdir -p $MATHSTRAL
tar -xf mathstral-7B-v0.1.tar -C $MATHSTRAL

2. Chat

mistral-chat $HOME/7B_MATH --instruct --max_tokens 256

Blogs: Blog Codestral Mamba 7B: https://mistral.ai/news/codestral-mamba/ \ Blog Mathstral 7B: https://mistral.ai/news/mathstral/

What's Changed

• add a note about GPU requirement by @sophiamyang in https://github.com/mistralai/mistral-inference/pull/158
• Add codestral by @patrickvonplaten in https://github.com/mistralai/mistral-inference/pull/164
• Update README.md by @patrickvonplaten in https://github.com/mistralai/mistral-inference/pull/165
• fixing type in README.md by @didier-durand in https://github.com/mistralai/mistral-inference/pull/175
• Fix: typo in ModelArgs: "infered" to "inferred" by @CharlesCNorton in https://github.com/mistralai/mistral-inference/pull/174
• fix: typo in LoRALoaderMixin: correct "multipe" to "multiple" by @CharlesCNorton in https://github.com/mistralai/mistral-inference/pull/173
• fix: Correct typo in classifier.ipynb from "alborithm" to "algorithm" by @CharlesCNorton in https://github.com/mistralai/mistral-inference/pull/167
• Fix: typo in error message for state_dict validation by @CharlesCNorton in https://github.com/mistralai/mistral-inference/pull/172
• fix: Correct misspelling in ModelArgs docstring by @CharlesCNorton in https://github.com/mistralai/mistral-inference/pull/171
• Update README.md by @patrickvonplaten in https://github.com/mistralai/mistral-inference/pull/168
• fix: typo in HF_TOKEN environment variable check message by @CharlesCNorton in https://github.com/mistralai/mistral-inference/pull/179
• Adding Issue/Bug template. by @pandora-s-git in https://github.com/mistralai/mistral-inference/pull/178
• typo in ModelArgs class docstring. by @CharlesCNorton in https://github.com/mistralai/mistral-inference/pull/183
• Update README.md by @Simontwice in https://github.com/mistralai/mistral-inference/pull/184
• Add mamba by @patrickvonplaten in https://github.com/mistralai/mistral-inference/pull/187

New Contributors

• @didier-durand made their first contribution in https://github.com/mistralai/mistral-inference/pull/175
• @CharlesCNorton made their first contribution in https://github.com/mistralai/mistral-inference/pull/174
• @pandora-s-git made their first contribution in https://github.com/mistralai/mistral-inference/pull/178
• @Simontwice made their first contribution in https://github.com/mistralai/mistral-inference/pull/184

Full Changelog: https://github.com/mistralai/mistral-inference/compare/v1.1.0...v1.2.0