mistralai/Magistral-Small-2507

Magistral Small 1.1

Building upon Mistral Small 3.1 (2503), with added reasoning capabilities, undergoing SFT from Magistral Medium traces and RL on top, it's a small, efficient reasoning model with 24B parameters.

Magistral Small can be deployed locally, fitting within a single RTX 4090 or a 32GB RAM MacBook once quantized.

Learn more about Magistral in our blog post.

The model was presented in the paper Magistral.

Updates compared with Magistral Small 1.0

Magistral Small 1.1 should give you about the same performance as Magistral Small 1.0 as seen in the [benchmark results](#benchmark-results).

The update involves the following features:

• Better tone and model behaviour. You should experiment better LaTeX and Markdown formatting, and shorter answers on easy general prompts.
• The model is less likely to enter infinite generation loops.
• [THINK] and [/THINK] special tokens encapsulate the reasoning content in a thinking chunk. This makes it easier to parse the reasoning trace and prevents confusion when the '[THINK]' token is given as a string in the prompt.
• The reasoning prompt is now given in the system prompt.

Key Features

• Reasoning: Capable of long chains of reasoning traces before providing an answer.
• Multilingual: Supports dozens of languages, including English, French, German, Greek, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Nepali, Polish, Portuguese, Romanian, Russian, Serbian, Spanish, Turkish, Ukrainian, Vietnamese, Arabic, Bengali, Chinese, and Farsi.
• Apache 2.0 License: Open license allowing usage and modification for both commercial and non-commercial purposes.
• Context Window: A 128k context window, but performance might degrade past 40k. Hence we recommend setting the maximum model length to 40k.

Benchmark Results

| Model | AIME24 pass@1 | AIME25 pass@1 | GPQA Diamond | Livecodebench (v5) | |---------------------------|---------------|---------------|--------------|--------------------| | Magistral Medium 1.1 | 72.03% | 60.99% | 71.46% | 59.35% | | Magistral Medium 1.0 | 73.59% | 64.95% | 70.83% | 59.36% | | Magistral Small 1.1 | 70.52% | 62.03% | 65.78% | 59.17% | | Magistral Small 1.0 | 70.68% | 62.76% | 68.18% | 55.84% |

Sampling parameters

Please make sure to use:

• top_p: 0.95
• temperature: 0.7
• max_tokens: 40960

Basic Chat Template

We highly recommend including the following system prompt for the best results, you can edit and customise it if needed for your specific use case.

> First draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input. > > Your thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response. Use the same language as the input.[/THINK]Here, provide a self-contained response.

The [THINK] and [/THINK] are special tokens that must be encoded as such.

*Please make sure to use [mistral-common](https://github.com/mistralai/mistral-common) as the source of truth*. Find [below](#usage) examples from libraries supporting mistral-common.

We invite you to choose, depending on your use case and requirements, between keeping reasoning traces during multi-turn interactions or keeping only the final assistant response.

Usage

The model can be used with the following frameworks;

Inference

• `vllm (recommended)`: See [below](#vllm-recommended)
• `transformers`: See [below](#transformers)

In addition the community has prepared quantized versions of the model that can be used with the following frameworks (*alphabetically sorted*):

• `llama.cpp`: https://huggingface.co/mistralai/Magistral-Small-2507-GGUF
• `lmstudio` (llama.cpp, MLX): GGUF, MLX-bf16, MLX-8bit, MLX-6bit, MLX-4bit

Training

Fine-tuning is possible with (*alphabetically sorted*):

• `axolotl`: https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/magistral
• `unsloth`: https://docs.unsloth.ai/basics/magistral

vLLM (recommended)

We recommend using this model with the vLLM library to implement production-ready inference pipelines.

_Installation_

Make sure you install the latest `vLLM` code:

pip install -U vllm \
--pre \
--extra-index-url https://wheels.vllm.ai/nightly

Doing so should automatically install `mistral_common >= 1.8.2`.

To check:

python -c "import mistral_common; print(mistral_common.__version__)"

You can also make use of a ready-to-go docker image or on the docker hub.

Serve model as follows:

vllm serve mistralai/Magistral-Small-2507 --reasoning-parser mistral --tokenizer_mode mistral --config_format mistral --load_format mistral --tool-call-parser mistral --enable-auto-tool-choice --tensor-parallel-size 2

Ping model as follows:

Python snippet

from typing import Any
from openai import OpenAI
from huggingface_hub import hf_hub_download

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

TEMP = 0.7
TOP_P = 0.95
MAX_TOK = 40_960

client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)…