ibm-granite/granite-switch-4.1-8b

Granite Switch 4.1 8B

📝 Read the launch blog: Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

Model Summary: Granite Switch 4.1 8B is a modular LLM that composes a base model, multiple adapters and an encoder together so that a single hosted endpoint can move between natural-language responses, general and specialized decisions, and speech-to-text.

Adapters are activated on demand via control tokens in the chat template. For full details on model composition and adapter configuration, see [BUILD.md](BUILD.md).

• Base Model: ibm-granite/granite-4.1-8b (8B params, 128K context)
• Adapters: 2 — decision (aLoRA, general-purpose Decisions) and guardian-core (aLoRA, from granitelib-guardian-r1.0)
• Audio: embedded speech-to-text via ibm-granite/granite-speech-5.0-470m-turboctc (470M params, English)
• License: Apache 2.0
• Release Date: October 14, 2026
• Backends: HuggingFace Transformers, vLLM
• Automatically Composed with: granite-switch

Granite Switch is also available in granite-switch-4.1-30b (same Chat + Decisions + Guardian + speech configuration, 30B base), and with the complete Granite Libraries bundle (no audio) in granite-switch-4.1-8b-preview, -30b-preview, and -3b-preview.

Included Adapters & Encoders

| Switch Component | Description | |---|---| | **Guardian Core aLoRA** | Detects safety risks: harm, jailbreaking, profanity, violence, sexual content, social bias, unethical behavior | | **Decision aLoRA** | Given a state + a question with labelled options, returns a softmax probability distribution over the option labels in one forward pass | | **Granite Speech 5.0 TurboCTC Encoder** | An ASR model for quick speech to text transcription tasks |

Speech-to-Text (Audio Input)

Transcription runs on the same endpoint as chat and decisions, so you don't need a separate speech server and how the model is called doesn't change. Incoming audio is transcribed by the embedded **Granite Speech 5.0 TurboCTC** model (470M params, English, non-autoregressive conformer CTC encoder), the fastest ASR model on the OpenASR Leaderboard . The transcript is spliced into the chat context in place of the audio content part. The speech encoder and the LLM are not yet jointly trained — there is no trained projection coupling them together yet; see docs/AUDIO.md in the granite-switch repo for the full design and limitations (English only, lowercase/unpunctuated output, no decode-time steering).

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")

response = client.chat.completions.create(
model="ibm-granite/granite-switch-4.1-8b",
messages=[{"role": "user", "content": [
{"type": "text", "text": "Answer the question in the audio."},
{"type": "input_audio", "input_audio": {"data": "", "format": "wav"}},
]}],
)
print(response.choices[0].message.content)

Decision Index (JEV) — decision aLoRA

The decision activated-LoRA adapter was evaluated on the public Decision Index fast subset (1,068 rows) in the edition-v0.3 scoring, using the adapter's documented invocation recipe: a fixed decision-engine system prompt, a State: turn (run on the frozen base), and a ### DECISION turn presenting the question and single-token-labelled options, with the first assistant token read as a softmax over the option-label logits. Each configuration was measured two ways — the adapter served locally and the same adapter served behind a hosted vLLM endpoint — which agree to within ~0.6 points, confirming the serving path is faithful; hosted-serving numbers are shown below.

| Overall | Knowledge | Language | Retrieval | Tools | Arts | |---|---|---|---|---|---| | 42.9 | 29.1 | 48.9 | 49.8 | 59.8 | 18.0 |

Scores are the Decision Index v0.3 public component (0–100); per-area skills are shown ×100.

---

Generation

git clone https://github.com/generative-computing/granite-switch.git
cd granite-switch

# Pick what you need:
pip install -e ".[compose]" # Compose models with adapters
pip install -e ".[hf]" # HuggingFace inference
pip install -e ".[vllm]" # vLLM inference
pip install -e ".[audio]" # Audio input (adds vLLM's audio deps)
pip install -e ".[dev]" # Everything

Recreating this checkpoint

This checkpoint is produced by composing the base model with guardian-core from the Guardian library and decision from the Core library — --include-adapters drops every other adapter each library ships — plus the ASR cascade (--enable-audio, default Granite Speech 5.0 TurboCTC):

python -m granite_switch.composer.compose_granite_switch \
--base-model ibm-granite/granite-4.1-8b \
--adapters ibm-granite/granitelib-guardian-r1.0 ibm-granite/granitelib-core-r1.0 \
--include-adapters guardian-core decision \
--enable-audio \
--output ./granite-switch-4.1-8b

See [BUILD.md](BUILD.md) for the exact adapter ranks/targets this checkpoint ships with, and the adapter catalog for every other adapter available to compose into your own checkpoint.

Using with Mellea

Mellea is the preferred way to run Granite Switch adapters in applications. It wraps every adapter in the Granite Libraries family (guardian, rag, core) as a direct operation instead of raw prompt engineering — see the full Mellea + Granite Switch guide for the complete set of...