upstage/Solar-Open2-250B


Solar Open 2

Solar Open 2 is Upstage’s 250B-A15B open-weight large language model, built for agentic use cases such as office productivity, document-intensive work, and coding. Its Hybrid-Attention Mixture-of-Experts (MoE) architecture with linear attention delivers highly efficient inference even in long-context settings.

**Technical Report** | **Blog** | **Upstage Website** | **Try Demo (~7/31)**

Highlights

 

• Agentic Specialist: Purpose-built for agentic workflows — tool calling, multi-step reasoning, and end-to-end task execution. Competitive with the strongest open-weight models on agent benchmarks.

• Minimal Inference Cost: A 250B-parameter MoE that activates only 15B per token, built on a hybrid attention stack that interleaves three linear-attention layers with one softmax-attention layer — large-model capacity at small-model inference cost.

• 1M-Token Context: The linear-attention layers encode token order intrinsically in their recurrent state, so positional encoding is removed entirely (NoPE), lifting the RoPE extrapolation limit. Only 12 of the 48 layers keep a KV cache, holding long-context memory to roughly a quarter of an all-softmax model of the same shape.

• Efficiently Trained at Low Cost: Initialized by selective weight transfer from Solar Open 1 (102B) — only the 2.3% of weights that survive the architectural change are carried over, and everything else is randomly initialized — which raises the starting point and accelerates early convergence at 250B scale.

• Multilingual: English, Korean, and Japanese.

---

Model Overview

| Field | Value | | ------------------------------- | ----------------------------------------------------------------------- | | Model Name | Solar Open 2 (250B-A15B) | | Architecture | Hybrid-Attention Mixture-of-Experts (MoE) | | Total Parameters | 250B (250,287,794,944) | | Active Parameters | 15B (per token) | | Layers | 48 | | Hidden Size | 4096 | | Attention | Hybrid — Softmax + Linear Attention, pattern [Softmax, Linear×3] × 12 | | Position Encoding | NoPE (no rotary positional encoding) | | Number of Attention Heads (GQA) | (Softmax) 64 query / 8 KV, (Linear) 64 query | | Number of Experts | 321 (320 routed + 1 shared) | | Number of Activated Experts | 8 routed (top-8) + 1 shared | | Vocabulary | 196,608 | | Context Length | 1M | | Pre-training Tokens | ~12 Trillion | | Supported Languages | English, Korean, Japanese | | Training Hardware | NVIDIA B200 GPUs | | Training GPU Time | 2M GPU Hours | | License | Upstage Solar License (see LICENSE) | | Hardware Requirements | Minimum: H200 \* 4ea / Recommended: H200 \* 8ea |

---

Performance

English Benchmarks

| Benchmark | Solar Open 2 250B-A15B | Solar Open 100B 102B-A12B | Command A+ 218B-A25B | Mistral Medium 3.5 128B dense, high | MiMo-V2.5 310B-A15B | DeepSeek-V4-Flash 284B-A13B, max | | --------------------- | ----------------------------: | -------------------------------: | --------------------------: | -----------------------------------------: | -------------------------: | --------------------------------------: | | Know. & Reasoning | | | | | | | | MMLU-Pro | 86.2 | 80.4 | 79.0 | 81.2 | 84.6 | 85.9 | | GPQA-Diamond | 86.3 | 66.2 | 75.6 | 77.5 | 83.0 | 88.9 | | HLE (w/o tools) | 28.8 | 11.5 | 11.4 | 12.8 | 24.3 | 32.3 | | LiveCodeBench (v6) | 92.4 | 56.5 | 86.1 | 84.9 | 89.1 | 92.3 | | ArtifactsBench | 55.9 | 43.4 | 42.8 | 49.8 | 59.3 | 61.0 | | HMMT2602 | 93.9 | 68.9 | 73.5 | 62.9 | 61.4 | 94.7 | | AIME2026 | 95.7 | 87.7 | 96.0 | 89.0 | 92.3 | 97.0 | | IF / Long | | | | | | | | Multi-Challenge | 61.0 | 40.5 | 45.8 | 49.8 | 39.0 | 62.0 | | IFBench | 80.0 | 57.7 | 73.9 | 69.0 | 67.1 | 80.3 | | AA-LCR | 62.3 | 36.0 | 46.0 | 61.0 | 62.7 | 63.7 | | Agent | | | | | | | | SWE-Bench Verified | 70.4 | 15.4 | 14.4 | 69.6 | 73.0 | 73.8 | | Terminal Bench Hard | 28.3 | 2.3 | 25.0 | 33.3 | 41.7 | 34.1 | | APEX-Agents | 16.6 | 2.4 | 1.6 | 6.1 | 13.4 | 13.2 | | MCP-Atlas | 58.2 | 34.4 | 27.2 | 30.7 | 63.9 | 58.2 | | τ³ (banking) | 19.6 | 7.4 | 5.8 | 5.8 | 8.7 | 22.3 | | GDPval-AA v2 (ELO) | 1128 | – | 712 | 929 | 1145 | 1187 |

Korean Benchmarks

| Benchmark | Solar Open 2 250B-A15B | Solar Open 100B 102B-A12B | MiMo-V2.5 310B-A15B | DeepSeek-V4-Flash 284B-A13B, max | Claude Haiku 4.5 closed | GPT-5.4 mini closed | | ------------ | ----------------------------: | --------------------------------: | -------------------------: | --------------------------------------: | -----------------------------: | -------------------------: | | KMMLU-Pro | 78.4 | 64.0 | 69.1 | 78.9 | 67.9 | 78.1 | | CLIcK | 90.7 | 78.9 | 78.4 | 89.2 | 53.5 | 89.6 | | HAE-RAE v1.1 | 73.8 | 73.3 | 61.7 | 73.1 | 38.5 | 69.4 | | Ko-AIME’25† | 97.7 | 80.0 | 88.0 | 98.0 | 81.7 | 90.7 | | HRM8K | 92.2 | 87.6 | 90.7 | 93.4 | 90.6 | 91.3 | | KBank-MMLU† | 80.8 | 65.5 | 71.0 | 79.5 | 68.9 | 79.0 | | KBL | 75.5 | 65.5 | 69.8 | 72.8 | 69.9 | 75.3 | | KorMedMCQA | 93.0 | 84.4 | 87.7 | 94.1 | 87.0 | 94.2 | | Ko-GDPval† | 86.8 | 3.4 | 81.0 | 85.0 | 68.3 | 59.4 |

† in-house benchmarks.

---

Quickstart

The examples below assume 8 GPUs with at least 141 GB of memory each, such as NVIDIA H200 or B200 GPUs. Actual memory requirements depend on the context length and serving settings.

Transformers

Use the Upstage Transformers branch with native Solar Open 2 support for local experimentation. For production serving, we recommend vLLM.

Install the dependencies:

> Install a CUDA-enabled PyTorch build for your platform before running this command. fla-core enables the optimized KDA kernels; without it, Transformers uses a substantially slower PyTorch fallback.

python -m pip install -U \
"git+https://github.com/upstageAI/transformers.git@v5.14.1-solar-open2" \...