Cerebras At Neurips 2025 Nine Papers From Pretraining To Inference

Cerebras Skip to main content

Cerebras Announces First Quarter 2026 Results >>

Dec 04 2025 Cerebras at NeurIPS 2025: Nine Papers From Pretraining to Inference

Cerebras is excited to be at NeurIPS 2025—and what a year it's been. We launched our inference API last August, opened a new data center in Oklahoma City, and watched demand for Cerebras Inference explode with the latest state-of-the-art open weight models. Our research team has been hard at work too, and this year they're presenting nine papers probing the foundational questions of modern AI practice: where does compute get wasted during training? How should reasoning models allocate tokens at inference? When do smaller models beat bigger ones? The work spans pretraining to inference—new findings on scaling laws, training efficiency, and smarter orchestration of test-time compute. Below is an overview of each paper, what we found, and who should care. Links to the full arXiv papers are included. If you're at the conference, stop by our booth and take a selfie with the wafer. If you're not, we'd still love to hear your thoughts via X or email. The Conductor and the Engine: A Path Towards Co-Designed Reasoning CODA (Conductor-Driven Architecture) optimizes test-time compute across planning, execution, self-refinement, and verification based on problem difficulty—helping 32B models outperform 235B+ competitors in our benchmarks. The insight is that most test-time compute gets wasted: external orchestration and internal model reasoning both try to handle high-level thinking, duplicating effort and burning tokens. CODA separates the roles cleanly—the Conductor plans, reflects, and verifies; the Engine executes—unlocking frontier-level performance without frontier-level model sizes. Key findings 32B models beat 235B+ competitors. Qwen3 32B with CODA outperforms Qwen3 235B, DeepSeek R1, and OpenAI o3-mini on AIME and LiveCodeBench. The 120B GPT-OSS model hits 87.5% on LiveCodeBench, topping Grok4. Adaptive path selection matters. Some math problems solve better through direct reasoning; others need code generation. CODA dynamically picks the right approach per problem—on 15 hard Numina Math questions, neither path alone reaches 100%, but adaptive selection does. Verification is the bottleneck. There's a consistent gap between CODA's final accuracy and the theoretical best-of-N ceiling, suggesting that better verifiers (potentially trained via RL) could unlock substantial additional gains.

Who this is useful for Teams deploying reasoning systems on memory-constrained hardware. Researchers exploring test-time compute scaling. Anyone wanting frontier-level performance without frontier-level model sizes. 📄 [ Paper Link ] Calibrated Reasoning: An Explanatory Verifier for Dynamic and Efficient Problem-Solving When scaling reasoning models at test time, practitioners face a frustrating bottleneck: these models are terrible at evaluating their own solutions. Multi-path exploration strategies like best-of-n sampling rely on the model knowing when it's right or wrong, but current reasoning models are heavily biased toward giving everything high scores regardless of correctness. This paper introduces an Explanatory Verifier trained via reinforcement learning (GRPO) that produces calibrated confidence scores along with natural language reasoning for why solutions are correct or incorrect. The key insight is comparing pairs of candidate solutions rather than evaluating them in isolation, which helps the verifier catch subtle errors. Key findings 1-3× token savings in best-of-n. The verifier achieves higher accuracy at low k values compared to self-consistency, and matches accuracy at higher k while using substantially fewer tokens—because it only generates additional candidates when both current options look wrong. Catches failure modes voting can't. When both candidate solutions are identically incorrect (a common failure mode on hard problems where models collapse into narrow, biased answers), majority voting fails completely. The verifier reliably detects these cases. Transfers across model scales. Though trained on Qwen3-8B outputs, the 8B verifier effectively evaluates generations from the larger Qwen3-32B, achieving 0.77 accuracy on AIME 2025 while using only 75% of the tokens versus self-consistency.

Who this is useful for Teams running best-of-n or majority voting at scale and burning tokens on redundant generations. Researchers working on reward models and verifiers for reasoning. Anyone who's noticed their reasoning model is confidently wrong and wants a better filter. 📄 [ Paper Link ] DREAM: Drafting with Refined Target Features and Entropy-Adaptive Cross-Attention Fusion for Multimodal Speculative Decoding Speculative decoding has become a go-to technique for accelerating text-only LLMs, but applying it to vision-language models is harder—visual and textual information need to stay tightly integrated throughout generation. DREAM is a speculative decoding framework built specifically for VLMs, combining three innovations: a cross-attention mechanism that injects intermediate features from the target model into the draft model, entropy-adaptive feature selection to guide draft training, and visual token compression to reduce latency without losing critical information. Key findings Up to 3.6× speedup over standard decoding. Across LLaVA, Pixtral, SmolVLM, and Gemma3, DREAM consistently outperforms prior speculative decoding baselines in both throughput and acceptance length. Cross-attention is the key architectural choice. Unlike text-only methods that concatenate features, DREAM's cross-attention preserves structured visual representations—removing it causes the largest performance drop in ablations. Visual tokens can be aggressively compressed. Retaining just 75% of visual tokens guided by attention-based importance scores yields a 7% speedup with minimal accuracy loss.

Who this is useful for Teams deploying vision-language models in latency-sensitive applications. Researchers working on efficient multimodal inference. Anyone running VLMs at scale and looking to cut decoding time without sacrificing output quality. 📄 [ Paper Link ] Don't be lazy: CompleteP enables compute-efficient deep transformers When scaling up transformer models, hyperparameters like learning rate that work well for smaller models often need expensive retuning as you increase depth. This...