Moe Guide Router

Cerebras Skip to main content

Cerebras Announces First Quarter 2026 Results >>

Aug 04 2025 Router Wars: Which MoE Routing Strategy Actually Works Daria Soboleva

MoE Fundamentals | Router Wars | Debugging Dead MoE Models | MoE at Scale | MoE Math Demystified Here’s what nobody tells you about Mixture-of-Experts (MoE): the router can single-handedly destroy your model. You can have perfect expert network architecture, tuned hyperparameters, and unlimited compute, but if your router collapses, you’re back to dense model performance regardless of number of experts you choose. The router’s job sounds simple – it needs to decide which expert handles each token. In practice, it’s where most MoE implementations go wrong. With wrong strategy you can spend weeks debugging and be completely lost. So which routing strategy should you use and what to expect from it? Let’s examine the most common approaches, their real-world tradeoffs, and what works in practice. The Routing Landscape: Oh So Many Flavors… Table 1: MoE routing reality. Behind the marketing hype, all production systems use some version of learned routing with engineering tricks layered on top. Let’s first address the elephant in the room. Why should you care about routing techniques from 2017-2022 when there are dozens of newer methods being published every month week? It’s because every current production MoE model today is built on top of them! In Table 1 you can see fancy names such as shared experts, capacity factors, adaptive auxiliary (aux) loss or expert bias (and there is many more.) These are just engineering tricks layered on core methods developed almost a decade ago. What are they trying to fix? Two fundamental problems: expert utilization (i.e. are all your experts actually being used?) and expert specialization (i.e. are your experts learning different things, or just copying each other introducing redundancy?). What about DeepSeek-V3’s novel routing method? It’s a vanilla learned routing with aux loss on the sequence level and extra engineering tricks to improve expert utilization. Qwen3’s routing breakthrough? It’s also a learned routing method with aux loss, however, on the global batch level – in other words, it simply relaxes load balancing regularization a bit more to make experts more specialized. Want to pick something off the shelf? Go ahead, use Table 1 and close this guide. But, when your shiny new routing method fails at 3am in the morning during a multi-million-dollar training run, you’ll be debugging one of the core approaches underneath all engineering layers that we will explore in a greater depth in the rest of this guide. Figure 1 shows our moe_layer definition from part 1 to illustrate where the router network sits within the MoE layer. Figure 1: Pseudocode for MoE layer implementation. The Three Fundamental Approaches Figure 2: Router choice makes or breaks MoE performance scaling. At 128 experts, learned and Sinkhorn routing both deliver 3x bigger quality gains than hash routing, with the gap widening as we increase expert count. Hash Routing: The Safe but Boring Choice Hash routing (Roller et al., 2021) is the most straightforward approach – the router from moe_layer simply assigns tokens using: where N is the number of experts and token_id is the token index in the vocabulary. It’s deterministic, easy to understand, and impossible to break. It also doesn’t work very well. Looking at Figure 3a, hash routing maintains perfect load balancing across all layers – every expert gets the same number of tokens (high expert utilization). But Figure 4a shows why this doesn’t help: experts end up learning overlapping, similar representations because token assignments are completely disregarding the token’s context (low expert specialization). A token representing “function” in code and “function” in a math paper might have similar token_ids but need completely different processing. Hash routing can’t tell the difference. As a result, with 16 experts, hash routing gives you only 1.5% loss improvement (compared to Chinchilla-optimal dense scaling (Hoffmann et al., 2022) at a fixed compute) and it barely increases with more experts (Figure 2). Figure 3: Expert load balancing across layers. Hash routing (a) keeps perfect balance, learned routing (b) collapses in early/late layers, Sinkhorn routing ( c ) maintains hash-level load balancing across all layers. Learned Routing: The Industry Standard With hash routing the problem is clear: ignoring context kills performance . Learned routing, first introduced in (Shazeer et al., 2017), takes the opposite approach – it learns how experts should handle each token. Concretely, the router from moe_layer is now a learned linear layer that outputs logits for each expert. To penalize the router for potential imbalances, we add an auxiliary loss: where f represents the fraction of tokens that activate each expert (binary: either a token’s top_k includes that expert or not), experts’ mixing weights we already defined in moe_layer , and coeff controls how hard you want to enforce balance. The results are impressive. With 16 experts, learned routing delivers a solid 4% loss improvement – nearly 3x larger gains than hash routing (Figure 2)! This is why every production MoE system uses some variant of learned routing (Table 1). The magic happens through specialization. Figure 4b shows learned routing creating clean, separated expert representations – each expert carves out its own specialty instead of producing overlapping patterns like in hash routing. But there is a problem: router collapse! Figure 3b shows that while middle layers balance well, early and late layers funnel most tokens to just 1-2 experts. This creates load balancing nightmares for distributed training (for example, when using expert parallelism (DeepSeek-AI et al., 2024)). This is why Deepseek-V3 (DeepSeek-AI et al., 2024) and Qwen2 (Qwen et al., 2024) MoE models use shared experts (always activated experts alongside the routed ones). Figure 4: The story of expert specialization (middle layer)[1]. Hash (a) creates a mess – experts learn similar representations because assignments are random. Learned routing (b) works beautifully in the middle layers - each expert finds its niche. Sinkhorn ( c ) enforces balance so strictly that experts can’t specialize properly. Sinkhorn Routing: The Per-Layer Load Balancer Learned routing delivers great performance but suffers from router collapse in some layers....