The Open Source AI Stack

The Open Source AI Stack Webflow Analyze/Optimize tracking bridge -->

🚀 DeepSeek V4 Pro 0813 vs. GPT-5.6 Sol on DeepSWE →

📈 GLM-5.3 vs. GLM 5.3 Flash on DeepSWE →

⚡ On-demand B200s now available on Together GPU Clusters →

🚀 Now serving MiniMax-M3 for efficient inference →

All blog posts

Inference

Published 9/9/2026

The Open Source AI Stack

Authors

Hassan El Mghari

Table of contents

40+ Models Chosen for Production...40+ Models Chosen for Production...40+ Models Chosen for Production...

As the quality of open source models have bridged the gap with closed source models, a lot of developers and organizations are looking to move to open source models for more ownership, control and economics. This post is a deep dive into the open model AI stack that developers need to consider as they move from closed to open source. Using open models for agentic software development does not require learning how to train models, buying a rack full of GPUs, or becoming an expert in machine learning. From the perspective of an application developer, the stack is surprisingly familiar to using closed-source models. You have a model that answers prompts and a harness that manages the interaction between you and that model. If you already know how to use Claude Code, you’re much closer to using open models than you probably think. The MIGHT Stack The stack can be broken down into these separate parts: Model: The component that interprets your request and decides what to do. Inference: The infrastructure & inference provider where the model actually runs. Gateways and routers: The layer that decides which model or provider handles each request, balancing cost, speed, and capability. Harness: The application that manages the conversation, gives the model access to tools, and connects it to your codebase. Tools (Skills and MCP): Knowledge that tells the model how to do specific tasks, including giving models and the harness access to relevant context

These layers are all independent from one another, which opens the door for technical decisions at each layer that better suit your development workflow. In turn, this allows you to experiment with new models as soon as they are released. New models appear constantly. Some are faster. Some are cheaper. Some are unusually good at a particular kind of work. If switching models takes a few minutes instead of rebuilding your workflow, you can actually experiment with them. In this post we will explore each layer of the stack, explain what it does, and look into how it can be customized. Let’s start with the model layer. Models A model takes your prompt and generates a response by predicting the most likely next tokens based on its training data. In our stack, it acts as the intelligence layer and it’s responsible for reasoning, decision-making, and deciding what changes to make in your codebase. Models come in a range of sizes, and in general, larger models tend to have greater capability, better reasoning, and more reliable performance on complex tasks. Most leading open models are now Mixture-of-Expert (MoE) models that contain many specialized “experts”, but only activate a small subset of them for each token generation. This allows them to have larger parameter counts but only activate a smaller of parameters and thus need less compute to run. Large models Large models are typically defined by the number of parameters they contain and the amount of compute used to train them. These models have an extraordinary capacity to recognize patterns, relationships, and abstractions from their training data. In practice, “large” usually also implies that the model was trained on more data, for longer, and with significantly more compute resources. This extra capacity translates into a few important benefits. Large models tend to be better at multi-step reasoning, where they need to keep track of several constraints at once and make decisions that depend on earlier parts of the problem. This makes them excellent choices when given an ambiguous or under-specified task since they can draw on a broader range of learned patterns to fill in missing details. A good example of a large open model is Kimi K3 , which has 1.8T total parameters & 104B active parameters. Reach for large models like Kimi K3 when you need to perform complex tasks, such as: Refactoring an existing authentication system Upgrading your codebase to a new framework Reviewing pull requests Understanding why an SQL database all of a sudden became slow

An advantage of large models is their robustness across different types of work. They can switch between writing code, explaining systems, debugging issues, and planning changes without needing tightly scoped instructions. This makes them especially useful in agent-style workflows where the model has to decide what to do next rather than simply follow a single instruction. Large models also make better use of longer conversations. When they are given many files, logs, or pieces of information at once, they are able to maintain coherence and connect relevant details across the entire input. This might lead you to believe that larger models are always better, but in practice there’s a tradeoff between large and small models. Next, we’ll look at some reasons to choose smalls model over larger ones. Small models The difference between small and large is less about quality and more about how much ambiguity they can comfortably handle. When a task is clearly defined and tightly scoped, small models can perform on par with much larger ones. If you remove ambiguity by being explicit about what you want, they become extremely effective. Small models excel at well-specified work since they don’t need to guess your architecture, infer hidden requirements, or explore multiple possible interpretations. They just execute the instruction as given. An example of a small open model is GLM 5.3 Flash which has 320B total parameters & 18B active parameters. To compare it to Kimi K3, it’s ~6 times smaller and ~20 times cheaper. These models are surprisingly good when the task is narrow and well-specified, for example: Update this function to accept another option. Write tests for this file. Explain a specific error. Review this 50-line function for bugs. Rename this API and update its callers.

There is not much ambiguity in these tasks. The model does not need to build a detailed understanding of your entire codebase or decide among several different...