Maestro Deep Research Agents

How AI21 Reached SOTA on Deep Research Benchmarks with Maestro | AI21

Skip to Main Menu

Skip to Main Content

Skip to Footer

Back to Blog

-->

Back to Blog

Every AI team eventually hits the same wall. The agent works. The demo impressed stakeholders. Then comes the hard question:

How do we make this good enough, fast enough, and economical enough to actually ship at scale?

This blog documents how we used AI21 Maestro , our agent optimization framework, to systematically navigate the quality–cost–latency tradeoff space of two challenging deep research benchmarks:

BrowseComp-Plus : Tests retrieval precision and synthesis across deep corpus search tasks

Deep Research Bench 1 : Tests long-form report generation and quality

On BrowseComp-Plus we achieved SOTA performance with 95.18% accuracy.

Before we explain how we did it, here is the final destination. The chart below is a live output from Maestro on BrowseComp-Plus — a full Pareto frontier automatically surfaced from a combination of model and tool configurations, scaling strategies, and execution policies. Every point on the red curve is an achievable operating point: pick your budget, read off your accuracy. The rest of this post explains how each technique contributes, and how Maestro automates the search across all of them.

The challenge of balancing accuracy, cost, and latency in production

Anyone who has deployed an LLM-powered agent knows that the accuracy, cost, and latency of each task are not independently optimizable; improving one almost always comes at the expense of at least one of the others. Call a more powerful model and you get better answers, but your cost-per-query triples. Add a verification loop and accuracy climbs, but latency balloons. Run multiple candidates in parallel and the best one wins, but you’ve multiplied your token spend. And when scaling AI features to large user bases, getting this tradeoff right is not optional; unit economics consistently thwart AI pilots from advancing to production ( RSM AI Survey, March 2026 ).

And yet reaching the right operating point across accuracy, cost, and latency requires searching a space that is, for practical purposes, composed of infinite permutations: model choice, prompt configuration, tool composition, agent harness design, scaling strategy, execution policies. The deeper problem with manual experimentation isn’t just that it’s slow and expensive – it’s that even when you find a configuration that works, you’ve done so by sampling only a handful of points from an enormous space. You don’t know what you missed. And when something changes – budget is narrowed or a new model is released – you have no principled way to know what to adjust or how the change will ripple through the other dimensions.

You’re back to trial and error, with no guarantee of convergence.

We built Maestro to systematically solve this search space optimization problem for any agent task. Below, we first share our experiments applying common agent optimization techniques today to BrowseComp-Plus. We then present Maestro’s solution, demonstrating the value of automatically searching across a compounded search space to identify the optimal execution path.

Optimization technique #1: Model and agent setup

The most natural starting point for any agent optimization effort is configuration: selecting the right LLM, adjusting prompts, picking tools, and tuning the agent harness. Different models carry different cost, latency, and success profiles; different prompt configurations shift which part of the problem space the agent solves well.

To make this concrete, we characterized the BrowseComp-Plus baseline landscape by running a set of representative model and tool variants and measuring their performance on the quality–cost and quality–latency axes.

Figure 1. BrowseComp-Plus Cost vs. Accuracy and Latency vs. Accuracy with individual model/tool variants – Sparse retrieval (BM25) – encodes text as high-dimensional vectors based on token occurrence. This method relies on exact lexical overlap, and is particularly effective when the query and relevant documents share vocabulary. – Dense retrieval (Qwen2-7b) – utilize a neural encoder to project text into low-dimensional continuous vectors that capture semantic meaning. This enables retrieval based on conceptual similarity, even in the absence of lexical overlap. – Late-interaction retrieval – using Reason-ModernColbert model which represents text as a set of contextualized token-level dense embeddings rather than a single vector. Retrieval is performed via fine-grained matching between query and document tokens, allowing for more precise alignment. – + – indicates Get_full_doc tool – Following the commonly used benchmark scaffold, some of the agents were equipped with a tool for retrieving a full document content.

Optimization technique #2: Scaling

When agent setup hits its ceiling, the next lever is scaling: generating multiple candidate solutions and selecting the best one at run time. Scaling comes in several forms, which can be combined and applied on top of any configuration choice.

2a. Single-variant scaling (best-of-N)

The simplest scaling strategy is to run the same agent configuration N times on the same query and select the best output, sometimes called best-of-N .

This technique exploits the fact that LLM agents are not deterministic. The same model, prompted with the same query, will sometimes succeed and sometimes fail. This run-to-run variance has been well documented in the scaling literature and it means that running multiple independent candidates and selecting the best one is a principled way to improve coverage: each rollout explores a different part of the solution space, and the probability of at least one succeeding grows with k. You can see this in the figure below:

Figure 2. BrowseComp-Plus Cost vs. Accuracy and Latency vs. Accuracy showing “oracle” scaling curves for variant@k – the probability that at least one of k rollouts is correct. Each variant was evaluated with different values of k. This represents the potential of best-of-N scaling, decoupled from run time candidate selection

But for this to be exploited in practice, you need a way to know which candidate is the best at run time. In a closed evaluation, you can check against ground truth (an oracle). In production, you need a runtime validation signal....