Introducing Baseten Hosted Tools

Introducing Baseten Hosted Tools Blaxel is joining Baseten . Read more

Product

Introducing Baseten Hosted Tools

Hosted tool execution on the inference backend augments model capabilities and lowers latency.

Authors

Sai Maddali

Marius Killinger

Marylise Tauzia

Last updated September 16, 2026

Share

TL;DR Today we’re introducing Baseten Hosted Tools, a new ecosystem for capabilities like web search, code execution in sandboxes, browser use and automation, and more to enhance models and agents running on Baseten. The first category of tools we are introducing is Web Search, and we are excited to launch it today with Exa, Keenable, Parallel, and You.com, bringing their search technologies into the Baseten platform with a consistent developer experience.

Until now, developers building apps and agents with open-source models had to assemble one tool at a time. While MCP provides unification to some degree, developers still need to handle the ReAct loop, task coordination, model steering, and eager or concurrent dispatch - all adding implementation complexity. As an application gains more capabilities, this integration work quickly becomes part of the product’s critical path. Baseten Hosted Tools makes that layer a simple config change, robust and low latency. Baseten Grounded Inference Our first Baseten Hosted Tool is web search: Baseten Grounded Inference . Baseten Grounded Inference gives models hosted on Baseten access to information that is as recent as the current day, specific, and absent from their training data. It unlocks product experiences that are difficult to build with a model alone: researching a company before a sales call, comparing current products and prices, investigating a developing event, finding recent technical documentation - generally grounding an answer in the entire knowledge base of the web. Web search runs inside the Baseten Inference Stack and can be enabled by adding the name of our hosted search tools as config to a standard Messages, Chat Completions, or Responses request.

The model decides whether a prompt needs web search based on its native tool use capability and may execute multiple iterations until it is confident. Via the system prompt and a configurable budget for iterations the behavior can be customized to the desired set point: ranging from one-shot to deep research agent. Baseten interleaves model inference and search requests in server-side a loop, while streaming tool-use and tool-result blocks as live progress to your client which you can use for progress display. Eventually the model produces a grounded response. A simple example of a grounded inference request looks like this: 1 curl https: //inference.baseten.co/v1/messages \ 2 -H "Authorization: Bearer $BASETEN_API_KEY" \ 3 -H "Content-Type: application/json" \ 4 -H "x-baseten-server-tools: true" \ 5 -d '{ 6 "model" : "zai-org/GLM-5.3-Fast" , 7 "max_tokens" : 1200 , 8 "messages" : [{ "role" : "user" , "content" : "What are latest news about Baseten?" }], 9 "tools" : [{ "type" : "baseten____web_search" }], 10 "baseten" : { "tool_settings" : { "max_react_iterations" : 5 }} 11 }' Tool selection and header are required.

baseten.tool_settings is optional.

Works for ChatCompletions , Messages and Responses in streaming and buffered mode.

must be replaced before the script can be run.

Launching with Exa, Keenable, Parallel and You.com Great tools require deep specialization. That’s why we’re building Baseten Hosted Tools as an ecosystem, starting with partners who have invested deeply in making the web useful to AI applications. At launch we offer access to 4 leading web search providers, listed below in alphabetical order in their own words: Exa gives your AI agent access to 100B+ websites, docs, papers, people, companies, news, and much more. From there, your agent can search the web in real time, read relevant pages, and answer with up-to-date sources. It's a token efficient way to search the web. Loved by 500k+ developers and companies. Exa powers search for Cursor, Vercel, Lovable, HubSpot, Clay, CodeRabbit, and more. Keenable provides independent web search infrastructure for AI labs and agents, powered by its own index of 100B+ web documents updated in real time to surface the latest content. It delivers state-of-the-art quality on agentic benchmarks. Latency is sub-250 ms p95 in US East, at $4 per 1,000 requests. Designed for high-volume low-latency inference, agentic workloads or reinforcement learning and data generation at scale. Parallel Web Systems provides web infrastructure for AI agents, powering agentic web research at companies like Notion, Harvey, Granola, Hex, and Dropbox. Its Search and Extract APIs are designed to match the intelligence and cost of open-source models, retrieving relevant web information and delivering it in a format models can use directly, so agents spend fewer tokens discovering, scraping, and processing pages. You.com delivers enterprise-grade web search APIs built for AI agents. Running on an independently crawled index, they serve 1B+ queries per month for global enterprises, frontier labs, and AI-native companies broadly. Their search APIs are built for both accuracy and speed. Highlights, their extraction mode, delivers citable, token-efficient context to downstream LLMs, keeping costs per task low, especially when paired with Baseten open-weight models. The same API scales from prototype to ZDR-compliant, high-QPS production without re-architecting your stack. Each partner brings a different approach, and making them available through Baseten lets developers choose the search experience that best fits their application without rebuilding the surrounding integration. Baseten Hosted Tools Baseten Hosted Tools gives developers access to production-ready capabilities from Baseten and our partner ecosystem. The benefits are: Keep agents fast: With Baseten Hosted Tools , tool calls sit directly in the agent loop. It is built for low latency, runs eagerly and concurrently, and delivers production-scale throughput.

Ship faster: Add capabilities without integrating and orchestrating every service independently.

Choose the best tool for the job: Access specialized providers through a consistent developer experience.

Operate everything together: Run models and the tools they call on the same production platform.

Integrated billing and usage reporting: like model API token usage, tool calls are differentiated...