Data-Native AI Agents: Why Agents Must Move to Your Data

Data-Native AI Agents: Why Agents Must Move to Your Data | Databricks Blog Skip to main content

Summary

External agents break at scale: When AI agents run in a separate stack, enterprises face compounding penalties: fragmented governance, rising egress costs, sluggish multi-hop latency, and observability gaps that make production deployment risky.

Governance cannot be retrofitted: Post-hoc controls fail because agents compute over data rather than just retrieve it. A financial summary shaped by ungoverned rows cannot be redacted after the fact. Policy must be enforced at query planning time, and only data-native agents embed governance directly into computation.

Data-native agents on Databricks: By running agents within the Data Intelligence Platform, teams get Unity Catalog governance, AI Search retrieval, MLflow tracing, Lakebase state management, and AI Gateway traffic control as a single integrated stack, enabling them to ship trusted AI features faster with security and lineage built in.

Most enterprise AI pilots clear the same low bar: connect an LLM to your data, drop in a vector database, demo it to leadership. The hard part shows up later. Security flags the governance holes. Latency in multi-step agents kills the user experience. The bill from your model provider keeps climbing. These problems usually trace back to one decision: pulling data out of governed systems and into an AI stack that was never built to enforce your policies. This post argues for a different architectural direction: move models and agents to the data, not the other way around. Instead of building a parallel AI infrastructure and wiring it back to your lakehouse, you treat agents as native workloads that run inside your data platform, under the same governance, security, and observability controls you already trust for your data. The lakehouse gave you one place to govern your data. The next question is whether your agents live inside that boundary or outside it. There are two emerging paradigms. External agents Agents and LLMs run in a separate AI stack. Data is exported or queried over the network into external vector databases, SaaS LLMs, or bespoke serving layers. Governance, security, and observability are re-implemented for AI on the side. Data-native agents Agents, models, tools, retrieval, and agent memory run inside the same platform as the data itself, under a unified governance and security layer. AI becomes another workload on your existing data stack. The Hidden Tax of External Agents Data has gravity. Compute is cheap to relocate; data is not, especially as volumes grow and modalities multiply. Pulling it out introduces a familiar set of penalties: Governance weakens. Reimplementing access control, lineage, and residency in every integration leaves gaps that audits find. Latency stacks up. Each hop to an external vector store, LLM, and back compounds across multi-tool agents. Costs fragment. Egress, duplicate storage, and per-token pricing across vendors hit the budget from three directions. Lifecycle becomes vendor coordination. Schemas, indexes, and models drift across systems that don't share a deploy pipeline. Observability fragments. Tracing one request end-to-end means stitching logs from three or four tools. Business context stays behind. Metric definitions, business glossary, and domain groupings live in your governance layer; an external agent rebuilds them from column names, or guesses.

Why Post-Hoc Governance Fails for Agents Of all these penalties, governance deserves special attention because it's the one that can't be patched after the fact. Most approaches to AI governance treat it as a filter applied after the agent has already accessed data, e.g., redacting sensitive fields from the response, blocking certain topics at the output layer, and auditing logs after the fact. This works for simple Q&A demos. It breaks down the moment agents start computing on data. Consider an agent that calculates a financial summary across rows subject to row-level security. The aggregation itself (the sum, the average, the trend) is a derived value that is shaped by which rows are included. If governance isn't enforced before the query runs, the result already encodes data the user shouldn't have influenced. No amount of downstream redaction can undo that calculation. The policy decision needed to happen at query planning time, not at response rendering time. This is the fundamental gap in perimeter-based or post-hoc governance: it assumes data can be safely censored after it reaches the agent. In practice, once an aggregation or transformation has occurred, the governance intent is already lost. Retroactive controls are fundamentally incomplete. There's a second cost to getting this wrong, and it shows up on the bill. The trouble isn't only what the agent is allowed to touch, though that's part of it. When governance is resolved after the fact instead of at the source, the agent ends up doing the reconciling itself: traversing audit logs, joining fragments across external systems, pulling the same record from several places to work out whether it can use it, re-reasoning over every partial result. None of that is the task. It's the agent compensating for a governed answer that was never handed up front. Blocked or redacted outputs only feed the spiral, since the agent reads them as failures and tries again. Sessions stretch out, each hop loads more context into the model, and one request quietly turns into thousands of billed tokens. That's the token-burning loop, and post-hoc governance is what sets it spinning. Data-native agents address these challenges by embedding policy enforcement directly into query planning and computation. Every intermediate result reflects the same governance constraints. Governance must be evaluated before and during execution, which you can’t afford to apply as an afterthought after the computation is complete. Custom Guardrails in Unity AI Gateway are the concrete form of this: composable, deterministic policies the gateway enforces on every request and response, not filters the model is asked to obey after the fact. Deciding policy at planning time, against data that already lives in one governed place, also means the agent gets a clean answer in a single pass instead of spelunking across systems to assemble or justify one. Agent State and Memory need governance So far, we've focused on how...