Structured Rag Enterprise Accuracy

AI21 Maestro’s accuracy fix for RAG’s blind spots | AI21

Skip to Main Menu

Skip to Main Content

Skip to Footer

Back to Blog

-->

Back to Blog

TL;DR

Traditional RAG falls short on complex enterprise queries, leading to incomplete or unreliable answers.

Structured RAG (S-RAG) transforms unstructured documents into structured, query-aware representation for precise, auditable reasoning.

AI21 Maestro’s hybrid architecture blends structured and embedder-based retrieval, improving accuracy by up to 60% with near-perfect recall.

Tailored for enterprise scale: automatically infers or lets users define schemas to handle millions of documents with full transparency and control.

Result: reliable, explainable answers you can trust in compliance, reporting, and mission-critical workflows.

We’ve spent years working with enterprises trying to turn their data into something usable. Every team had the same hope: RAG would finally unlock their knowledge for scaled use . But by the time we got involved, reality had already hit. The answers were fluent, yes, but they weren’t always reliable.

Let’s say you ask about performance trends or supplier comparisons, and the LLM-based system gives you only part of the picture. It might surface a few relevant excerpts but miss key data points, or summarize loosely connected information that doesn’t directly answer the question. In finance and compliance, those gaps aren’t harmless, they’re major risks.

We knew RAG wasn’t totally broken, but it needed help to become reliable enough for enterprise use. So we built Structured RAG (S-RAG), an enhancement that brings structure and reasoning to retrieval for precise, auditable answers. It’s built into AI21 Maestro, our AI orchestration system powering enterprise agents for mission-critical tasks.

Want to get more technical? Deep dive into our Structured RAG research paper .

Unlike traditional RAG, which retrieves text chunks and hopes the LLM answers correctly based on limited context, S-RAG extracts information offline from unstructured documents into a relational database. At runtime, it uses SQL queries to retrieve and reason over that structured data. This means analytical questions that require aggregation, comparison, or filtering — the kind enterprises rely on — can finally be answered accurately and transparently.

Example: When asked a simple financial question that most RAG systems fail to answer correctly, Maestro’s S-RAG retrieves the precise value directly from structured data rather than guessing based on text proximity.

Here’s what’s happening behind the scenes:

SELECT “current_liabilities” / 1000000 AS current_liabilities_millions FROM “SEC_Report” WHERE LOWER(“company_name”) = 'netflix' AND “fiscal_year” = 2017

Where embedder-based RAG falls short

Embedder-based RAG, the foundation of the first generation of knowledge agents, marked an important milestone in the evolution of retrieval-augmented generation. By embedding queries and documents into high-dimensional vectors, it aimed to retrieve semantically “close” text snippets and feed them to an LLM for reasoning.

But for enterprises, this approach falls short in three critical cases.

1. Answering aggregative questions

We’ve seen customers, especially in finance and compliance, hit walls with seemingly simple analytical queries:

“What was the maximum capital expenditure across all subsidiaries last year?”

“Who are the top five suppliers by on-time delivery rates?”

“How have safety incidents trended quarter over quarter?”

These are not “needle-in-a-haystack” lookups, where a few chunks of text hold the answer. To correctly answer these analytical queries, the system has to filter, compare, and aggregate data points across potentially dozens or hundreds of records.

Embedder-based RAG has no generalized way to perform these operations. It retrieves a predefined number of chunks of text and passes them to an LLM, which must attempt reasoning inside the limited context window using very limited arithmetic capabilities.

2. Questions that require exhaustive coverage

Enterprise teams often ask questions that demand complete, exhaustive lists:

“List all contracts expiring before 2025 with penalty clauses over $1M.”

“Which employees have certifications that will expire this year?”

“Show all markets where regulatory changes affect reporting requirements.”

Missing even one item is not an inconvenience, but a compliance risk. Yet embedder-based retrieval is inherently probabilistic and optimized to find the single best matching chunk rather than all relevant evidence. Because it only fetches a subset of documents based on similarity scoring, there can never be guarantees of full retrieval. This is unacceptable in domains like financial reporting or regulatory compliance, where exhaustive coverage is as important as correctness.

3. Dense or non-indicative corpora

Finally, consider corpora like technical documentation or regulatory filings. These are dense, repetitive, and attribute-driven. Documents may differ only by a few numbers or clauses, and queries may focus on precise attributes like clause numbers or model IDs. In these settings, embedding similarity collapses. Retrieval becomes noisy because embedder models haven’t learned the underlying semantics of those terms, so irrelevant but “close” passages often crowd out the real answer.

For example, in financial analysis queries, most reports look nearly identical and contain similar terminology — “total liabilities,” “shareholder equity,” “net income”. To an embedder, they all seem relevant. As a result, semantic retrieval often surfaces a stack of irrelevant documents that appear contextually close but don’t actually answer the question.

Enter Structured RAG.

Enhancing RAG with AI21 Maestro

AI21 Maestro solves these problems with Structured RAG, by expanding the range of queries RAG systems can accurately process in enterprise environments.

Instead of treating documents solely as unstructured text, AI21 Maestro leverages structure at ingestion to complement the drawbacks of embedder-based retrieval. It analyzes documents to detect recurring patterns. For example:

In financial filings , every report describes attributes like revenue, operating expenses, and capital expenditure.

In HR resumes , every record contains education, years of experience, and certifications.

In contracts , clauses follow standard sections like termination, penalties, and...