Rtl Pdf Parsing

Reaching SOTA RTL Parsing By Leveraging LTR Capabilities

Skip to Main Menu

Skip to Main Content

Skip to Footer

Back to Blog

-->

Back to Blog

A novel method for parsing PDF documents written in right-to-left (RTL) and underrepresented languages

Current parsing strategies break down when used on languages written from right-to-left (RTL), such as Hebrew and Arabic. We introduce a novel encoding technique to translate PDF documents from RTL formats into an easy-to-parse LTR format, significantly improving the overall parsing quality of RTL documents beyond the current SOTA.

The challenge: the parsing gap for underrepresented languages

In RAG architecture, document parsing serves as a foundational ingestion layer. If the parsing process fails to accurately capture the structural and semantic nuances of a document, it can introduce significant noise into the pipeline. This often creates a “garbage in, garbage out” challenge that is difficult to fully resolve through downstream optimization or sophisticated prompting.

As part of a systematic parser evaluation, we tested the effect of language on parsing quality using a synthetic dataset of documents containing text and tables. The results were stark: documents written in right-to-left (RTL) languages suffered a significant drop in parsing quality compared to LTR documents, and were plagued by formatting errors and flagrant hallucinations.

This wasn’t just an academic finding. We were building RAG systems that needed to support Hebrew and other RTL languages, and this parsing gap was seriously compromising our clients’ ability to work with their own data. Realizing that current parsers were inadequate for both our benchmarks and our customers’ needs, we set out to address this gap.

Parsing with GPT-4.1 resulted in numerous hallucinations in the text. For example, replacing “upper urinary tract (דרכי השתן העליונות)” with “lower urinary tract (דרכי שתן תחתונות)”.

Diving deeper into the problem

To solve this challenge, we developed a novel technique to convert these languages into a format that English-centric parsers can understand. By bridging this gap, we unlocked the full performance of existing tools for RTL languages.

Our first step was to create a robust Hebrew dataset modeled after OmniDocBench , a suite of metrics that is the academic and industry standard for parser evaluations. This ensured coverage of common enterprise formats such as complex tables, presentation slides, technical papers, and business reports.

We then ran this dataset through a process of automated parsing and human verification.

Pre-processing: A modified MonkeyOCR (a pipeline parser which uses VLM for both layout detection and OCR of the various sections) generated initial metadata for page elements, including type, reading order, and placement.

Annotation: Using a custom interface in our data annotation system, we utilized the pre-parsed data to rapidly tag content, type, and reading order.

Exploring solutions

Known strategies for parsing each have flaws. Training a new model from scratch is too data and compute-intensive; rule-based fixes can’t address erratic hallucinations and formatting bugs.

With the usual tools off the table, we had to get creative. We knew from the start that parsers excel with English and numerical data, so our strategy was to move the problem into a format the system already understood. We first tested this by representing words as numbers, but the parser either ignored the content or lost the document structure. This led us to the core of our experiment: Could we “trick” the parser into treating an RTL document as if it were written in English?

We tried several approaches to determine the optimal way to encode this “English” representation:

Direct translation: We first translated the RTL text directly into English. While this provided the parser with recognizable language, the differing word lengths and directionality caused severe formatting errors. Text frequently crossed table separators and broke the document’s original layout.

Length-based mapping: We then attempted to map each RTL word to an English word of an identical character count. However, because of variable character widths (e.g., “i” vs. “w”), the text still failed to align with the original document structure, leading to overlaps and layout instability.

Shape-based mapping (the solution): Finally, we refined the approach by selecting English words based on their visual geometry rather than just character length. By matching the physical “shape” and bounding box of the RTL text, we successfully preserved the document’s integrity.

This shape-based encoding proved highly effective. Our experiments reveal a key insight: graphical structure—both of the text and the document—matters more for parsing quality than semantic content . By prioritizing the layout, we successfully leveraged high-performance English parsing for RTL languages.

Cracking the code: using Word Shape Encoding to improve RTL parsing:

Having identified shape-based mapping as our most promising path, we implemented a solution using the following process:

Build a reference index: We took a large collection of English words, rendered each word and measured it in a standard font (“Arial”) and size (“10”). This allowed us to measure the precise width and height of each word and map those dimensions into a persistent index. This indexing phase is performed offline, creating a fixed “shape-to-word” index that serves as the foundation for the encoding process.

Encode the RTL document: We extract each word in the document (with the exception of English words and numbers), with its corresponding bounding box. Given the dimensions of the bounding box, we used a KNN algorithm to map each RTL word to an English word with a similar-sized rectangle, using the cosine similarity function to normalize different font sizes. The result was a one-to-one (reversible) mapping between each RTL word and an English word from the shape-to-word index.

Document rendering: Once the encoding is complete, we use the results to render an English-equivalent version of the original PDF. This involves two primary actions: text substitution and layout mirroring to convert the document from RTL to LTR.

Text substitution: We replace each RTL word with its English-mapped equivalent. While...