Aleph Alpha introduced Parsy, "our SOTA document parsing solution," a specialized model that scores 87.9 on ParseBench against 86.0 for the agentic harness that taught it. It parses hundreds of thousands of pages a day on a single H100, costs one to two orders of magnitude less per page than the harness, and now powers the extraction work in Aleph Alpha's customer solutions and the source highlighting in its chat applications. The blog post is by Alex Sheldrick, Lukas Tatge and Mohamed Yassine Elkhadiri.
Why it was built
Aleph Alpha's forward-deployed engineers kept tracing wrong answers in industrial and public-sector projects back to parsing. Datasheets, maintenance manuals, forms and test reports are full of nested tables, footnotes, charts and scans, and the parsers they tried recovered every word and number in a table but lost which value belonged to which unit and row, invented characters or attached footnotes to the wrong passage. Customers also needed documents to stay in their own environment, on-premises or in a private cloud, and no tool met both the quality and the sovereignty requirement.
From harness to model
The first version was an agentic harness that reconstructs each page from the extracted elements, each with a bounding box, then has a vision model compare the reconstruction with the original page. A judge model passes only pages with a high similarity score, re-extracts the rest and sends uncertain pages to an optional human reviewer. On ParseBench, a public benchmark of 2,078 human-verified pages from real enterprise documents, it scored 86.0 in Aleph Alpha's own evaluation, which would have ranked first on the public leaderboard until the end of August 2026. But running several models and up to three passes per page was too slow and costly for customers processing millions of pages a month.
So Aleph Alpha used the harness to label training data and fine-tuned its own model, with the goal of matching the harness in one pass:
- Data: public documents resembling customer material, collected in line with legal requirements, with no customer data. Labels pair each element with its category, bounding box and reading-order position.
- Cleaning: an independent layout model, an ink-coverage check for dropped text and a second OCR model flagged suspect labels, and people sampled each detector's flags before the failure type was fixed or removed across the corpus. One fix covered long lists and tables where several printed rows had been merged into one box.
- Result: the first version scored about two points below the harness on ParseBench, passed it in the sixth of twelve training rounds, rising from 84.0 to 87.9, and has stayed above it. Aleph Alpha says the later rounds gain little because the remaining errors are mostly formatting conventions. It also reports that running Parsy's output back through the harness's judge lowers its score.
The results
Parsy against the highest-scoring published systems on three benchmarks, from Aleph Alpha's blog post. Parsy scores marked by Aleph Alpha as measured with each benchmark's official code.
- ParseBench: 87.9 across all five categories, above LlamaParse Agentic Plus at a reported 86.6, then LlamaParse Agentic at 83.1, Pulse Ultra 2 at 81.6 and Upstage Document Parse 2 at 80.5. Parsy is not on the public leaderboard, which covers self-serve APIs.
- fr-bench-pdf2md (721 difficult French pages): 76.4, first, ahead of OCRia-1 at 72.1, LightOnOCR-3-4B at 71.4, Chandra-OCR-2 at 65.5 and Mistral OCR 4.1 at 54.1.
- olmOCR-bench (1,403 PDFs): 85.0, fourth and within three points of the best: OXR-1.0 at 87.8, Infinity-Parser2-Pro at 87.6, Chandra-OCR-2 at 85.8, with OCRia-1 at 84.6 just behind.
ParseBench overall among vision-language models, best configuration per model, from Aleph Alpha's post.
Most top ParseBench systems are pipelines that run several models and passes per page, while Parsy is a single model working in one pass. Among vision-language models the next best is Upstage's Document Parse 2 at 80.5, 7.4 points below Parsy, with Claude Opus 5.5 at 78.9, Gemini 3 Flash at 75.0 and GPT-5.6 Sol at 74.5.
Reproducing ParseBench
ParseBench re-run with the public scoring code on October 27 and 28, bars measured by Aleph Alpha and lines the public leaderboard, from its post.
Aleph Alpha says it found questionable ground truth on 145 ParseBench pages and shared an interactive viewer with 316 clear cases with the maintainers, who are preparing a corrected dataset version. It also re-ran Parsy, the three LlamaParse pipelines, Gemini 3 Flash, Nemotron-3.5-Parse and LightOnOCR-3-4B with the public scoring code. The open-weight models matched their leaderboard entries within about a point, but the LlamaParse pipelines first did not. LlamaIndex traced the gap to differences between its internal evaluation and the public pipeline and refreshed the scores the same day. LlamaParse Cost Effective and Agentic now match within half a point, while Agentic Plus remains about four points above Aleph Alpha's measurement (82.5 against the 86.6 on the leaderboard), mostly on formatting and charts, and the two are looking into it. In Aleph Alpha's runs Parsy leads all five categories: tables 92.3, charts 86.7, text content 91.1, text formatting 89.3 and visual grounding 80.3.
Beyond parsing
Because Parsy keeps a bounding box for every element, a chat answer can point to the exact region of the source page it came from, shown highlighted on the rendered page instead of as a document name and page number. A review interface shows the original page beside its reconstruction, with a score on every page. Throughput depends on page density, and boxes and structure make up about a third of the output.
Availability
Parsy is available through Aleph Alpha's customer engagements, as a capability its forward-deployed engineers bring to customer projects, with no public release. The benchmark figures are Aleph Alpha's own, scored with each benchmark's official code. It follows the company's open-weights Kolibri-1, and sits in a field that includes Mistral OCR 4.1.