Context
Adit Abraham, co-founder and CEO of Reducto, explains why AI agents still fail on a basic prerequisite: reliably reading the documents where real work lives. For PMs building AI products, the episode is a reminder that model intelligence alone is not enough when the input layer is messy PDFs, scans, spreadsheets, charts, and forms. The discussion matters most in healthcare, legal, finance, and insurance, where a dropped word is not a minor OCR bug but a bad decision. Reducto's perspective is operational, not theoretical. The company processes billions of pages each month and had just crossed its seven billionth page on the morning of the recording.
The Big Idea
AI agents will not reliably do consequential knowledge work until the document layer becomes dependable. In high-stakes workflows, getting the input right matters just as much as making the model smarter.
Adit's argument is that document understanding is not a solved preprocessing step. It is a core product and infrastructure problem. Traditional OCR is too brittle for messy real-world files, and generic frontier models can still silently miss information unless they are wrapped in specialized parsing, validation, grounding, and review systems.
Key Insights
Documents block real agent work
Documents are not edge cases. They are where a large share of white-collar work actually lives.
- Adit frames documents as artifacts of human work.
- In healthcare, legal, finance, and insurance, people read from PDFs, spreadsheets, and scans, then write back into them.
- If agents are going to do that work, they need roughly the same fidelity a human had when reading the file.
- For a PM, this changes the product map. The bottleneck may not be model reasoning. It may be whether the agent can even ingest the source material reliably.
OCR solved only narrow cases
Older OCR and template systems worked when documents were predictable. Adit uses the example of forms like a W-2, where a system can rely on fixed page structure and pixel locations. That breaks once the document is less tidy.
- Real documents have two-column layouts, merged cells, multi-page tables, bad scans, charts, blueprints, and inconsistent formatting.
- The job is no longer just character recognition. The system has to recover structure and meaning.
- Adit says user expectations also changed in 2023. People started uploading documents to ChatGPT and expecting it to just work, while vision-language models made that expectation feel more plausible.
Frontier models fail on recall
Adit points to a Micro One benchmark on long extraction tasks. His takeaway is subtle but important.
- Frontier models often looked strong on precision. When they extracted a fact, it was often correct.
- Their more dangerous failure mode was recall. They silently dropped information.
- That matters because omissions are hard to catch in production. A wrong answer can be audited. A missing answer often looks clean and complete.
- For PMs, this is a warning against trusting polished outputs from a single long-context call without checking whether the model missed rows, fields, or sections.
Accuracy compounds across workflows
A document workflow is rarely one step. Adit describes end-to-end tasks like loan packets and healthcare workflows that include parsing, splitting, extracting fields, and sometimes editing forms at the end.
- If each step is only moderately accurate, the errors stack across pages and files.
- Adit says some hard healthcare packets are so messy that legacy OCR can be sub 50% in practice.
- He gives a concrete consequence: if a system reads chest pain instead of no chest pain, the downstream search or claim decision can change.
- He also cites a striking healthcare stat: 80% of appeals overturn the original decision, which he presents as evidence that current paper-heavy decision processes are already failing people.
Agents can audit extraction
One of the most interesting claims in the episode is not that models extract data well, but that agents can sometimes review and improve their own extraction.
- Reducto's Deep Extract harness does not rely on a single pass.
- The agent extracts information, reviews its own output, applies validation rules, and checks for inconsistencies.
- Adit says this is the first time he has seen agents outperform humans on many structured extraction tests.
- His reasoning is that human ground truth is often flawed. Humans drop rows, skip fields, and partially annotate documents, then those flawed labels get treated as truth.
Cost unlocked always-on parsing
Lower cost changes product behavior, not just margins.
- Adit says Reducto's list price is one cent per page for high-quality parsing and extraction.
- At that price, the goal is not to reserve accurate parsing for only a few critical files. The goal is for every page that a model needs to read to go through the document layer first.
- Reducto now processes billions of pages each month, which suggests this is moving from niche workflow tooling into general AI infrastructure.
- For PMs, this means some workflows that looked uneconomic a year ago may now be cheap enough to automate broadly.
Reading is not enough
Adit's roadmap goes beyond parsing. He wants agents to reliably work with documents, not just inspect them.
- He calls out document editing as the next interaction layer.
- His example is localization. Translating text is easy for modern models. Applying that translation correctly inside the original document is much harder.
- Reducto is building toward a headless version of tools like Microsoft Word so agents can edit files directly.
- The product implication is important: document infrastructure becomes more valuable when it sits inside the workflow, not just at the ingestion edge.
Focus beats general-purpose labs
Adit does not argue that big model labs are incapable of solving documents. He argues their incentives are elsewhere.
- Labs are racing on broad intelligence and general multimodality.
- Reducto can narrow the problem set and optimize aggressively for document-specific accuracy, latency, and cost.
- Adit's example is that a general vision encoder may need to handle everything from selfies to stylized image generation, while Reducto only needs to win on document tasks.
- That specialization is why he believes an independent company can stay ahead on this layer.
Mental Models & Frameworks
Documents encode human work
Treat documents as a primary interface to business operations, not as an inconvenient file format.
- A patient record, a loan packet, a contract redline, or a customs form is not just input data. It is a packaged unit of workflow, context, and decision history.
- Use this model when deciding whether document understanding belongs in core product architecture or as an afterthought. If the user's job starts from a file, document reliability is part of the product itself.
Pipelines beat one-shot calls
Adit's view is that document understanding is usually a pipeline problem, not a single model-call problem.
- Detect file and page complexity first.
- Decompose the document into workable units.
- Route hard regions, like charts or multi-page tables, through specialized handling.
- Validate the output and send the agent back when the result fails a check.
Use this model when someone proposes a single prompt over a long document. It may demo well, but it often fails on the messy cases that matter most in production.
Cite to the word
Trust improves when the system can point to the exact evidence behind an answer.
- Adit emphasizes citations not just at the document level, but at the word or region level.
- That helps humans review decisions faster.
- It also helps agents reconcile conflicting versions of a file, such as multiple contract drafts, because the system can compare specific evidence instead of vague summaries.
Use this model in any workflow where users need to trust a field extraction, especially when the result affects money, compliance, or care.
Ground truth can be wrong
Do not assume a human-labeled dataset is automatically correct.
- Adit says some human ground truth contains dropped information and partial annotations.
- That means eval quality depends not just on the model, but on the quality of the benchmark itself.
- Use this model when a new system appears to beat or miss the benchmark. Sometimes the system exposed flaws in the labels, not flaws in the model.
Complexity is layered
Adit describes document complexity as nested, not flat.
- Some files are simple and some are hard.
- Within a hard file, some pages are simple and some are hard.
- Within a hard page, some regions are simple and some are hard.
Use this model to avoid over-engineering every page the same way. Complexity-aware routing is one way to manage cost without giving up accuracy on the pages that truly need special handling.
Decision Principles
Principle: Benchmark on messy data
- When: choosing a document stack for a real workflow.
- Why: clean public benchmarks often hide the exact failure modes that matter in production, such as blur, tilt, long tables, duplicates, and contradictory versions.
Principle: Optimize workflow reliability
- When: a product spans parsing, splitting, extraction, editing, and downstream action.
- Why: page-level accuracy can look acceptable while end-to-end workflow reliability is still too low to automate a consequential decision.
Principle: Require grounded outputs
- When: users or agents will act on extracted fields instead of merely reading them.
- Why: citations, validation rules, and review loops reduce silent failure and give people confidence to trust the result.
Trade-offs & Nuance
Frontier models versus specialized stacks
Frontier models are not useless for document work. Adit explicitly says they can outperform many specialized parsers on precision. The trade-off is that they may drop information on long extraction tasks, and they are usually not optimized for the cost and latency profile needed at scale. A specialized stack adds more engineering complexity, but it gives tighter control over routing, validation, observability, and pricing.
Automation versus real consequences
Jennifer asks whether slow adoption in healthcare and other sectors is mostly cultural. Adit's answer is more nuanced. Some of it is trust and adoption speed, but a large part is that the consequences are different.
- If a developer uses AI to write buggy code, the team can often roll back the change.
- If a document workflow misreads a healthcare record or financial form, the error can affect care, approval, or payment.
- That means product teams in high-stakes domains need more guardrails than teams shipping low-risk internal tools.
Accuracy versus latency and cost
Adit says Reducto historically used traditional computer vision for structure and vision-language models as a correction loop because there were accuracy, cost, and latency trade-offs. Newer model work, including R1, tries to reduce those trade-offs, not eliminate the need to think about them. PMs still need to decide where a slower, more expensive review step is worth it and where a simpler pipeline is enough.
Common Mistakes
Mistake: Treating OCR as solved
This mistake usually comes from underestimating how much real-world documents vary.
- Teams assume that because OCR has existed for decades, document understanding is a commodity.
- They then discover multi-page tables, scans, handwriting, charts, and broken layouts only after the workflow fails in production.
- The better approach is to treat document understanding as a domain problem with real edge cases, not as a generic API checkbox.
Mistake: Trusting silent omissions
A clean-looking extraction can still be incomplete.
- Adit's warning about recall matters here. Missing rows, missing clauses, and missing fields are often more dangerous than obviously wrong values.
- The better approach is to build checks that test completeness, not just correctness of returned values.
Mistake: Believing benchmark marketing
Adit argues that many benchmarks have become marketing artifacts.
- A provider can look state of the art on a public set and still break when a document is tilted one degree or blurred slightly.
- The better approach is to evaluate on representative customer data and, when possible, use a thoughtful LLM-as-judge rubric to compare faithful outputs that may be formatted differently.
Practical Application
Map high-stakes document moments
List every point in your product where a document changes a decision, a recommendation, or a workflow handoff.
- Note the file type, who trusts the output, and what happens if one word or one row is missed.
- Prioritize the steps where a document error creates a real-world consequence, not just a support ticket.
- This helps separate low-risk automation from workflows that need stronger review and grounding.
Build a messy eval set
Do not start with your cleanest sample files.
- Collect documents with bad scans, multi-page tables, charts, duplicates, contradictory versions, and inconsistent formatting.
- Include the kinds of documents your team dreads today, not just the ones that already parse well.
- Use that set as the standard for vendor evaluation and internal quality reviews.
Measure recall on long tasks
Add a completeness score to every document eval.
- Track not only whether extracted values are right, but whether the system found every expected value.
- Pay special attention to long invoices, long packets, and dense tables, since Adit says those are where silent drops often appear.
- This prevents polished but incomplete outputs from looking production-ready.
Add source-linked review
Show the evidence behind each extracted field.
- Attach each answer to the page, region, or exact words that support it.
- Use those links in the review UI so a human can confirm or reject the result quickly.
- This is especially useful when multiple versions of a file disagree and a reviewer needs to see why the system chose one answer.
Pilot self-correcting extraction
Start with one narrow workflow and add a second-pass review loop.
- Have the first pass extract fields.
- Have a second pass check rules such as whether invoice line items sum to the total, whether fields are missing, or whether two files contradict each other.
- Measure whether the extra step cuts manual review time enough to justify the added latency.
Questions to Consider
- Where in your product are users assuming upload and it just works, even though the underlying documents include bad scans, missing structure, or contradictory versions?
- If your product automated a healthcare claim, loan packet, or compliance form tomorrow, which step would fail first: parsing the file, splitting the document, extracting the fields, or editing the final form?
- Are your document benchmarks based on customer-like distributions, including blur, tilt, charts, and long tables, or are they based on unusually clean sample files?
- Which decision in your roadmap is blocked less by model reasoning and more by low trust in the document input layer?
Bottom Line
The limiting factor for many AI products is not model intelligence. It is whether the system can reliably turn messy human documents into trustworthy machine input. A PM building agentic workflows should treat document understanding as core infrastructure, then evaluate it on real data, optimize for end-to-end reliability, and design for grounding before autonomy.
Resources Mentioned
| Resource | Type | Why it was mentioned |
|---|---|---|
| Reducto | Company website | The feed links to Reducto as the company building the document infrastructure discussed throughout the episode. |
| Micro One long extraction benchmark | Benchmark | Adit cites this benchmark because it highlighted a critical production failure mode: frontier models silently dropping information on long extraction tasks. |
| Adit Abraham on X | Social profile | The feed links to Adit's account for following his work on document processing infrastructure. |
| Jennifer Li on X | Social profile | The feed links to Jennifer Li, who hosts the discussion and frames the product and infrastructure questions. |
Concepts to Explore
Vision-language models
Vision-language models combine image understanding with text generation. In this episode, they matter because documents are not just text strings. They are visual layouts with columns, tables, charts, and spatial structure that older OCR systems often lose.
Precision and recall
Precision asks whether returned answers are correct. Recall asks whether the system found all the answers that were there. Adit's strongest warning is that a system can look good on precision while failing badly on recall, which is especially dangerous when the missing information is silent.
Intelligent document processing
Traditional intelligent document processing often meant OCR plus templates. The episode shows why that definition is too small for agentic products. Modern document processing also needs routing, validation, grounding, observability, and sometimes document editing.
Case Studies Mentioned
Healthcare packet search tool
Adit describes a healthcare customer that helps doctors search large patient record packets. These packets can be physical, stacked inches high, and hard to digitize accurately. He gives a concrete failure case where reading chest pain instead of no chest pain can change what a doctor sees and can affect claim or treatment decisions. The lesson for PMs is that document quality errors can become product trust failures, not just backend defects.
Hedge fund research pipelines
Adit says large hedge funds and private equity firms use document infrastructure to aggregate signals faster and more robustly. In that setting, the document pipeline is part of the firm's competitive edge, because faster access to more information can translate into alpha. The product lesson is that input quality can be strategy, not just operations.
Contract version reconciliation
Adit uses contract redlines as an example of why document work goes beyond extraction. Teams often have multiple versions of a term sheet or contract, and it is hard to know which file is latest unless someone labeled it clearly. Reducto's focus on citations down to the word helps agents compare conflicting versions and reason toward the likely truth.
Paper workflows still dominate
Adit gives two examples to show how much manual document work remains.
- 16% of tax filers in the United States still file by paper mail.
- Many Medicare prior authorization decisions are still wrong enough that 80% of appeals overturn the original outcome.
The lesson is that AI value in document-heavy workflows is not speculative. There is already a large stock of rote, paper-bound work waiting for better infrastructure.
Tools & Products
| Tool / Product | What it does | Why it was mentioned |
|---|---|---|
| Reducto | Document processing infrastructure for PDFs, spreadsheets, scans, and other file types | The core company discussed in the episode, positioned as the input layer for AI agents doing document-heavy work. |
| Reducto R1 | Reducto's specialized model for document tasks | Adit says R1 improved accuracy, speed, and cost relative to relying only on off-the-shelf models. |
| Deep Extract | Reducto's extraction harness | It lets agents extract, review, validate, and correct structured outputs instead of relying on a single pass. |
| ChatGPT | General AI assistant from OpenAI | Adit uses it to explain changed user expectations. People upload documents and assume the system should understand them. |
| Harvey | Legal AI product | Adit cites Harvey as an example of an AI-native company building serious workflow software on top of document-heavy inputs. |
| LibreOffice | Open source office suite | Adit mentions it to illustrate that even faithfully rendering office documents is still harder than many people assume. |
People to Follow
Adit Abraham
Adit Abraham is the co-founder and CEO of Reducto. He is worth following if you care about the infrastructure layer beneath AI agents, especially document parsing, extraction quality, grounding, and workflow reliability.
Jennifer Li
Jennifer Li is the a16z general partner who hosts this conversation. She is useful to follow for how she frames AI infrastructure questions from the perspective of product, adoption, and workflow design.
Experiments to Try
Experiment: Stress-test noisy documents
- Hypothesis: the current document benchmark overstates real-world quality because it uses files that are too clean.
- Change: take a sample of production documents and add light perturbations such as blur, small tilt, lower contrast, and imperfect scans.
- Test: compare your current document pipeline, a one-shot frontier model call, and any specialized parser you are considering.
- Measure: field recall, field precision, percentage of silent omissions, and manual review time per document.
Experiment: Add self-review extraction
- Hypothesis: a second-pass review loop will catch enough omissions and inconsistencies to reduce total human review.
- Change: after the first extraction pass, run a second agent that checks for missing rows, mismatched totals, contradictory values across files, and unsupported answers.
- Test: run the workflow on one narrow use case, such as invoices, intake packets, or prior authorization forms.
- Measure: corrected errors, remaining omissions, end-to-end latency, and minutes of human review saved.
Notable Quotes
"Nobody thinks about complexity when they upload a document to chat GBT. Right? Like, you just expect it to work." (Adit Abraham)
"It's the first time where I find that agents can be better at structured extraction than humans on many tests." (Adit Abraham)
"The best version of Reducto is one where you can kind of abstract away the PDF processing problems, even if the PDFs contain these like sub-intricacies along the way." (Adit Abraham)
