Last updated: September 23, 2026
An OCR project stalls because the accuracy number it reports was never the number the business needed. The way out is to stop tuning extraction, write the validation library the project skipped, and re-score the same documents against decisions your own people already made.
Fourteen months in, the project had produced one artifact anyone could point at. A dashboard.
It refreshed at six every morning. Extraction accuracy, 97.8%, green tile, a sparkline drifting up and to the right since March. Two analysts sat about forty feet from that screen, keying the same remittance advices they had keyed the year before, at roughly the same pace.
Nobody in the room could tell me what the 97.8% was measured against.
We found it eventually. It was character accuracy across every page the system had ingested, and most of those pages were cover sheets, terms boilerplate, and a two-line footer that repeated on every document from the largest payor. Clean type, high contrast, no tables. The tile was measuring the easiest paper in the building and reporting it as the health of the project. It was a real number. It described nothing.
The invoice numbers on the remittance detail, the part cash application actually runs on, were never measured separately. Not once in fourteen months.
This piece is about what to do when you are in that position, which is more common than the case studies suggest. I have written the general mechanic elsewhere, six steps from ingest to ground truth, in the document-to-decision method, and the staffed version of it is KORE1’s document-to-decision build. What follows is narrower. One step of those six, the one almost every stalled project left out, and how to restart from where you already are without writing off what you spent.
KORE1 publishes me as a guest and runs a data engineering and data science staffing practice, which places the people who end up building most of what follows. That’s the bias. I do not sell software. Roughly half of this argues you should spend less on it.
What Stalled Means Here
A stalled OCR project is one that reads pages accurately and changes no decision. Extraction runs on schedule, accuracy gets reported, and the work the project was funded to remove is still done by hand, at the same volume, beside it. Nothing is broken. That’s the difficulty.
Failed projects get killed. Stalled ones get renewed, because every individual component works when you look at it. That’s how they survive audits.
Three signs, and you usually have all three.
- Ask five people which decision the pipeline changed and you get five different answers, none of them a decision.
- The accuracy figure has no denominator that anyone can produce on request. Ours took two days to reconstruct and the answer embarrassed the people who had been reporting it, none of whom had done anything wrong.
- Headcount on the manual process is flat. Not reduced, not redeployed. Flat, and still carrying the four costs the project was approved to remove.
The third one settles it. If the keying continued, the pipeline was never trusted, and a pipeline nobody trusts is a reporting system with extra steps.
The Accuracy Number With No Denominator
Character accuracy is the number vendors quote, because it is the number that flatters. Here is what it turns into by the time it reaches a document a person has to act on.
Take a 99% character engine, which is a good one. The arithmetic below holds whether that engine is Tesseract or a frontier multimodal model, because it is a property of how fields are assembled rather than of how pages are read. Treat the errors as independent, which they are not, and it runs like this.
| What you measure | Example | Accuracy at 99% per character |
|---|---|---|
| One character | A single digit | 99.0% |
| One nine-character field | An invoice number | 91.4% |
| One twelve-character field | A bank account number | 88.6% |
| Every field on one document | Twenty fields, eight characters each | 20.0% |
One document in five arrives with every field right.
Four in five need review.
Real errors cluster rather than scatter, so your true number is better than 20%. It is never close to 99%, and the gap between those two figures is the entire reason your analysts kept checking everything. They worked out empirically what the tile was hiding, and they were correct to keep checking.
The research says the same thing in more careful language. Lin Sun and co-authors published a benchmark of OCR robustness for retrieval in April 2026 and measured one subset of it at 82.9% OCR accuracy and 52.8% on the downstream task it fed. A thirty-point gap between reading the page and getting the answer right. Their sentence for it is flatter than mine. “High OCR accuracy does not necessarily translate into strong downstream RAG performance.”
There is a structural version of the problem too. Jonathan Bourne, Mwiza Simbeye, and Joseph Nockels argued in an April 2026 paper on page-level OCR evaluation that the standard metric assumes the text was parsed correctly in the first place, and when parsing fails, character error rate “becomes undefined.” A table read as one long line of prose can score beautifully. The numbers are all there. They are in the wrong columns. The fields that fail this way are not random either, and on collateral documents I have written up the three that go wrong most often.
So the accuracy tile was not lying. It was answering a question nobody at the firm had asked. The denominator was the whole story.
Somebody Already Wrote the Standard, and It Is Free
The part that annoyed me on that engagement is that the rule the project skipped exists in writing, has existed since June 2023, and costs nothing to read.
The National Archives regulates how federal agencies digitize permanent records. 36 CFR 1236.56 says that when a digitization project is complete, the agency must validate that the digitized versions meet the standard, and that “separate staff must conduct the validation, independent from the staff that performed the digitization QC inspections.”
Separate staff. Independent. A federal regulation treats validation as a different activity from quality control, performed by different people, against criteria written in advance. Somebody thought hard about this.
Then it goes further. The section lists what has to be verified, including that “all required metadata are accurate, complete, and correctly labeled.” Only after that does it say the digitized records become permanent records. And an agency cannot dispose of the paper if the digitized versions fail the standard. The original stays in the box. Failure has a cost written next to it. Most projects have none.
Translate that out of regulatory language and you get the discipline every stalled project is missing. What correct means, written down before anyone scans anything, checked by somebody other than the person who built the scanner, with a defined consequence when it fails. A validation library also needs something agreed to check against, and that starts with a canonical record keyed at the right grain.
Volume alone does not rescue you either. The Treasury Inspector General reported in February 2026 that the IRS made limited progress on paperless processing, with contractors having scanned about 517,000 of the 9.8 million paper Forms 940, 941, and 1040 received during the 2025 filing season. Five percent. On the historical side the IRS declined to extend a scanning contract, and the report gives the reason as low volumes and quality of scans.
That is the largest document operation in the country, running under a federal mandate, against a written standard. It is hard there too. Your project did not stall because your team is worse than the IRS.
What a Validation Library Actually Contains
A validation library is the set of rules that decide whether an extracted field is trustworthy, written in the language of your business, before a model has seen a page. Not a confidence threshold. Not a spellcheck. A list of statements that must be true about a document for its data to be usable, each one testable by a machine and recognizable to the person who does the work.
It is a written artifact. Mine live as one file per document type, in a repository, and they run to about four pages each.
Six classes of rule cover most document types I have worked on. The examples below are from receivables and lending because that is where I spend my time, and they translate straight across to claim files, submissions, and scheme extracts. They translate to collateral reporting too, where the move that pays is verification rather than extraction on a borrowing base.
| Rule class | What it asserts | Example |
|---|---|---|
| Arithmetic tie-out | The parts add to the whole on the page | Remittance line items sum to the check amount |
| Referential | The value exists in a system you control | Every invoice number matches an open invoice |
| Format and domain | The value could exist at all | An advance rate stored as 0.85, never as 85 |
| Temporal | The dates make sense in sequence | Invoice date is not after the statement date |
| Cross-document | This file agrees with the last one | Opening balance equals prior closing balance |
| Population | The batch looks like batches normally look | Payor concentration inside its usual band |
Look at what none of those rules reference. The model. The vendor. The confidence score. Not one of them appears.
Every one of them is a statement about your business that stays true when you swap the engine, and that is the whole point of writing them first. Rules written after you have seen a model’s output encode what that model got wrong last quarter. Replace the model and the rules stop meaning anything. Rules written first outlive four generations of extraction technology, which at the current pace is about eighteen months.
The population rules are the ones teams skip and the ones I would keep if I could only keep one class. A single field being wrong costs you a correction. A batch quietly arriving from a payor who changed their remittance format last Tuesday costs you a month of cash application that has to be unwound, and no field-level rule will ever see it coming. Population rules do.
One practical note on authorship. The library gets written by the person who does the work, with someone beside them who can write precisely. An engineer alone produces rules that are elegant and wrong about the business. An analyst alone produces a list of exceptions rather than a set of assertions. Both, in a room. Two weeks per document type.
Why the Dashboard Came First
The first thing I build is never the dashboard.
Not because dashboards are useless. Because of what a dashboard requires and what it does not. It does not require anyone to agree on what a field means. It does not require a decision to be named, an owner to be found, or a definition to be settled between two departments that have been avoiding each other on this subject for years. A dashboard can be built entirely from what the machine already produces.
Which makes it the only deliverable available to a project that has not done the hard part yet, and it looks like progress in a steering committee deck. So it goes first, and then it goes on going first, and eventually it is month fourteen. Nobody chose that.
There is a second problem underneath. An accuracy tile fed by model confidence is a measurement of the model’s opinion of itself. It is sincere and it is uninformed. The model has a view about its own certainty and no view whatsoever about whether the page said what it thinks the page said.
What I build first instead is the smallest thing that changes a decision. On a policy maturity monitoring build, the manual process was a team checking obituaries by hand against a book of policies. We did not start with a view of the workload. We started with the notifier that read the sources and flagged the matches, because that was the output somebody could act on the same afternoon. Processing time went from nearly four weeks to three days. The reporting came later, and by then it had something worth reporting.
Build the thing that changes what happens on Tuesday. Measure it afterward. That order is the whole argument.
The Restart, in Six Weeks
This is the sequence I run when a project has already spent its budget and has to show something to a sponsor who is out of patience. It assumes you keep the extraction you already have. No new licenses.
1. Name One Decision
Not a process. A decision, made by a named person, on a schedule. Cash gets applied to this invoice. This draw gets funded. This file goes to underwriting. If you can’t name it in a sentence, the rest of the plan doesn’t apply yet, and finding that out in week one is the cheapest thing in this article.
2. Pick the Document That Decision Runs On
One type. The recurring one, not the interesting one. Pull the last two hundred of them and set them aside, including the ugly ones people normally exclude from a sample, because the ugly ones are the population you actually process.
3. Settle the Schema
Every field the decision needs, typed, with one agreed definition each. Two weeks. If a field takes more than an hour to define, you have found something real and it deserves the time. The full mechanic is in the six-step write-up, so I won’t repeat it here.
4. Write the Validation Library
The six rule classes above, against your schema, with the person who does the work in the room. Two weeks, and nobody touches a model during them. What you end up with is something a new hire can read on their second day, and something a vendor has to answer to.
5. Re-Score What You Already Built
Run your existing pipeline over the two hundred documents and grade the output against the library rather than against itself. Report three numbers: fields that passed, fields the library caught, and fields that were wrong and passed anyway. That third number is the only one that has ever mattered. Almost nobody reports it.
6. Replay Against Decisions People Already Made
Take the documents where you know what a person concluded at the time, run the pipeline end to end, and compare the two conclusions. Not the fields. The conclusions. This is the step that turns a demo into evidence, and it is the step a credit committee, an auditor, or a reinsurer will ask about first.
Six weeks, two people, no new software. At the end you either have a pipeline you can defend or a written explanation of exactly why the one you bought cannot get there, and both of those are worth more than another quarter of dashboard.
The Two Seats This Needs
Staffing is not a footnote on a restart. It is often the schedule. The IRS report above lists, among the reasons its scanning program ran behind, that contractors struggled to hire because workers could not wait the four to five weeks the agency took to grant background clearances. A document program held up by a hiring queue. That detail is more representative than the technology section of most vendor case studies.
The first seat already works for you. It is the analyst who has been keying the documents, and they are the only person who knows what the rules are, because they have been applying them from memory for years. Their calendar is the real constraint on a restart. If they can’t come off the queue for two weeks, backfill the queue rather than delay the library by a quarter, which is what credit analyst staffing is for.
The second seat is the engineer who turns the library into tests that run on arrival, builds the harness that scores the replay, and leaves behind something a colleague can maintain. The work sits closer to data engineering than to analytics, and the good ones ask what a field means before they ask what the schema is. Few firms have that person spare, and a six-week restart is a weak reason to open a permanent requisition, so the shape fits a contract engagement. KORE1 fills it through ETL developer staffing and through data engineering staff augmentation when the build runs a few months and the run state afterward is light. A technology search there averages about 17 days from kickoff to hire, which matters here only because the restart is already late.
If you are writing the requisition yourself, the vetting that separates candidates on exactly this kind of work is covered in KORE1’s data engineer job description template and in its guide to hiring an ETL developer. Ask about validation and lineage. The answers separate people quickly.
What the Steering Committee Asks Next
We Already Paid for the Platform. Does a Restart Mean Ripping It Out?
Usually no. The tool you already bought does ingest, classification, and extraction competently, and those are the parts you are keeping. What you are adding is the schema and the validation library it expected you to supply. Two conditions would change my answer. If the platform will not accept your rules, or will not give you a page reference for every field, then it cannot be defended to an auditor and the money is already spent either way.
How Is This Different From the Rules Our Vendor Configured During Implementation?
Ownership and order, mostly. Vendor rules get written after the model runs, live in the vendor’s configuration, and describe what that model tends to get wrong. Yours get written before, live in your repository, and describe what your business requires to be true. Swap vendors next year and one set survives.
Our Accuracy Is 99%. Why Does the Team Still Check Everything?
Because 99% of characters is roughly 91% of a nine-character invoice number, and your team learned that by experience long before anyone built a tile. Publish field-level accuracy on the fields the decision uses and the checking behavior will start to explain itself. Expect the first honest number to be uncomfortable.
Who Writes the Library, and How Long Does It Honestly Take?
Two people, two weeks, one document type. The analyst who does the work supplies the rules and settles disputes. Someone who writes precisely turns them into testable statements. Longer than two weeks usually means the document type was too broad or the decision was never named, and both are worth stopping for.
Can We Point a Large Language Model at the Documents and Skip This?
You can point one at them today and it will read most pages better than the engine you bought in 2021. It still won’t tell you which of its answers to trust. A model reports certainty about its own reasoning, never correspondence to the page, so the library is what turns a fluent reader into a defensible one. The model choice gets easier once the rules exist, not before.
What Do We Tell the Sponsor Who Approved the Original Business Case?
Show them the denominator. In my experience the conversation goes better than people expect, because the sponsor has usually suspected something was off and has had no way to name it. Bring the field-level number on the fields that matter, the six-week plan, and one decision the restart will change. Sponsors forgive a diagnosis. They stop funding a green tile.
Find the Denominator This Week
Open whatever accuracy report your project publishes and answer one question about it. Accuracy of what, over which pages, on which fields.
Time yourself. If the answer takes more than an afternoon to reconstruct, you’ve learned the thing this whole article is about, and it cost you an afternoon rather than another fourteen months. Write it on one page and take it to the sponsor before the next renewal.
I collect these, and the reconstructed denominators are usually more interesting than the accuracy figures. Connect with me on LinkedIn if yours surprises you. When the restart turns out to need an engineer your team doesn’t have, KORE1 can scope that seat.
