Last updated: September 12, 2026
Document-to-decision is a six-step pipeline that turns a recurring document into a field a system can act on: ingest, canonical schema, validation library, extraction, uncertainty tags, and equivalence against ground truth. The order matters more than the model. Schema and validation come before extraction, which is why one mechanic runs a borrowing base certificate, a credit agreement, and a claim file without being rebuilt for each.
The certificate arrived on the fourth business day, the way it does every month. Eleven pages. Page four was the one that mattered, an aging schedule scanned from an Excel print, and beside a $1.4 million line somebody had written in the margin, in pen, per JS exclude.
The analyst read the margin. The pipeline did not.
Everything else came back clean. Advance rates, concentration caps, the dilution calculation, several hundred fields lifted off a scanned table, all correct. Eligible receivables were wrong by $1.4 million and availability moved with them, because nobody had ever written down what eligible meant in a form a machine could check. JS knew. JS had known for six years.
One person is the database, and everyone knows their name.
I have built this pipeline five times, on borrowing bases, on regulated medical files, on credit agreements, and on scheme extracts that showed up as a zip of forty-year-old fixed-width text. The six steps never changed. The schema did. That is the whole claim, and the rest of this is the mechanic in order, with the places it breaks marked.
One disclosure, since you are reading this on a staffing site. KORE1 runs data engineering and data science staffing and places the people who build most of what follows, and I publish here in partnership with them. I do not sell software. If you finish this and conclude it is two weeks of work for people you already employ, that is a normal outcome.
What Document-to-Decision Actually Means
Document-to-decision is the practice of treating a recurring document as an input to a decision rather than a file to be stored. The output is not a summary. It is a set of typed, validated fields, each carrying a confidence state, scored against what a person concluded from the same document. The rest is storage.
Three words in there do the work. Recurring, because a document you see once a year is a reading task and not an engineering problem. Typed, because a number that arrives as text has not been extracted, it has been photographed. Scored, because until you have compared the pipeline to a person on files you already closed, you have a demo.
Most of what gets sold as document AI stops one step early. It produces fields. It does not produce a defensible statement about whether those fields were right, and that is the part a credit committee, an auditor, or a reinsurer asks about first.
The Order Everyone Uses, and Why It Fails
Open any vendor page on intelligent document processing and you will find the same five stages: ingest, classify, extract, validate, integrate. Validation sits at position four. It is the last gate before the data goes somewhere.
Put it there and you will write your rules after you have seen the model’s answers. That is backwards, and it is expensive in a specific way. Your validation ends up encoding what the model happens to get wrong this quarter instead of what your business requires to be true. Swap the model and the rules stop meaning anything. Your rules should outlive the model.
The failure is quiet, which is what makes it bad. Pritesh Jha described the mechanism in an April 2026 paper on document processing pipelines, and the sentence worth keeping is that model confidence scores “measure inference certainty, not correspondence to the document, and extraction errors pass silently into downstream consumers.” A model is confident about its own reasoning. It has no opinion about the page.
So move validation up. Write the schema second and the validation library third, before a model has seen a single page. Extraction then becomes a fill task against a definition that already exists, and correctness is something you declared rather than something you inferred. Order is the lever.
The Six Steps
1. Ingest, and Stop There
Normalize whatever arrives and do nothing else. A born-digital PDF, a phone photograph of a fax, and a 1997 fixed-width extract all need different handling before they are comparable, and the temptation at this stage is to start pulling fields because pulling fields feels like progress.
Resist it. Ingest has one deliverable, a page-addressable record: every future field must be able to point back to a document, a page, and a region on that page. If you cannot click a number and land on the pixel it came from, nobody senior will ever sign off on the output, and they will be right not to.
Keep the original. Always. The processed copy is a derivative and derivatives get regenerated.
2. Write the Canonical Schema Before Anyone Opens a Model
One canonical record per borrower, policy, or claim. Every field gets a name, a type, a unit, an allowed range or value set, and a named source of authority, which is the document or the clause that decides it when two sources disagree. Name the authority.
The field list is the easy half and takes an afternoon. The hard half is agreeing, in writing, once, what eligible means. Or which EBITDA definition governs when the credit agreement has three. Or whether date of service or date of report drives the impairment. These are not data questions. They are business questions that have been answered informally for years by somebody who is very good at their job and has never been asked to write it down.
Put that person in the room. Not their manager. This is the step that fails when it gets delegated, and it fails silently, because a schema written by people who do not do the work looks completely reasonable on a page.
You will also need people who can turn that agreement into something that runs. Most firms find they are short in exactly one place, which is why data engineering staff augmentation tends to be the first outside spend on a project like this rather than a software purchase. The gap is people.
3. Build the Validation Library Next
Three tiers, and the order is deliberate. Syntactic rules check shape: a date parses, an amount is numeric, a tax identifier has the right form. Task rules check arithmetic that must hold: the aging buckets sum to the total, ineligibles reconcile to the detail, member benefits tie to the valuation within tolerance. Domain rules check the things only your business knows: an impairment cannot be rated without a supporting date and source page, a covenant cannot reference a defined term that is not defined. Domain rules are yours.
Arthur Hemmer and his co-authors formalized roughly this split in a 2025 paper on neurosymbolic extraction from transactional documents, layering syntactic, task, and domain validation over model output and reporting material gains in field-level accuracy. The finding is useful mostly because it is boring. Arithmetic constraints that your controller could have written on a napkin beat model tuning, reliably, and they keep working on documents the model has never seen.
Of everything on this list, the validation library is what survives longest. Models will change three times in a decade. The rule that says ineligibles must reconcile to the aging detail was true in 2015 and will be true in 2035.
4. Now Extract
By this point extraction is a cost question rather than an accuracy question, because correct has already been defined and is machine-checkable. Run the cheapest thing that clears the library. Escalate only the fields it fails.
Set expectations by field class, not by document. Header fields, which means the counterparty, the date, the total, the policy number, are close to solved and clear the high nineties on clean inputs across most current tooling. Line items, nested tables, and anything that continues across a page break remain genuinely hard, and they are also where the money usually is. Tables break models. Budget your review capacity accordingly.
The margin note from page four never gets solved at this step. It gets solved at step two, by a definition, or it does not get solved.
5. Tag the Uncertainty Instead of Hiding It
Every field leaves the pipeline in one of three states. Trusted, meaning it cleared the library. Needs review, meaning it parsed but failed a rule or landed outside a plausible range. Unreadable, meaning the source region was illegible and no value should be invented for it.
That third state is the one teams skip, because an empty cell looks like a rounding problem rather than a decision, and skipping it is how a blank quietly becomes a zero, and a zero on an eligibility line becomes availability somebody funds against.
Route the review queue by value at risk rather than by confidence score. A shaky field on a $900 invoice line and a shaky field on a $4 million concentration cap score identically to the model and nowhere near identically to you. Sort by exposure. Your reviewers will clear a fraction of the volume and cover most of the risk.
Regulators have landed in the same place. The NIST AI Risk Management Framework asks under its MEASURE function for performance assessment carrying explicit measures of uncertainty, plus formalized reporting and documentation of results. An uncertainty tag on every field is the cheapest way to satisfy that, and it happens to be the thing that makes the output usable internally.
6. Score Equivalence Against Ground Truth
Prove the mechanism on something that already happened.
Take twelve months of a document you have already processed by hand, run the pipeline against the originals, and score two things separately. Field-level agreement tells you whether the extraction works. Decision-level agreement tells you whether it matters, and the two diverge more often than anyone expects, in both directions. I have seen a pipeline miss nine percent of fields and reproduce every funding decision, and I have seen one at ninety-nine percent field accuracy flip an outcome because the field it missed was the only one with teeth.
Report decision-level agreement. That is the number a committee understands.
The one I carry from the underwriting build is 99.9% decision accuracy across more than 50 million pages of regulated medical files, measured this way, against decisions underwriters had already issued. Cycle time fell 33% in the same period. Neither number would have been defensible without the replay, and the replay is cheap. The files exist. The answers exist. Nobody has to forecast anything.
One Mechanic, Four Document Types
Here are the same six steps against four documents that look nothing alike. The mechanic is identical in every row. Only the definitions move.
| Document | What the schema has to settle | The validation rule that earns its keep | Ground truth you replay against |
|---|---|---|---|
| Borrowing base certificate | Eligibility in writing: aging buckets, cross-aging, concentration caps, contra accounts, what a credit memo does | Ineligibles reconcile to the aging detail, and the advance rate applied matches the one in the agreement | The last twelve certificates and the availability your team actually funded |
| Credit agreement | Which definitions govern, where the add-back list ends, reporting dates, cure rights | Every defined term used in a covenant exists in the definitions section and resolves to one meaning | Covenant compliance memos your credit team has already written |
| Life or annuity submission and medical file | One person per body of evidence, impairments mapped to a coding standard, date of service versus date of report | No impairment is rated without a supporting date and a source page reference | Decisions underwriters already issued on closed files |
| Legacy scheme or policy extract | One canonical policy record, benefit basis, effective versus issue date, currency and indexation | Member benefits sum to the valuation total within a stated tolerance | The prior valuation an actuary already signed |
Insurance solved this thirty years ago and called it an experience study. Credit is arriving at the same place with different vocabulary.
Where the Days Actually Go
Before you scope any of this, find out what the work costs you now. Not in dollars. In days, and in whose days.
- Getting the file at all. Chasing a borrower, a TPA, or a broker for a document that was due on the fourth and arrived on the ninth.
- Retyping. Somebody is keying numbers off a PDF into a spreadsheet right now, and that spreadsheet has their initials in the filename.
- Reconciling to the administrator, which is its own small industry inside your firm and has no owner.
- Waiting on JS.
- Version control. Which of the four files in that folder is the one that went to the committee?
Then look at the trend line, because the reason this became urgent is not that documents got harder. It is that there are more of them. The Secured Finance Network’s 2025 Market Sizing Study put secured finance volume up 34.5% since 2022 while outstanding levels rose 4.8%. Your balance sheet barely moved. The paper crossing it went up by a third, and headcount did not.
The same gap shows up at the regulator’s altitude. The Financial Stability Board’s May 2026 report on private credit vulnerabilities is direct about it: “Data gaps hinder effective oversight of the sector. Differences in definitions across jurisdictions and limited fund and loan level information make it hard to assess exposures.” Read that as a supervisory concern if you like. I read it as the same problem as page four, one level up. Nobody wrote the definitions down.
Who Actually Builds This
The first thing I build is never the dashboard. It is the schema and the library, and that work needs three people for roughly eight weeks, not a department.
One is the person who does the work today, part time, to settle definitions. One is a data engineer who can model the canonical record and stand up the pipeline. One reviews the queue and owns the exceptions once it runs, which is the role people underestimate, because exceptions are where the schema turns out to have been wrong and somebody has to notice that and say so out loud. On a lean team the third is often the first person wearing a second hat, which is fine and arguably better, because they wrote the definitions.
The engineer is where firms stall. Pipeline work of this shape is unglamorous, sits close to the business, and asks for someone comfortable reading a credit agreement, which narrows the field fast. The seat stays open. It is a contract-shaped problem more often than a headcount-shaped one, and I have watched firms lose a quarter to a requisition that was never going to close on the timeline the project needed. KORE1 runs about a 17-day average time to hire on technology roles and places into ETL developer and data engineering seats on that basis, which is the practical reason most of these builds start with an outside engineer rather than a requisition. If you are scoping that hire yourself, our guides to hiring an ETL developer and to writing the data engineer job description cover the vetting that separates candidates on exactly this kind of work, and the data engineer interview questions worth asking about validation and lineage.
A fractional head of data beats a nine-month search for a full-time one. That has been true on every one of these I have run, and it is more true when the scope is a single document type. If a full-time search is already open, our guide to hiring a director of data engineering covers what to screen for.
One related trap worth naming, because it eats the same budget. If your master data is dirty going in, the pipeline will faithfully extract clean fields onto the wrong entity, which is a separate problem with a separate fix. Colin Boothe has written the practical version of that one on using AI to clean the data your AI needs. Sequence matters: entity resolution first, then documents.
What Operators Ask Me First
Can a Model Really Read a Credit Agreement or a Borrowing Base Certificate?
Reliably enough to be useful, and not reliably enough to be unsupervised. Structured fields on a clean page clear the high nineties. Nested tables, continuation pages, and anything handwritten do not, and those are exactly where the exposure sits. Treat extraction as a first pass with a review queue attached, sized by value at risk rather than by page count. The pipeline’s job is to make a reviewer fast, not to remove them.
We Already Bought a Platform. Does This Replace It?
Usually not. Most platforms handle ingest, classification, and extraction competently, which is steps one and four, and leave you to supply the schema, the validation library, and the replay. The tool you already bought is a car with no road under it. Build the road and the tool generally starts earning. If the platform will not accept your validation rules or will not expose a page reference for every field, that is a real reason to look elsewhere, and it is the only reason I would give you.
Do We Need a Data Hire Before We Start?
No, and waiting for one is the most common way this stalls for a year. Steps one through three need the person who does the work and a few weeks of engineering. Nine-month searches for a head of data fail for the same reason the tool failed. Nobody has written down what the work is yet. Do one document type first. The definition of the role falls out of it, and you will interview much better.
How Long Until the First Thing Works?
Four to eight weeks for one document type, running in parallel with the manual process rather than replacing it. Weeks one and two settle the schema. Weeks three and four build the validation library and the first extraction pass. The rest is the replay and tuning the review queue. Anyone promising a working pipeline across your whole document estate in a quarter is quoting a demo.
How Do We Show an Auditor or a Reinsurer What the Model Did?
You show them four artifacts: the schema, the validation library, the uncertainty tags on the population, and the replay results against decisions a person already made. That package answers nearly every question an examiner, a rating agency, or a capacity provider asks. It is also, not coincidentally, the same package that convinces your own credit committee. Build it for yourself and the external version is free.
Which of Our Pilots Should We Kill?
Any pilot that cannot name the decision it changes, and any pilot with no financial owner. Those two questions retire most of a portfolio in an afternoon. A pilot that reads documents and produces a report nobody acts on is not early. It is finished, and it is doing what it will always do. Keep the ones where a named person’s Tuesday gets shorter.
Pick One Document and Write Its Schema This Week
Choose something you process every month and already understand. Start with one. Put the person who does the work in a room with somebody who can write, and try to define every field on one page.
If you cannot agree what a single field means inside an hour, stop and sit with that. You have just found the thing that was actually blocking you, and it was never the model.
I write about this work monthly in Operating Notes, and I am glad to compare notes on a specific document if you are mid-build. Connect with me on LinkedIn. If what you need is the engineer rather than the mechanic, talk to the KORE1 team about the seat.
