Model Calibration Study: Underwriting Cases Against What Emerged
Three seasoned deals, the case each was approved on, and the one assumption that keeps missing in the same direction.
A model calibration study measures the gap between what you underwrote and what actually emerged, on your own seasoned deals. KORE1’s Calibration Study runs it on three of them and names the assumption that misses the same way every time. The method belongs to consultant Khurram Tehseen.
Last updated: October 4, 2026
Most credit committees can quote their loss rate. Ask which assumption in the last ten cases came in high, though, and the room goes quiet. Then somebody tells a story about one deal. Usually a vivid one.
One deal is a story. The same miss on three deals is a habit, and a habit can be corrected before the next memo reaches committee. The study is one piece of the data operations work KORE1 does for private credit managers, and the analysts who keep its correction running long after the memo is filed and the consultants have moved on come through our finance and accounting staffing practice.
How a Model Calibration Study Shows a Lean
Each row is one assumption from the underwriting case. Each vial is one deal, and the bubble sits off center by how far the case missed what emerged, on the side it missed. These three deals are invented to show the read. They aren’t a client’s.
- Revenue, first twelve months Mixed. Noise for now.
- Add-backs that became real EBITDA Three for three. A lean.
- Days to collect receivables Mixed. Noise for now.
- Capital spending, first year Mixed. Noise for now.
Look at the second row. Three borrowers, three sponsors, and every time the add-backs counted at close turned into less EBITDA than the case assumed, which is the kind of quiet, repeated optimism that never shows up when each deal is reviewed on its own.
No single deal proves anything. Three can point. A pattern like that is something a committee can act on.
The case on one side, what happened on the other, one line at a time.
Why a Calibration Study Starts With Three Seasoned Deals
Three is small on purpose. It’s enough to tell a habit apart from one bad borrower, and few enough that every page of every file gets read by a person instead of sampled.
It isn’t proof, either. Actuaries have a word for how much weight a set of data can carry, and Actuarial Standard of Practice No. 25 defines credibility as a measure of the predictive value given to a particular set of data. Three deals don’t carry much. They carry enough to tell you where to look next.
Seasoning counts for more than the number of deals. A deal that closed last spring hasn’t had time to show what emerged, so we pick deals that have been through enough reporting cycles for the case and the actuals to sit side by side. Young book? The study says so on page one.
Insurance has run this kind of check for decades and calls it an experience study. Khurram ran them at a global life insurer, which is the record behind our life and annuity data operations work. He holds AI to the same rule, which is why testing a model on your own credit agreements starts from deals that already closed.
The Numbers Behind a Model Calibration Study
- 3 Seasoned deals in every Calibration Study Each one scored case against outcome, assumption by assumption
- $30B Bank size the April 2026 model risk guidance mainly targets Federal Reserve SR 26-2, issued April 17, 2026
- 15 yrs Of supervisory experience the new guidance says it reflects SR 26-2 superseded SR 11-7, the 2011 letter
- 92% Of KORE1 hires still in the seat after twelve months Measured across every desk the firm recruits for
The guidance is written for banks, and a private credit manager usually isn’t one. Read it anyway. Its outcomes analysis section is the useful part. SR 26-2 describes comparing model outputs to real-world outcomes, watching for persistent deviations, and recalibrating when they show up, and none of that needs a bank charter, a model risk department or a new platform to do on a handful of your own deals.
Put “underwriting case” where it says “model” and you have this study.
Five Steps From Credit Memo to Correction
The order is the method. The case gets rebuilt from the memo and the model before anyone opens a single page of the actuals, which is what keeps hindsight from quietly scoring each deal in its own favor.
-
Step 1
Pick the deals
Three that have seasoned, ideally from different sponsors or sectors, chosen by age and type rather than by how they turned out.
-
Step 2
Rebuild the case
Each underwriting case as it stood at committee, assumption by assumption, from the memo and the model that went with it.
-
Step 3
Line up what emerged
Reporting since close, compliance certificates, borrowing bases and amendments, matched to the assumption each one tests.
-
Step 4
Score the lean
Size and direction of the miss on every assumption, deal by deal, then the rows that lean the same way three times.
-
Step 5
Write the correction
A short memo the committee can read in the meeting, plus one change to how the next case gets underwritten.
The memo is a few pages. Short enough to be read in the meeting it’s tabled in.
What the Calibration Study Hands Back
First, the memo. One table of assumptions against deals, the lean marked, and beside every number the document it came from, so anyone who disagrees can open the same file and check the number without having to take our word for it.
Then one correction. Usually a haircut or an extra check on the assumption that leans, applied to the next case, not a new model. Khurram’s rule is to start with a scorecard and save machine learning for the day your history can carry it. Three deals carry a scorecard.
Sometimes there’s a hole. One assumption can’t be tested at all, because nobody captured the field at close. A number that exists but was never checked gets an uncertainty tag instead of a guess, and both go on the list a Data Foundation Diagnostic usually picks up from.
Then somebody has to own the correction. It needs a name. That’s governance, and it can stay light, the way the piece on AI governance for mid-market companies describes it. The person rerunning the comparison as deals season is often a risk or compliance hire reporting to the CFO or the chief risk officer, which is where our compliance analyst staffing and risk analyst searches come in.
What a Calibration Study Needs From Your Team
-
Per deal
The memo and the model
The committee memo and underwriting model for each deal, as they stood on the day of approval.
-
Per deal
Reporting since close
Financials, compliance certificates and borrowing bases from every period since, in whatever form they arrived.
-
Per deal
Amendments and waivers
Every reset, waiver and amendment, since each one marks a moment where the case met reality.
-
Once
Someone from the room
A person who sat on the committee, for an hour or two, to say what the numbers meant then.
Common Questions
In plain terms, what does a model calibration study do?
It’s a check of your underwriting against reality, run on deals you’ve already closed. You take the case each deal was approved on, set it next to what happened, and look for misses that repeat. A model can rank borrowers well and still be calibrated badly, approving the right deals on numbers that run high every time.
Why only three deals?
Three is enough to separate a habit from one bad borrower, and small enough to read every page. It finds a likely lean, not a proven one. If the lean is real, the next three seasoned deals will show it too, which is why the comparison gets rerun as the book ages.
Do we need a statistical model before we can run one?
No. If your committee approves deals off a spreadsheet case and a memo, that case is what gets calibrated. Plenty of managers don’t have enough default history to train a statistical model at all, and a scorecard corrected by its own track record beats a model trained on too little.
How is this different from independent model validation?
Validation asks whether a model is sound and used as intended, and a calibration study asks only whether your cases came true. The Federal Reserve’s revised model risk guidance, SR 26-2, treats that comparison as one part of validation and calls it outcomes analysis. This study is that part, done on your own deals. Narrower, and faster.
How much time does it take from our people?
A couple of hours from one person who sat on the committee, plus help finding the files. Most of the elapsed time goes to locating the original cases and the reporting since close, which tend to be spread across a deal team drive, a portfolio monitoring system and somebody’s inbox. Once those are in one place, the scoring is the quick part.
What if our deals haven’t seasoned yet?
Then the study says so first, and that answer is worth having. It means you’re underwriting without a way to check yourself yet. The work shifts to saving today’s cases in a form that can be scored later, which is more of a data job than a credit one.
What happens once the study finds a lean?
Two things, a correction and an owner. The correction goes into the next underwriting case. That’s the easy half. The owner is a named person who reruns the comparison as more deals season, often a risk or compliance analyst, sometimes a hire KORE1 makes for exactly that seat.
Choose Three Deals and Check the Level
Tell us which deals have seasoned and we’ll tell you what the study needs to start. That’s the whole first conversation. No deck.
Prefer to talk to the person who runs it? Khurram Tehseen takes messages on LinkedIn.
