AI Research Scientist Interview Questions 2026

Last updated: November 4, 2026

By Jennifer Burdick, Recruiting Manager, KORE1

AI research scientist interview questions in 2026 should test whether a candidate can tell a real result from noise, covering seed variance, fair baselines, benchmark contamination, and a written proposal that says when to stop. Publications get someone into the room. How they treat evidence decides whether they should stay in it.

A warehouse robotics company in Pittsburgh’s Strip District learned that the slow way. About 140 people, a fleet of picking arms in two customer distribution centers, and a grasp-planning model that dropped roughly one item in nine. They hired a research scientist at $214,000 after a strong job talk. Her new method beat the published baseline by 2.3 points on grasp success, the slides were clean, and the panel of three asked good questions about the architecture.

Nobody asked how many random seeds were behind the 2.3.

It was one. Four months later the team ran her method and the old baseline five times each on their own arms, and the gap disappeared inside the spread between runs of the baseline alone. She had not lied about anything. One seed was simply how her old lab reported results, and nobody in Pittsburgh knew to ask. The company lost a quarter of roadmap and most of a GPU budget chasing a number that had never been real.

I have been recruiting for nearly 15 years, 13 of them at KORE1, and research reqs now come through our AI research scientist staffing desk at a steady clip. The debrief I hear most after a bad research hire sounds like that one. The paper was fine. The interview never tested the habit underneath it.

On the money side, plainly. KORE1 earns its fee when you hire a scientist we found, so of course I hope your finalist comes from us. You do not need us to use anything below. Several clients have taken these questions and run the whole loop on their own referrals.

Some of this ground already has a home. Which of the three research tiers you are really hiring, how to read a publication record, and what each tier costs are all in our hiring guide for AI research scientists, along with the paper-critique exercise. Model-serving and retrieval questions sit in our LLM engineer interview questions, and production ML system design is in the ML engineer interview guide. This page is narrower. It is the questions you ask once you already believe the candidate can do research, to find out whether their results survive contact with your data. What to print in the posting before any of this starts is in our research scientist job posting template.

A Benchmark Gain Is a Claim Until Someone Checks It

An AI research scientist interview is a structured assessment of whether a candidate can design experiments, judge evidence, and choose problems worth months of compute. The strongest loops ask candidates to defend numbers from their own work, explain why a result might be wrong, and write a short research plan with a stopping point.

Why weigh this above the CV? The CV is rehearsed. A candidate with three first-author NeurIPS papers has cleared peer review three times, and that counts for something, but three sets of reviewers only ever saw the finished story. You get the unfinished one. Every week. For years, if it goes well.

The field has put its own standard in writing. The NeurIPS paper checklist asks authors whether they report error bars for the experiments behind their main claims, what source of variation those bars capture, and whether the bar is a standard deviation or a standard error. A research scientist should be able to answer those same questions about their own work, out loud, without notes. Most can. A few go quiet. Write that down.

The pool you are fishing in is small. The Bureau of Labor Statistics counted 37,200 computer and information research scientists employed across the entire United States in May 2025, at a median wage of $140,300 and a 90th percentile of $230,630, per its occupational wage profile. That category is broader than AI research and its pay runs well under what AI labs offer. Still. Thirty-seven thousand people, nationwide, every specialty lumped in. Lose the good ones to a loop that tests the wrong thing and there is no deep bench behind them. What the AI labs themselves pay is far higher, and 2025 H-1B filings for research scientists put Meta’s median base at $183,422.

Six Questions About Whether the Number Is Real

Ask each one about the candidate’s own work first. If they have nothing published or shareable, hand them a short results table from your own team and ask about that instead. Listen for the order they reason in. Do they question the measurement before they explain the result, or after?

Your method beats the baseline by 1.8 points on one seed. What do you run before you believe it?

More seeds, for both methods. A strong candidate says five as a floor and asks how expensive one run is before naming a higher number. They report a mean and a spread, and they say which spread, because standard deviation and standard error answer different questions.

The very strong ones bring up the literature on this without being asked. In reinforcement learning, Henderson and colleagues showed in 2018 that the same algorithm with the same hyperparameters, split into two groups of five random seeds, could produce average learning curves that looked like two different methods. Language models are not immune either. Dodge and colleagues fine-tuned BERT hundreds of times on four GLUE tasks, changing nothing but the seed, released all 2,100 trials, and found that weight initialization and data order contributed about equally to the swings.

Weak answers sound confident and short. “It is a consistent improvement.” Consistent across what?

The baseline numbers in your results table came from the original paper. Is that a fair comparison?

Usually not, and a good researcher knows exactly why. The new method was tuned by the person who invented it, on today’s hardware, with this month’s libraries, often with a bigger search over learning rates and warmup schedules. The baseline was tuned by someone else, years ago, under a smaller budget. Copy its number into your table and you are comparing a well-fed method to a starved one.

Push on the fix. Did they re-run the baseline themselves? With the same tuning budget, counted in trials or GPU hours, as their own method got? Strong candidates have a story here, usually involving a baseline that suddenly got much better once someone tuned it properly, and an uncomfortable meeting afterward. If they have never re-run a baseline, ask why. Sometimes the answer is honest and fine. Often it is not.

How would you know if the test set leaked into pretraining?

This one separates people who evaluate large models from people who only train small ones. The concern is simple. If benchmark questions, or close copies, sat in the training data, a high score measures memory, not skill.

Researchers at Scale AI built a fresh grade-school math set called GSM1k, written to match the style and difficulty of the widely used GSM8k, and reported accuracy drops of up to 8 percent for some model families when they moved to the new questions. The frontier models mostly held up. Several others did not.

Good answers name methods. N-gram overlap checks between training corpus and test set. Canary strings planted in a benchmark so a model that can complete them has clearly seen it. A held-out set written after the model’s training cutoff. Checking whether a model can reproduce benchmark items verbatim. The best answers also admit the limit, which is that with a closed model you often cannot inspect the training data at all, and then the fresh held-out set is the only honest option you have.

You have budget for one large training run. How do you decide how big it should be?

Scaling laws, fitted on small runs first. A candidate who has done this will talk about sweeping model size and token count at small scale, fitting a curve, and extrapolating with some humility about how far the curve can be trusted.

Most will cite DeepMind’s Chinchilla paper from 2022, which trained more than 400 models and concluded that parameters and training tokens should grow together. Chinchilla itself was 70 billion parameters trained on four times more data than the 280-billion-parameter Gopher, using the same compute budget, and it beat Gopher on a wide range of tasks. Fine. Now ask the follow-up, the one that separates reading from understanding. When would you deliberately train a smaller model for longer than that paper’s rule suggests?

The answer you want involves inference cost. A model that will serve millions of requests can be worth over-training, because you pay for training once and for inference forever. Candidates who get there on their own have usually sat in a meeting where someone from finance asked about the serving bill.

Why does attention divide the dot product by the square root of d_k?

A derivation question, on purpose. You are not checking whether they memorized the 2017 transformer paper. You are checking whether they can rebuild a design choice from first principles in about two minutes, which is something researchers do constantly and rarely get credit for.

The reasoning goes like this. If the components of a query and a key are independent with mean zero and variance one, their dot product has mean zero and variance d_k. As d_k grows, the dot products grow, and the softmax gets pushed toward a near one-hot output where gradients are tiny. Dividing by the square root of d_k brings the variance back to one. Vaswani and colleagues say nearly exactly this, in a footnote.

Watch what happens when they get stuck. A good candidate starts writing variances on the paper. A weaker one says “for numerical stability” and stops. Both may be good engineers. Only one of them is going to invent your next architecture change.

Your loss spikes at step 40,000 and recovers. Do you restart the run?

It depends on what caused it, and the candidate should want to find out before deciding. Was it a bad batch, a learning rate that was too high for that stage, an optimizer state problem, or a data shard that changed character? Strong answers mention looking at the specific examples in the batches right before the spike.

Ask whether they have heard how a large lab handled this. Google’s PaLM paper reported roughly 20 loss spikes while training its largest model, even with gradient clipping on. The fix the team settled on was not elegant. Restart from a checkpoint about 100 steps before the spike and skip roughly 200 to 500 data batches. Candidates who know that story tend to be practical about instability, which matters a great deal when every restart costs real money.

The Proposal Round, and the Stopping Rule Inside It

This is the round I push every research client to add, and the one most of them skip until a hire goes wrong.

Forty-eight hours before the onsite, send the candidate a one-paragraph problem from your actual roadmap. Strip anything confidential. They bring back a single page. On it they write the hypothesis, the cheapest experiment that could prove it wrong, a rough compute estimate, and one more line that almost nobody writes without being asked. What result, by what week, would make them stop?

That last line is the stopping rule. It is the most predictive thing I have seen in a research interview, and I have seen a lot of research interviews. People who write a specific one, with a number and a date, tend to run disciplined programs. People who write “if results are not promising” tend to run a direction for nine months because it is almost working.

A climate-risk analytics firm in Boulder, Colorado, added the round after a hire left them with two years of half-finished experiments. The next scientist they hired, at $198,000, wrote on her proposal page that if a new wildfire-spread model did not beat the existing gradient-boosted baseline on held-out 2023 fire-season data by the end of week five, she would stop and write up why. Week five came. It did not beat it. She stopped, wrote two pages, and moved to the second idea on her list, which worked. Their CTO estimated the stopping rule saved about $55,000 in rented GPU time and most of a quarter. He has never once asked her about the first idea since.

Grade the page in the room, with the candidate, for about an hour. Here is the rubric we give clients.

Two cautions. Pay candidates for the page if your process allows it, since you are asking for real thinking on a real problem. And never ship the idea from a rejected candidate’s proposal. It happens more than people admit, and the research community is small enough that it gets around.

A Few Questions Only the Applied Tier Has to Answer

If the seat is applied research, meaning published methods adapted to your product, add a short round that tests the trip from a notebook to production. Frontier researchers can skip it. Applied scientists live there.

• “Your offline metric went up 3 percent and the online A/B test came back flat. Where do you look first?” Good candidates go to the gap between the offline evaluation set and real traffic before they blame the test.
• Give them a latency ceiling, say 40 milliseconds on a CPU, and ask what they would give up first to hit it. Distillation, quantization, a smaller context, or a cheaper model in front of the expensive one. Any of those is fine. Having an order is the point.
• Ask who they hand the model to when they are done. If the answer is “the engineers,” ask what they write down for them. Silence here predicts a lot of Slack messages later.
• One quick one. When was the last time they recommended not using machine learning at all?

How the Day Fits Together

Most clients ask what order to run things in. This is the version we usually suggest for a senior hire, spread over one onsite day or two shorter virtual sessions. Keep the proposal round early, while everyone is fresh.

Long day. Researchers holding other offers will sit through it when every round feels like a real conversation about real work, and they bail quickly when two rounds ask the same thing. Our recruiters average more than 15 years in technical hiring, and the most common fix we make to a client’s research loop is cutting a duplicated round rather than adding a new one.

Things That Should Worry the Panel

None of these alone sinks a candidate. Two or three together usually do.

• Every result they describe is a success. Research does not look like that, so either they are hiding the failures or they never ran anything risky.
• They cannot tell you how many seeds sit behind the headline number in their best paper.
• A baseline that was never re-run, defended as “standard practice.”
• Asked how they would test their own idea cheaply, they describe the full-scale version.
• Their proposal has no stopping rule, and when you ask for one they say they would know it when they saw it.
• They talk about compute as unlimited. At your company, it is not, and the first budget conversation will be painful.

One reassuring note before the questions panels bring back. In our experience the people who handle these rounds well also tend to stay, and across our technical searches, 92 percent of placed hires are still on the client’s payroll at the one-year mark. A loop that tests judgment is not just better at picking. It is better at keeping.

Questions Panels Bring to the Debrief

Should a research scientist still get a coding round?

Yes, but make it research code, such as finding a planted bug in a short PyTorch training loop, rather than a timed algorithm puzzle that tests something they will never do at work.

Good planted bugs are the quiet ones. A missing zero_grad call. Dropout left on during evaluation. A data leak between the train and validation split. The candidate who finds the leak in five minutes is the candidate who will find it in your pipeline.

Can a candidate use an AI assistant to prepare the proposal page?

Allow it and say so in writing, because you will grade the reasoning in the room, and a candidate who cannot defend their own page live has told you what you needed to know.

Then ask how they used it. One candidate in Ann Arbor told a panel she had the assistant check her GPU-hour arithmetic and nothing else. Good answer. Honest, too.

Do we have to pay candidates for the take-home proposal?

You do not have to, but paying a few hundred dollars for a one-page proposal on a real company problem signals respect and noticeably reduces drop-off among candidates who already hold offers.

Check with your legal team about how to handle ownership of the ideas on the page. Then put that answer in the same email as the prompt, so nobody has to ask.

What if a candidate cannot discuss their current lab’s work because of an NDA?

Switch to their published papers or their PhD work, and lean harder on the proposal round, which uses your problem instead of theirs and avoids the NDA question entirely.

Constant with people leaving frontier labs. Not evasion. Respect it and the candidate notices, and so do the three friends they talk to afterward.

Is a research engineer interviewed the same way?

Mostly, with more weight on the code round and on scaling questions, and less on choosing problems, since a research engineer usually executes a research agenda rather than setting one.

Keep the seed and baseline questions word for word. A research engineer who shrugs at variance just ships noise faster.

Do the questions change for language model research versus vision or reinforcement learning?

The rigor questions stay the same across subfields, while the derivation question and the planted bug should come from the area the person will actually work in.

For a reinforcement learning hire, ask about reward hacking they have personally caught. For vision, ask about a dataset whose labels turned out to be wrong. For language models, the contamination question carries the most weight, and our AI and ML engineer staffing team can help calibrate the subfield questions if your panel has nobody from that area. Most of these seats fill as a direct hire, since research programs run for years.

Grade the Stopping Rule Before the Slides

Back in Pittsburgh, the robotics company now asks the seed question in every research interview, and the scientist from that first hire runs the round herself. She is still there, which surprised a few people. Her method did not work. Her habits, once someone asked about them, turned out to be fine. She simply had never been asked.

Write down what a good answer sounds like for each question before the first candidate arrives. Have someone on your team take the proposal prompt cold and time it. Then run the loop the same way for every finalist, so you are comparing people and not comparing days.

Want someone outside the building to pressure-test your loop first, or a slate of research scientists who have already handled rounds like these? Send the role to our research recruiters. On the first call we will say which tier the seat belongs to and which of the questions above will carry the most weight for it.