LLM Engineer Job Description Template 2026

Last updated: September 13, 2026

By Mike Carter, Managing Director, KORE1

An LLM engineer job description should say which layer of the model the hire owns (API integration, self-hosted serving, or the weights) and who runs the upgrade when a provider retires the model underneath your product. For the build seat most companies actually need, expect a base of $150,000 to $195,000 for a mid-level engineer, or $195,000 to $265,000 for a senior one. The retirement part sounds like an edge case. It stopped being one this year. Anthropic retired seven Claude models between January and August 2026, five of them on roughly two months’ notice.

On the morning of June 15, a freight audit software company in Long Beach stopped reading invoices.

Nothing crashed in the dramatic sense. Every call to the model came back as an error, and a queue of carrier invoices, the PDFs their customers pay them to check for overbilling, started piling up behind it. The model was Claude Opus 4. Anthropic had retired it that day, exactly on the date it announced in April.

Announced to whom, though? The notice went to the email address on the API account. That address belonged to the engineer who had built the extraction pipeline in 2025 and left the company in February.

The fix everyone expected took ten minutes. Anthropic lists Claude Opus 4.8 as the replacement, so the platform lead changed one model name in a config file and redeployed. Every request failed again, this time with a 400 error. Their code set temperature to zero, a habit a lot of teams picked up for extraction work, and Claude models from 4.7 onward reject any temperature other than the default. Finding that took an afternoon.

The five weeks went somewhere else. Was the new model reading invoices as well as the old one? Nobody knew. There was no set of invoices with known-correct line items to check against, so two people from operations compared 600 of them by hand while a pair of customer audits ran late. Somewhere in week three, the VP of engineering opened a req for an LLM engineer and sent me the draft. The second paragraph listed the stack. It said Claude Opus 4.

I laughed. Then I felt bad about laughing. It was an ordinary draft, the kind I read every week, and the ordinary part was the problem.

Some context on me. I’m Managing Director at KORE1, and a good part of my week goes to intake calls for our LLM engineer staffing practice, the calls where a hiring manager describes the job they really have before it gets formatted into the job on the page. We earn a fee when a client hires someone we introduced. Nearly everything below you can use without us.

Models Now Retire on a Two-Month Clock

Anthropic publishes its model lifecycle in the open, and its model deprecations page promises at least 60 days’ notice before a publicly released model is retired. Sixty days is the floor. Lately it has also been the typical window.

Here is every Claude retirement that took effect in 2026, taken from Anthropic’s own table.

Read the last row twice. The API name for Opus 4.1 carries the date August 5, 2025, and the model was retired on August 5, 2026. That model got one year.

None of this is a knock on Anthropic, which is more transparent about lifecycle than most. OpenAI keeps a public deprecations page of its own, with shutdown dates for older models. The dates can also shift depending on how you reach a model, since the same page notes that Amazon Bedrock and Google Cloud set their own retirement schedules for Claude. Bedrock’s model lifecycle policy says as much, and for models launched there from September 7, 2026 onward, most get a six-month legacy period before end of life. Some get 45 days. My point is a narrow one. If a hosted model sits underneath your product, a migration is coming, probably inside twelve months, and the job description should say who owns it.

Owning it is more than swapping a model name. In the searches we run, it comes down to a handful of habits, and teams skip them in roughly this order.

• Somebody reads the notices. It sounds too small to write down, and in Long Beach it would have bought two months.
• A regression set exists before the notice arrives. A few hundred real inputs with outputs someone has confirmed are correct, scored automatically, so a model swap produces a number instead of a meeting.
• The code around the model gets checked for anything the new version handles differently, from sampling parameters to prompt formatting to how it calls tools. The temperature change was one example. There will be others.
• Finance hears about per-token price changes before the first invoice on the new model.

The sentence I would add to the posting, right under the product description, goes something like this. “You’ll own model upgrades end to end, including the regression suite that tells us a new model is safe to ship. Our last forced migration took five weeks, and we’d like the next one to take five days.”

Oddly, that line recruits. Engineers who have run a migration read it and remember their own bad week, and they apply because somebody finally wrote it down. Engineers who haven’t tend to scroll past, which saves you a phone screen.

API, Serving, or Weights

“LLM engineer” covers at least three jobs that share a vocabulary and not a lot else. Our LLM engineer salary guide makes the pay version of that argument. The posting version is shorter. Ask how close to the model’s weights this person will actually work, then write the answer into the first third of the document.

Calling Hosted Models Through an API

This is most of the market. The engineer builds product on a hosted model, Claude or GPT or Gemini, and owns prompts, retrieval, tool calls, evaluation, cost, and the migrations described above. Plenty of them go the whole job without logging into a GPU box, and that’s no knock on anyone.

If your product serves more than one customer from a shared search index, give retrieval its own sentence. OWASP’s 2025 list of LLM application risks added vector and embedding weaknesses as a separate category, and one of the named examples is cross-context information leaks. In plain terms, one customer’s documents turning up inside another customer’s answer. Nobody wants that call. Put “permission-aware retrieval” in the must-haves and the candidates who have built it will notice the phrase immediately. When retrieval is most of the job, our guide to hiring RAG engineers covers that narrower search, and for evaluation status and tool permissions on this seat, the generative AI engineer job description we published goes further than I will here.

Running Open-Weight Models on Your Own GPUs

Different person entirely. Name the open-weight models you run, whether that’s Llama, Qwen, or OpenAI’s gpt-oss. Name the serving framework too. vLLM and SGLang are the usual answers, with NVIDIA’s TensorRT-LLM turning up in shops that squeeze every GPU. Then name the hardware, because an engineer who has actually tuned batch sizes across eight H100s and one who has read a great deal about it sound almost the same for the first twenty minutes of an interview.

Most of the week goes to throughput per GPU-hour. Some goes to quantization, FP8 or 4-bit, and then the tedious part after, proving the answers didn’t get worse. Long context windows bring their own KV cache trouble. And there’s always a Monday when traffic triples and p99 latency shows who planned capacity and who guessed. Clients are sometimes surprised that the best fit comes from our MLOps engineer staffing bench rather than from anyone with “LLM” in a job title. If that’s your seat, our job description built for the MLOps seat is worth reading alongside this one.

Changing the Weights

Rarer than postings suggest. Fine-tuning shows up in something like half the LLM drafts I read, and it gets used in a much smaller share of the actual jobs.

When it’s real, name the method, whether that’s LoRA or QLoRA adapters, supervised fine-tuning, or preference tuning with DPO. Name the tooling too (Hugging Face TRL, Axolotl, Unsloth), and name the data, especially who labels it and how much of it exists today. Then say how often training happens. A team that trains one adapter a quarter wants an application engineer who has done some fine-tuning. A team retraining every week wants a specialist, and our salary guide puts that specialist at $210,000 to $300,000 base, a separate conversation with finance.

Can one person cover two layers? Often, usually API work plus a light fine-tuning practice. All three in one person is a very small pool, and it’s priced like one.

If People in the EU Use the Product

A lot of American hiring managers skip this part. Skip it only if nobody in the EU touches what you build.

The EU AI Act‘s transparency duties under Article 50 took effect on schedule on August 2, 2026. They cover telling people when they’re dealing with an AI system and labeling AI-generated content. The heavier high-risk obligations slipped. The Digital Omnibus on AI, published in the EU’s Official Journal on July 24, 2026, pushed the deadline for standalone high-risk systems to December 2, 2027, per a Cloud Security Alliance research note that tracked the change.

Somebody has to build the disclosure and the labels into the product. At most of the companies we work with, that somebody is the LLM engineer, whether or not the posting mentions it. One sentence fixes that. “Our assistant serves customers in Germany and France, and this role implements AI disclosure and content labeling with our privacy counsel.” Engineers who’ve shipped under a privacy team read that as a sign of a serious shop, and engineers who haven’t will ask about it in the first interview, which is a perfectly good conversation to have.

Put the Pager in the Posting

Stanford’s 2026 AI Index put organizational AI adoption at 88% and counted 362 documented AI incidents, up from 233 in 2024. Its summary says responsible AI practice isn’t keeping pace with capability. I believe it.

Those incidents land on somebody’s phone. Usually an LLM engineer’s.

This spring a healthcare scheduling company in Scottsdale called us about a search, and the story behind it came out on the intake call. Their assistant booked appointments through a tool call. A retrieval change pushed on a Thursday afternoon started pulling a closed clinic location back into answers, and by Saturday the assistant had booked 31 patients into a building that had been empty since January. The assistant wasn’t on anyone’s rotation. Nobody had thought to put it on one, and the posting for the engineer who built it never mentioned on-call at all.

So write it down. Who carries the pager, how often, and roughly how many pages came in last quarter. You’ll lose a few applicants who wanted a job with no pager. They would have left the first time it went off anyway.

LLM Engineer Salary Bands for the Posting

Post a real range. A growing list of states requires one for covered employers, and senior LLM candidates skip postings without a number no matter where the company sits. These bands come from our 2026 salary guide, built from placement data and public salary sources.

Two warnings about the numbers you’ll find elsewhere. Some job-board salary pages average this title well below these bands, because plenty of listings put “LLM engineer” on a software role that added one API call. And the frontier-lab packages that circulate on social media fold in equity from a handful of companies you are almost certainly not competing with for this hire. Neither one is your market.

Serving and weights seats price toward the top of each band. For a figure matched to your metro and scope, the salary benchmark assistant will get you closer than any national table, this one included.

LLM Engineer Job Description Template

Brackets are decisions only your team can make. Anything in parentheses is a note to the hiring team and comes out before the posting goes live. If a section doesn’t apply, delete it instead of filling it with something vague, since vague reads worse than missing.

Title

[LLM Engineer / Senior LLM Engineer / LLM Engineer, Inference / LLM Engineer, Post-Training] (Add the layer after a comma when the seat is serving or weights. Job boards match on titles first, and the extra words save a round of screening.)

What We’re Building

[Company] makes [the product, in words a customer would use]. This role [the outcome, such as moving invoice extraction to a supported model and cutting cost per document by 30%, or taking our support assistant from 3 pilot customers to 120]. You’ll report to [role] and work most closely with [teams].

The Model Layer You’ll Own

• Layer: [API integration on hosted models / self-hosted serving of open-weight models / fine-tuning and post-training / a mix, with a rough percentage for each]
• Models in production today: [model names and versions], reached through [the provider’s API directly / a cloud platform such as Amazon Bedrock or Google Vertex AI / our own GPUs]
• Most recent forced model migration: [month and year], which took [N weeks]. Next announced retirement: [date, or none announced yet]
• Regression suite: [none yet, and you’ll build it / N labeled examples reviewed by hand / N examples scored automatically on every model or prompt change]
• If serving: [GPU type and count] running [vLLM / SGLang / TensorRT-LLM]

What You’ll Do

• Build and run [product features / the inference platform / the fine-tuning pipeline] on [models], from design through production monitoring
• Own model upgrades, which means tracking provider deprecation notices, running the regression suite, and shipping each migration well before the retirement date
• Keep the evaluation set current as the product changes, with [domain team] deciding what counts as a correct answer
• Hold cost per [request / document / conversation] under [$X] and p95 latency under [N milliseconds]
• [API seat] Build retrieval over [sources] that respects each user’s permissions, so no customer’s documents surface in another customer’s results
• [Serving seat] Tune throughput and latency, including quantization, batching, and capacity planning for [traffic pattern]
• [Weights seat] Run [LoRA / supervised fine-tuning / DPO] on [data], and show each new adapter beats the base model on the regression set before it ships
• [If EU users] Implement AI disclosure and content labeling with [privacy or legal team]
• Take a turn on the [assistant / inference] on-call rotation, [one week in N]

Must Have

• [3+] years writing production software in Python[, plus TypeScript or Go if your services use them], with tests and CI you’d defend
• At least one LLM system running for real users, and a specific account of something that broke after launch
• Experience measuring model quality against a labeled test set, beyond reading outputs and deciding they look fine
• [API seat] Retrieval with embeddings, a vector store, and access controls
• [Serving seat] Hands-on GPU inference work with at least one of vLLM, SGLang, or TensorRT-LLM
• [Weights seat] A fine-tuning run you can walk through from data preparation to the evaluation that justified shipping it

Nice to Have

• A model migration you led, forced or voluntary
• Tracing and feedback capture in [Langfuse, Arize Phoenix, Datadog, or whatever your team runs]
• Work under [HIPAA / SOC 2 / the EU AI Act]
• Familiarity with OWASP’s 2025 LLM risk categories, particularly prompt injection and vector and embedding weaknesses

On-Call, Location, and Pay

Work location: [remote within [time zones] / [N] office days per week at our [city] office / fully in office in [city]]. On-call: [rotation], averaging [N] pages a month last quarter. Pay: [$X] to [$Y] base, with [bonus target], [equity], and [benefits] on top. (A mid-level API seat runs $150,000 to $195,000 nationally. Serving and weights seats sit higher, so explain why in this paragraph before a candidate has to ask.)

What Engineering Leaders Ask Us Before Posting

Should the posting name a specific model, or stay model-agnostic?

Name it, version included, because a model name tells a candidate more about the job than any adjective, as long as someone updates the posting when the model changes.

The Long Beach draft is the cautionary version. It named a model that had been retired for weeks, and a strong candidate reads that as a team that doesn’t track its own stack. Fixable, though. Add one plain sentence saying you expect to change models over time, and nobody holds the current one against you.

How many years of LLM experience can we fairly ask for?

Three years is a fair ceiling, since ChatGPT only went public at the end of November 2022 and most companies didn’t put LLM features into production until 2023 or later.

Postings that ask for five or six years get two kinds of applicants. Researchers who worked on transformers before they were a product, and people who count a 2021 chatbot project very generously. Ask for years of production software instead. Then describe the LLM depth you need in terms of what they’ve shipped, not how long ago they started.

Our provider just announced a retirement date. Contractor for the migration, or a full-time hire?

Wrong order, slightly, because the migration has a fixed deadline and a permanent search may not finish inside a 60-day notice window, so a contractor usually handles this one while you hire for the next.

Count it out. A clean LLM search takes four to seven weeks once the posting is right. Add a notice period on the candidate’s side and the 60 days are gone. Bring in a contract LLM engineer to run the swap and build the regression set, write into the agreement that the set and the notes stay in your repository, and let the permanent hire inherit a process instead of a fire.

We only call hosted APIs and own no GPUs. Should GPU experience still go on the posting?

Usually not, since GPU and serving experience pulls applicants who want infrastructure work, and they tend to get restless on a team that will never run its own inference.

The exception is a roadmap with a date on it. If you plan to move a high-volume workload to open-weight models next year to cut cost, say so and list serving under nice to have. I watched that single sentence turn a routine API search into one with three finalists who had run vLLM in production. It raised the band, too. Plan for that.

Is an ML engineer posting close enough if we add a few LLM bullets?

Machine learning engineer postings pull people who train and deploy predictive models, and bolting LLM bullets onto one usually produces a document describing two jobs that attracts neither kind of candidate.

There’s real overlap, mostly on the weights seat. The week still looks different. Our breakdown of the LLM engineer vs ML engineer split walks through who owns what, and if the role turns out to be classical ML after all, start from the ML engineer job description template instead.

Once the posting is live, what slows an LLM search down?

Two weeks, sometimes three, disappear in the interview loop, because panels without LLM experience fall back on algorithm puzzles that say nothing about evaluation, retrieval, or migration judgment.

Our LLM engineer interview questions were written for exactly that panel. Our IT searches close in 17 days on average, counting every role type we fill. LLM searches run longer, closer to the four to seven weeks in our guide to hiring an LLM engineer, and in my experience most of that difference gets lost between the first interview and the offer.

Check the Retirement Date Before the Skills List

Before anyone polishes a bullet about passion for AI, open your provider’s deprecation page and find the model you run. Write down its retirement date, or the earliest date it could be retired. Then write down the name of the person who will handle that migration.

If the name is blank, you’ve found the first paragraph of your job description.

We fill these seats as direct hire searches, as contract-to-hire when the scope is still settling, and as short contracts when a retirement date is already on the calendar. If you’d like someone who reads LLM postings every week to look at yours before it goes up, send it to our team. We’ve filled technical seats out of Irvine since 2005, and twelve months after a KORE1 placement starts, 92% of those people are still in the job. Clear postings deserve some of the credit.