Skip to content
Tech Interview Prep
Technical interview guide

LLMOps: Deployment, Versioning & Cost Management

Safely shipping changes to prompts, models, and RAG configuration in production, and keeping the resulting system's cost under control.

Read
45 min
Practice MCQs
25
Interview Q&A
25
Edition
v4
Editorial status
Reviewed

Scope: SLSA 1.2, OCI Image Spec, OpenFeature specification, Google SRE release/canary guidance, FinOps Framework, Model Cards, Datasheets, PagedAttention, GPTQ, NIST GenAI, and OWASP Supply Chain references reviewed 2026-09-04.

Curated: · Written: · Reviewed:

25 questions

MCQ-12A support agent handles 500,000 sessions a month. Each session averages 1.6 model calls, retries included, at $0.01 per call. 80% of sessions are resolved by the agent; the rest are escalated to a human. Finance wants one cost figure for the unit-economics model, to set against the revenue each resolved session brings in. Which figure should the team report?
MCQ-13An assistant service handles 1,000,000 requests a month. Each request sends a 10,000-token prompt: an 8,000-token system-and-context prefix that hits the prompt cache on every request, plus 2,000 tokens of user question and history, and returns a 500-token reply. The provider bills $3.00 per million input tokens, $15.00 per million output tokens, and $0.30 per million cached input tokens. One request in five also runs an agent loop of two extra calls, each 2,000 input tokens and 200 output tokens, none of it cached. What is the monthly bill?
MCQ-15A log-triage service handles 400,000 requests a day. Each request sends 1,200 input tokens, a 900-token instruction prefix plus 300 tokens of retrieved log context, and returns 2,500 output tokens. The triage decision and its confidence are in the first 300 output tokens; the remaining ~2,200 are a free-text explanation no downstream system reads. Input is billed at $3 per million tokens and output at $15 per million. The instruction prefix is identical on every request and stable for hours; the provider bills cached input at 10% of the standard rate. Which single change cuts the daily bill the most?
MCQ-14A multi-tenant agent platform admits a request through an orchestrator, which runs a planner that fans out to 40 workers; each worker can spawn its own sub-planner, so the tree's depth and width depend on the task. Last week one request burned 1.2M tokens in 90 seconds: a tool returned a malformed response, a worker treated it as a new sub-task, and the tree kept growing. Every individual model call stayed under the 4,096-token max_tokens cap, and the tenant was well under its $2,000 monthly budget when the request was admitted. Which change would have prevented the runaway spend?
MCQ-21A support-triage agent makes one paid LLM call per ticket and writes the reply straight into a customer email that the same worker sends; no human reviews it. Each tenant gets a $50 daily budget, checked before every call and resetting at 00:00 UTC. At 14:20 a tenant's budget is exhausted with 40 tickets still queued. Failed jobs are retried by the queue with exponential backoff, and the ticket SLA is 24 hours. The tenant's eval set scores the pinned model at 94% and the cheapest catalog model at 61%. What should the worker do when the budget check fails?
MCQ-11A document-extraction service runs on Provider A: 200k-token context, server-side JSON-schema enforcement, 30-day retention in eu-west-1. Provider B is the fallback, triggered on 429 and 5xx: 32k-token context, JSON mode that guarantees syntactically valid JSON but does not enforce the requested schema, 30-day retention in us-east-1. Prompts are a system prompt plus retrieved chunks, averaging 18,000 tokens with a p99 of 41,000. In the first week, 4% of requests fail over to B; of those, 15% return HTTP 400 with no completion and 10% are rejected by the downstream parser. Which change removes both failure modes?
MCQ-18One A100 40 GB serves a 7B chat model. Profiling shows GPU utilization falling to about 30% toward the end of every static batch: median replies reach EOS after about 40 tokens, the 99th percentile after about 900, and each slot stays occupied until the longest sequence in its batch finishes. Peak KV cache occupancy is 55% of blocks, requests are queued waiting for slots, and the team wants more tokens per second on the same hardware. Which change removes the measured waste?
MCQ-17A team is consolidating inference onto one GPU pool that batches: requests arriving within a 2-second window run together in a single process that shares memory, and each request waits up to 2 seconds before it starts. They must move one stream of traffic onto this pool. Which one fits?
MCQ-20An inference API runs active-active in two regions. Steady-state traffic averages 250 req/s and the daily peak is 4× that. At peak, client retries add a further 200 req/s. A canary release mirrors 10% of peak traffic to the new model version, so both versions serve that traffic concurrently. Each region must absorb the full peak on its own if the other fails. What is the minimum per-region capacity that covers this load?
MCQ-19A team post-training-quantizes a ResNet-50 image classifier to INT8, calibrating activation ranges on 500 daytime photos. On a held-out daytime set, top-1 is 76.1% versus 76.4% in FP32. On a nighttime set, top-1 falls to 61.2% versus 74.0% in FP32. What is the most likely cause?
MCQ-3A service ships by rebuilding per environment. The Dockerfile starts with `FROM node:20-alpine`, a mutable tag. Monday's staging build produced image digest `sha256:1f3c…` and passed a 6-hour soak test. Thursday's production build, from the same git commit, produced digest `sha256:9ab7…` and crash-loops: the `sharp` native addon cannot load because the base image published Wednesday dropped the libvips shared library it links against. The team wants production to run the exact artifact staging validated. Which action makes production run the artifact staging validated?
MCQ-4A team trains a LoRA adapter on `Llama-3.1-8B-Instruct` at revision `8c1e` in bfloat16 (r=16, alpha=32, target modules q_proj and v_proj) and measures 81% on a held-out set. They deploy it on a serving stack that loads the same repo name at revision `f92a`, a later re-upload, also in bfloat16, and reads r, alpha and target modules from the adapter's own config. The load reports no missing or unexpected keys and no shape errors, the tokenizer files are byte-identical across the two revisions, and the base model alone scores 79% on the same held-out set. With the adapter attached, the score falls to 57%. Which check most likely explains the drop?
MCQ-22An experiment assigns users to prompt variant A or B by hashing `user_id`; each trace records the variant and a timestamp. Two weeks in, the serving stack is upgraded: the `gpt-4o-latest` alias is repointed from `gpt-4o-2024-05-13` to `gpt-4o-2024-08-06` over a 6-hour canary, and the retrieval index is rebuilt. Variant B's win rate falls from 61% (n=4,200) in week 1 to 52% (n=4,100) in week 2; A holds at 58% (n=4,150) both weeks. The team wants to separate the prompt effect from the stack effect using the data already collected. Which change to the trace record makes that possible?
MCQ-24A team is retiring the pinned model version `summarize-v2`. Telemetry shows 3 of 47 callers still send traffic to it, all through the `summarize-prod` alias. The gateway caches responses for 24 hours, keyed by (model version, prompt hash). Eval runs for v2 sit under a 12-month retention hold that expires in 5 months. What should the team do?
MCQ-16Acme's support assistant keeps one Redis cache shared by every tenant. The key is sha256(model_id + the user's normalized message), TTL 24 h, hit rate 61%. On Monday the platform team ships prompt template v9, which rewrites the refund-policy paragraph in the system prompt, and rebuilds the RAG index with new help-center articles; model_id is unchanged. For the rest of Monday, what do users get back?
MCQ-1An LLM support assistant's release row holds only git_sha, image_digest and model_revision; all three are unchanged across the last three deploys, yet three behaviors moved. A document deleted from the corpus 3 days ago is still cited: the retrieval sidecar loads index/latest.json through a CDN with a 7-day max-age and got a copy 6 days old. The get_invoice tool began returning a tax_jurisdiction field: the assistant fetches that service's /openapi.json on every call, and that service deploys about 40 times a day. The system prompt sits in a ConfigMap patched in place, and the app re-reads the mounted file every 6 hours, so two pods of the same release served different prompts for 4 hours. What must the release step do so that one recorded release can be redeployed to reproduce the behavior and any change traces to a single component?
MCQ-2An admission controller admits an image only when a cosign-verified SLSA provenance attestation shows the builder identity is `github.com/acme/payments/.github/workflows/release.yml@refs/heads/main`, the signing certificate's Fulcio identity matches that workflow, and the attestation's subject digest equals the digest being deployed. Last Tuesday an image passed all three checks and was deployed; on Thursday it was found to contain a backdoor. The attestation records git commit `4b1e7c9` on `main` as the source it was built from, and that commit's diff is what added the backdoor. Which control closes the gap this incident exposed?
MCQ-23A fraud-scoring service runs 40 pods across three clusters; each pod pulls model artifact `fraud-v7` from the registry at startup and caches it in memory for its lifetime. The service was last restarted two days ago and must keep serving through the next restart. This morning the registry shows the `fraud-v7` tag was moved six hours ago to a digest pushed from an IP outside the CI network, and that digest carries a valid signature because the pipeline's signing key was found in a leaked CI log. Two other services also pull `fraud-v7` at startup. A rolling restart of all 40 pods is scheduled in 20 minutes. The registry has no per-tag deny list; deleting a tag makes pulls fail closed. Which action should you take before the restart?
MCQ-7A checkout service runs a canary of a rewritten pricing engine on 10% of traffic. The router draws a variant independently for every request, so each request has a 10% chance of hitting the canary. A checkout session makes exactly 3 requests. What fraction of checkout sessions will be served by both the canary and the control?
MCQ-5A payments service handles 800 requests/s during its peak hour. A release candidate changes the idempotency-key path, and a latent bug in it double-charges roughly 1 in 50,000 requests. The team plans to skip the pre-release soak test and rely on a canary set to 5% of traffic for 20 minutes, run during peak. What is the probability the canary run triggers the bug at least once?
MCQ-8A team is canarying a new retrieval-augmented answer service at 5% of production traffic. The rollout policy has four numeric gates: p99 latency ≤ 60 ms, eval-set quality drop ≤ 1.0 point, cost per request ≤ +5%, server-error rate ≤ 0.1%, and a separate list of critical gates that are non-compensable: no gain on the numeric gates can offset one. After 30 minutes the canary reads: p99 58 ms; eval quality −0.4 points; cost +3%; server-error rate 0.08%. A tenant-isolation probe then confirms that a request from tenant A can read a cached response belonging to tenant B. Which observation should halt the canary immediately?
MCQ-25A team runs a customer-support triage model behind an API and is deciding whether to promote a new version from a 5% rollout to 100% of production traffic. Which of these is sufficient to justify the promotion?
MCQ-6A checkout service runs at 4,000 rps at peak. To validate a rewritten pricing engine before users see it, the team mirrors 100% of production checkout requests to the new engine, compares its prices against the live engine's, and discards the new engine's responses. The new engine publishes one `price_quoted` event per request to the same Kafka topic the live engine uses. A promo service consumes that topic and decrements a counter for each promo code; it has no deduplication and keys counters by promo code. The promo service currently handles 4,000 events/s at 30% CPU. What happens when the mirror is switched on?
MCQ-9A support agent runs multi-step workflows. Each step overwrites a checkpoint at Redis key `wf:{id}`; refund steps call the payments API with an idempotency key. Release v2 added a `tool_calls` field to the checkpoint and renamed `scratchpad` to `notes`; it also changed the refund idempotency key from `refund:{wf_id}` to `refund:{wf_id}:{step}`. v1's checkpoint parser requires the `scratchpad` field and rejects a checkpoint that lacks it. Forty minutes after v2 shipped, 340 workflows are mid-flight, 1,180 refunds have been issued, and the step-3 error rate has risen from 0.4% to 12%. The team rolls back to v1. Which plan is correct?
MCQ-10A payments service stores orders in Postgres. The team is migrating `orders.state` (`OPEN`/`CLOSED`) to `orders.status` (`pending`/`settled`) with expand-migrate-contract. They have added `status`, backfilled all 180M rows, and deployed code that writes both columns and reads `status`. The rollout is 60% complete: 24 of 40 pods run the new code, and 16 still run the old code, which writes only `state`. A user cancels an order, `OPEN` to `CLOSED`, and the request lands on one of the 16 old pods. Before the write the row held `state = 'OPEN'`, `status = 'pending'`. What does the row hold immediately after that write?