Skip to content
Tech Interview Prep

Top 100 MLOps Engineer Practice MCQs

Every question on one page: open one, choose an answer, and see why at once. Signed in, your latest answer to each question counts, and you can answer again after 24 hours.

Curated: · Written: · Reviewed:

Reviewed 100Review pending 0

Answers to reviewed questions count toward this role and the overall leaderboard; your latest answer to each one is the one that counts.

100 questions

MCQ-9A feature store builds training rows for a fraud model. For a training row whose decision timestamp is 2024-06-10T12:00:00Z, the account_balance feature table holds these rows: | event_time | balance | ingested_at | | --- | --- | --- | | 2024-06-01T09:00Z | 100 | 2024-06-01T09:04Z | | 2024-06-05T14:00Z | 250 | 2024-06-05T14:03Z | | 2024-06-09T20:00Z | 700 | 2024-06-11T08:00Z | | 2024-06-10T15:00Z | 50 | 2024-06-10T15:02Z | | 2024-06-12T10:00Z | 900 | 2024-06-12T10:01Z | Which balance should the training row carry?
MCQ-1A fraud model's offline holdout AUC is 0.91. In production it scores 0.62. The training pipeline computes `amount_30d` as the sum of a customer's transaction amounts in dollars. The serving pipeline reads the same transactions from a Kafka topic where amounts are in cents. A sample of 100 live customers shows served `amount_30d` averaging 4,210 while the training transformation recomputes 42.1 for the same customers; `tenure_days` and `region_id` match exactly. The model is scheduled for promotion tomorrow. What is the most likely cause and the first fix?
MCQ-17A fraud model reads `txn_amount_cents` (integer cents) from an upstream events table. On 3 March the upstream team changed that column to decimal dollars under the same name; the model's top feature is now 100x smaller, precision fell from 0.91 to 0.62, and the drop was found 11 days later by a manual audit. The upstream CI passed, and the model team's serving test, which asserts the column exists and is numeric, also passed. Which change would have caught this before it reached production?
MCQ-88A payments team pins every training extract by content hash in its feature store. In March an upstream job swapped the `amount` and `fee` columns in the extract. Both are positive dollar values in the same range, so row count, null rate and per-column ranges were unchanged; the hash changed and the pipeline recorded the new version. Training reproduced exactly from it, and the model's held-out AUC came out at 0.91, the same as February's, because the held-out split was drawn from the same March extract. Two weeks after deploy, the fraud model's alert volume had fallen 94%. Which change would have caught this before the model shipped?
MCQ-76A fraud model scores merchants on 'mean order value over the trailing 15 minutes'. Training features are rebuilt nightly by a batch job that reads the full order log and computes event-time 15-minute windows. Serving computes the same feature in a streaming job with event-time 15-minute windows and 60 s of allowed lateness; orders arriving more than 60 s after their window closes are dropped. A replay of one day (500,000 orders) shows 3% arrive more than 60 s late (the latest by 22 minutes), and those late orders average $80 while the on-time orders average $40. Which conclusion does the replay support?
MCQ-92A ranking model's offline AUC is 0.91, but live CTR fell 12% after the last retrain. The training pipeline computes a user's click count in pandas over a trailing 30-day window; the serving path computes the same named feature in Java over a trailing 7-day window. Both teams agree the model itself is fine. Which ownership arrangement keeps this class of failure from recurring?
MCQ-18A ranking model's online CTR fell 12% over nine days while the daily offline AUC evaluation stayed flat. Both the batch and streaming feature jobs report success on every run and neither has missed its freshness SLA. Sampling 5,000 entities shows the online `user_7d_click_rate` differs from the offline value for the same user and effective timestamp on 4,880 of them: the streaming job windows over event time, the batch job over processing time. Serving has a 40 ms p99 budget and the window spans seven days of events. Which safeguard should the team add?
MCQ-19A real-time card-fraud scorer reads `txn_count_1h` from Redis. An hourly Spark job overwrites the key and no expiry is set. During a 30-hour Spark outage the job wrote nothing, Redis kept serving the pre-outage count, and the incident review found fraud losses on the affected cohort at three times baseline. When the key is missing the scorer substitutes a zero and flags the prediction low-confidence. Offline replay of the last 90 days gives AUC by feature age at scoring time: | Feature age | AUC | | --- | --- | | 0-1h | 0.91 | | 1-3h | 0.90 | | 3-6h | 0.89 | | 6-12h | 0.84 | | >12h | 0.79 | Which change should ship to stop stale values being served?
MCQ-68A fraud-detection model's live precision falls from 0.71 to 0.52 at 03:00 UTC. The model artifact was last deployed Monday 18:00. At 02:50 the upstream job feeding the device_age_days feature began emitting 0 for every device; the feature monitor shows its mean dropped from 412 to 0 at that moment, and no other feature moved. Serving code is unchanged. What is the right diagnosis and first action?
MCQ-67A slot-ranking model has served 12,000 QPS with a stable 4.0% click-through rate for 40 days, and across that window the slot's CTR never left 3.8%-4.3%. The model binary has not been deployed for 26 days and its version hash in serving is unchanged. At 14:00 today the upstream job that computes the user_7d_click_rate feature finished a backfill that corrected a timezone bug, and the feature store keeps daily snapshots of every feature for 30 days. From 14:10 the population mean of user_7d_click_rate in the serving logs moves from 0.21 to 0.83, p99 scoring latency stays flat at 18 ms, and the slot's CTR falls to 1.9%. The on-call engineer is the platform engineer on duty, not the model's author or the pipeline's owner. What is the correct first action?
MCQ-69A feature summed refunds as positive revenue for 14 months; the bug is now fixed in the code that computes it. The model retrains weekly on a rolling 18-month window, and the feature's mean steps from 0.42 to 1.09 at the fix date. Which decision should the review accept?
MCQ-30A fraud model scores card transactions. Over 30 days, the PSI of the transaction-amount feature rose from 0.03 to 0.28, while precision on resolved chargebacks held at 0.71 against a 0.72 baseline. In the same window, the crypto-exchange merchant segment showed PSI 0.04 on every feature, but its precision fell from 0.68 to 0.41. What should the team conclude and do?
MCQ-87A daily feature job reads a warehouse table and writes about 4.1M rows into a feature store; the scheduler marks a run healthy whenever the process exits 0. After a source-table permission change the job kept exiting 0 for nine days while writing 0 rows. The feature store's TTL is 7 days, so from day 8 the model served the feature's default value and ranking NDCG@10 fell from 0.42 to 0.31. Which check would have surfaced the failure on day 1?
MCQ-89A team plans to change the definition of the feature `tenure_bucket` from 'months since signup, bucketed' to 'months since first payment, bucketed'. The feature registry lists one consumer, `churn_v2`. A lineage scan of 180 days of training logs shows three more readers: `ltv_v1`, `upsell_v3`, and `support_router_v2`. `ltv_v1` feeds a daily revenue forecast. The change ships in tomorrow's batch run. What is the most likely outcome?
MCQ-98A fraud-scoring service loads a model whose training signature records `merchant_category` as one of 38 known strings, `amount_usd` in [0.01, 25000], and `country` as ISO-3166 alpha-2. Two weeks ago the serving endpoint was changed to coerce instead of reject: unknown categories map to `__unknown__`, out-of-range amounts clip to the nearest bound, unknown countries map to `__unknown__`. Callers integrate directly against the HTTP endpoint. Since the change, 19% of caller B's requests carry an unknown `merchant_category` (its payloads append a trailing space), caller C sends amounts in cents so its $300 charges arrive as 30000 and clip to 25000, and the model's precision on the fraud label has fallen from 0.71 to 0.58. What is the correct remediation?
MCQ-7A fraud model retrains nightly: it runs the same SQL against the live `transactions` table, splits the result 80/20 at random into train and validation, and reports validation AUC. Tuesday's run (lr=0.01) read 1,204,338 rows and reported 0.883. Wednesday's run (lr=0.02) read 1,264,401 rows and reported 0.891; the only data change was a backfill of 60,000 late-arriving rows at 02:00 Wednesday. The team wants to know whether the learning-rate change is responsible for the gain. What should they do?
MCQ-16A nightly demand-forecasting pipeline validates its input table against a schema and a row-count range, then trains. After an upstream currency-conversion job changed, `amount_usd` came back null for 30.4% of rows against a 0.2% baseline. The run completed, the model imputed zeros, and the validation AUC came in 0.3 points higher than the previous run. Six weeks later a regional revenue anomaly appeared. Which change to the pipeline would have caught the fault on the night it happened?
MCQ-31A payments-fraud model is monitored on 40 features. Every hour a two-sample Kolmogorov–Smirnov test compares each feature's last hour against a fixed reference window, and the on-call is paged whenever any feature returns p < 0.05. Over a 30-day stretch with no known incident the channel paged constantly and the team muted it; four months later a genuine six-hour shift in one feature went unnoticed. The engineer replays the same 30 quiet days to size the problem. How many pages per day should the current rule produce on traffic with no real drift?
MCQ-44A daily-aggregate pipeline feeds the model. A timezone bug shifted every aggregate by one day and was found by a customer, not by CI. The team must choose an approach to testing the ML code before the model takes live traffic. Which one is defensible?
MCQ-45A team's model pipeline runs this on every pull request: lint, type checks, unit tests on the feature transforms, a smoke training run on a 2,000-row fixture, and contract tests against the serving interface. 6 minutes total. Merging to main trains on the full 40M-row dataset and promotes the artifact immediately; a nightly job trains and evaluates the same data and pages on-call if AUC drops below 0.75. Last week a PR changed the `income` imputation; the 2,000-row fixture has no nulls in `income`, so the PR build passed, the merged model scored AUC 0.63 against the previous 0.81, and it served for 14 hours. The team's rule is that the PR build stays under 10 minutes (a 70-minute PR build was disabled after two weeks), and a merge must be live within 15 minutes. Which change should the review accept?
MCQ-70An ML platform trains transformations in Python/Spark and serves them in Java. The serving path must stay pure JVM with p99 under 20 ms, and no transformation change may ship unless training and serving agree on every entity in a fixed 50k-entity regression set. A previous pair of implementations passed every unit test and still diverged on null handling for one field, affecting 3% of requests for seven months. Which policy fits?
MCQ-71A notebook reports AUC 0.874 on 812,000 rows. Its pipeline conversion (same feature code, same model class, same hyperparameters, same seed) reports AUC 0.831 on 1,204,000 rows. The pipeline reads the raw training table; the notebook reads a CSV that an earlier cell wrote to a local path outside version control, and the notebook's cells were last run out of order. Which action best establishes which of the two numbers to trust?
MCQ-72A ranking model's NDCG@10 fell from 0.42 to 0.36 after a retrain. The tracker stores each run's hyperparameters, final metric and start time. The feature store retains every snapshot for 12 months and the git history is intact. The team must explain the drop before the next release. Which action fits?
MCQ-73A team has 200 GPU-hours left before the model freeze. Their random-search log shows best validation AUC of 0.868 at 40 trials, 0.870 at 300 trials, and 0.871 at 900 trials; the 300-to-900 stretch consumed 200 GPU-hours. Labeling 5,000 additional examples costs 50 GPU-hours and has historically moved validation AUC by about 0.008 per 5,000 examples. What should the team do with the remaining 200 GPU-hours?
MCQ-12A fraud model must clear an offline gate of AUC ≥ 0.80 before taking live traffic. The gate's holdout is 2M transactions from Q1 2024. Last month's served traffic is 1.5M transactions, and 60% of it comes from merchants onboarded after 2024: a segment the holdout does not contain. Holdout AUC vs last-month served AUC: A 0.88 / 0.74, B 0.85 / 0.82, C 0.84 / 0.83. The team plans to ship A on the strength of its 0.88. Which action is correct?
MCQ-22A fraud-detection team retrains weekly and promotes a challenger when its offline AUC beats the champion's by more than 0.02. The challenger scores 0.847 on last week's labelled window; the champion's registry entry, written when it was promoted three months ago, reads 0.821. Before signing off, an engineer re-scores both models on the same settled window from last week: champion 0.858, challenger 0.847. What should the team do?
MCQ-13A payments team is deciding whether to promote a retrained fraud model. The promotion gate says no slice holding at least 1% of scored traffic may lose more than 3 points of recall against the incumbent. Holdout results: | Slice | Share of traffic | Incumbent recall | Candidate recall | | --- | --- | --- | --- | | all traffic | 100% | 0.61 | 0.65 | | card-present | 84.8% | 0.60 | 0.65 | | cross-border | 11.0% | 0.71 | 0.70 | | new-merchant (<30 days) | 4.2% | 0.58 | 0.51 | What should the review decide?
MCQ-24A ranking model change ran behind a 2% canary for 4 days on 400,000 requests a day, against a 10% baseline conversion rate. The team's convention is a two-sided 95% test at 80% power. The canary showed no significant regression, so the change went to 100%, and conversion then settled 0.2 points below baseline: about 800 conversions a day. Which change would have caught the regression before full rollout?
MCQ-28A team is comparing two ranking models on live traffic. The metric they will act on is per-user engagement measured over two weeks, and most users return on several days. The two models' offline NDCG scores are within 0.3 points of each other. Which assignment design measures the effect they care about?
MCQ-32A fraud-scoring challenger was ramped 5% → 50% on Thursday and to 100% on Friday. Its gates were error rate and p99 latency within 10% of the champion, plus aggregate precision within 0.4 points of the champion. Aggregate precision came out 0.900 for the champion and 0.897 for the challenger, so the gate passed. On Monday, scoring both models on the same traffic window, the new-account slice, 3% of scored requests, showed precision 0.82 for the champion and 0.72 for the challenger. The retrospective asks why the gate passed. Which conclusion does the evidence support?
MCQ-33A fraud-scoring service has run for eight months with an availability SLO of 99.9% and a p99 latency SLO of 120 ms. Both held: availability 99.95%, p99 96 ms. Meanwhile precision on the high-value-transaction slice, the slice the model was built to replace manual review on, fell from 0.70 at launch to 0.51, below the 0.62 the manual process achieved. No alert fired at any point. What should the team change?
MCQ-6A fraud team's nightly job trains a candidate and, whenever aggregate AUC on the held-out set improves at all, the same job writes the new version to the production alias using its own service account. Tuesday's run lifted aggregate AUC from 0.912 to 0.918, but recall on the card-not-present slice fell from 0.74 to 0.31; the model served for 9 days until a merchant reported missed fraud. Which change addresses the failure?
MCQ-29A payments team monitors a fraud model whose labels (confirmed fraud) settle at 45 days. The dashboard reports precision on the 2% of transactions labeled within 24 hours by customer report, and has read 0.91 for six months. Backfilling 45-day labels for the last quarter shows precision fell 0.88 → 0.79 → 0.71. The 24-hour-labeled set has a 9.1% fraud rate against 3.4% overall; a 7-day window covers 5% of volume at a 6.2% fraud rate. The proxy did fall to 0.86 in the week before the February settled drop. Which change should the review require?
MCQ-100A platform team runs 14 models. Promotion gates on offline AUC ≥ 0.80. Fraud labels are confirmed only after a 45-day dispute window. Last quarter a fraud model served a stale feature table for 11 days before a downstream report caught it. The team wants one measure, refreshed weekly, that would have flagged that failure within days. Which one?
MCQ-21A fraud model is retrained whenever any input feature's PSI exceeds 0.1. Last quarter the trigger fired 22 times; each retrain costs about $2,000 in compute and engineer time. A new signup flow pushed one feature's PSI to 0.31 and held it there, causing 19 of the 22 firings, and each of those 19 retrains produced a model within 0.002 AUC of the incumbent. In week 9, settled-label AUC fell from 0.842 to 0.791 and stayed there; no feature's PSI moved, so the trigger never fired, and the drop cost about $7,600 a week in missed fraud until the next review caught it. Measured on settled labels, each 0.01 of AUC below 0.842 costs about $1,500 a week. Which change should the review accept?
MCQ-39A fraud model scores card transactions. 8-bit quantisation cuts p99 latency from 45 ms to 22 ms and cost per 1,000 predictions from $0.40 to $0.22. Aggregate AUC moves 0.940 → 0.939, but recall on transactions over $5,000: 0.4% of traffic: falls from 0.71 to 0.44. What should the engineer do?
MCQ-96A payments team is deciding whether to replace its incumbent fraud model with a vendor's. The vendor's public benchmark reports AUC 0.94 and precision 0.91 at its recommended threshold of 0.50, on a benchmark whose fraud rate is 0.5%. On a held-out sample of 10,000 of the team's own transactions, where 4% are fraud, the vendor model at threshold 0.50 flags 900 transactions and 320 of them are fraud; the incumbent at its own threshold flags 700 and 350 of them are fraud. On that same held-out sample the vendor model's AUC is 0.92 and the incumbent's is 0.93. Which conclusion does the evidence support?
MCQ-91A card-fraud model was validated on existing-customer card traffic, where fraud runs at 0.4%. At the operating point that shipped, 0.62, it catches 75% of the fraud and flags 0.1% of legitimate transactions. A new-account team wants to run the same model at the same operating point over first-month account activity, where fraud runs at 0.1% and about 100,000 transactions are expected. What should the handoff to that team state?
MCQ-90A lending team retrains its loan-approval model. Offline AUC rises from 0.812 to 0.845. At the unchanged 0.50 cutoff the approval rate stays at 62.0% and the realized loss rate on approved loans is 3.1% before and after. The entire AUC gain comes from better ordering among applicants scoring below 0.05, a band the policy auto-approves regardless of score. What should the team do?
MCQ-57A lending model is up for promotion. The gate, agreed before the evaluation, is false-negative-rate parity: the 95% confidence intervals of the two protected groups' FNRs must not overlap. On the holdout set, group A (12,000 decisions) has FNR 0.09 and group B (800 decisions) has FNR 0.22; aggregate accuracy is 0.91 against a 0.90 bar. What should the promotion review do?
MCQ-15An ad platform bids p × $40 on each impression, where p is a propensity model's conversion probability and $40 is the value of a conversion. Over the last quarter AUC held at 0.86, unchanged. Last month's settled outcomes by decile: decile 8 predicted 0.62, observed 0.30; decile 9 predicted 0.78, observed 0.42; decile 10 predicted 0.91, observed 0.55. The team has one sprint before the budget review. Which action does the evidence justify?
MCQ-14A fraud model serves at threshold 0.80: 400 of 100,000 daily transactions go to manual review, catching 62% of the 1,000 frauds that occur each day. During an incident an on-call engineer edits the serving config to 0.45; the queue rises to 9,000 a day and recall to 81%. A review costs $3 in analyst time and a missed fraud costs $400 in chargebacks. What should the team do about the threshold?
MCQ-11A recommender is retrained weekly on the previous week's logged impressions and clicks. Offline AUC rises every cycle, but distinct items served per week has fallen from 12,400 to 900 over eight cycles. Which change should the team make before promoting the next model?
MCQ-80A risk model scores 200,000 decisions a day. Six reviewers clear about 720 cases a day between them, and a reviewed outcome has to be fed back within 24 hours to be useful. The pipeline sends a fixed 5% random sample (10,000 cases) to review, worked in arrival order. Last month the confident band (88% of volume) had a 0.4% error rate and the uncertain band (12%, score 0.4–0.6) had 9.1%. Reviewers spend most of the day on routine cases. Which change addresses the cause of the missed errors?
MCQ-81The fraud model scores every transaction; only those above 0.7 reach the review queue, and reviewers label what they see. A one-off audit of 500 randomly drawn transactions found 96 fraud cases, of which the model had flagged 46: 0.48 recall, against 0.91 measured on the queue. With a fixed reviewer budget, which labelling approach should the team adopt so the next promotion is judged on production recall?
MCQ-10A subscription service retrains a 60-day churn model on the 1st of each month, using every decision date up to the run date. Rows whose 60-day outcome window has not yet closed are written as not-churned. Decision dates in the final 60 days show a 1.8% churn rate; earlier dates show 5.4%. Offline AUC is 0.86 and has risen every month, while production under-predicts slow churn by roughly 3x. What should the team change?
MCQ-64A code-review assistant uses an LLM grader to score candidate review comments against a rubric. Comments the grader passes are auto-posted. The platform team samples 400 past grader decisions and has a senior reviewer label each one: 200 comments are acceptable and 200 are unacceptable. The grader passed 100 of the acceptable comments and passed 2 of the unacceptable comments. The team's release policy says auto-posting may continue only while at most 4 percent of auto-posted comments are unacceptable. What should the team do with auto-posting this week?
MCQ-65A RAG service answers questions over a 2M-document corpus. End-to-end quality on a frozen 500-query labelled set fell from 0.82 to 0.63 after an index rebuild that re-embedded the corpus with a new embedding model. Per-request logs show recall@10 on the same 500 queries fell from 0.88 to 0.51, while the generator's answer accuracy on queries where the gold document was in the top-10 held at 0.94 (was 0.95). What should the team do?
MCQ-63A release gate compares a candidate model against the incumbent on a held-out set, one generation per case, with the decoder at temperature 0.8. The candidate wins by 4.0 points and the team wants to ship on that number. Which justification for the evaluation protocol is sound?
MCQ-61A support-triage pipeline calls a hosted foundation model through the provider's `latest` alias. A nightly probe set of 2,000 labelled tickets across five categories has held at 98.9–99.4% agreement with baseline for months. On 14 March, billing agreement drops to 61.2% while the other four categories stay at or above 98.9%, and the billing routing error rate rises from 1.2% to 38%. The pipeline's own code and prompts are unchanged since a deploy six weeks earlier, and the provider's changelog lists a model update on 13 March. What should the on-call engineer do?
MCQ-62A support-triage service calls a hosted LLM with a system prompt held in a config service and edited in place; model weights ship as versioned artifacts with a release ID. A one-word edit to the system prompt for clarity moved the refusal rate on the support workflow from 2% to 19%. The offline eval suite, the same fixed 500-example set used for model releases, scored 0.91 before and 0.90 after, inside the team's 0.02 noise band, so the change shipped. Serving logs record the model release ID but not the template, and the cause took nine days to locate. Which change to the release process would have made that diagnosis take minutes?
MCQ-77A retrieval service keeps 8M chunks in a single HNSW index, all embedded with encoder v1 (768-dim, L2-normalised). A migration re-embeds chunks with encoder v2 (also 768-dim, L2-normalised) and upserts them into the same index over a 5-hour backfill; the query path switches to v2 when the backfill starts. Recall@10 against a fixed labelled set is 0.87 before the migration, 0.31 during the backfill, and 0.91 after it finishes. During the backfill, recall@10 on the 3M chunks already re-embedded is 0.90 and on the 5M chunks still on v1 is 0.04. No errors are logged. What explains the 0.31?
MCQ-79A two-stage cascade ranks feed items: a cheap filter scores every item and passes the top slice to an expensive ranker. On a held-out set the filter's recall against labeled relevant items is 0.72 (precision 0.94), the ranker's recall on the items it receives is 0.95, and the cascade's end-to-end recall is 0.68. Both per-stage dashboards are green. Which conclusion does the data support?
MCQ-74A team trains a 0.5B-parameter model on one A100 80 GB with per-worker batch 256 and no gradient accumulation. They move to synchronous data parallelism on 8 A100 80 GB workers, keep per-worker batch 192, and add 4 gradient accumulation steps. Training throughput rises from 1.0 to 3.1 times, but validation AUC falls from 0.81 to 0.75 at the same step count. They want the next run to compare distributed and single-node training at the same effective batch size before changing the learning rate or worker count. What per-worker batch should they set on the 8 workers while keeping the 4 accumulation steps?
MCQ-75A 7B-parameter model trains with AdamW in bf16 mixed precision, keeping fp32 master weights and fp32 optimizer moments, on a single 80 GB A100. At sequence length 2048 and batch size 4 it OOMs during the first backward pass at step 1. The team halves the batch to 2 and it still OOMs at step 1. Which accounting explains why the batch reduction did not help?
MCQ-2A credit-risk model has served for 9 months when an auditor asks the team to reproduce the exact training run behind it. The registry entry pins the git commit, dependency lock hash, container image digest, feature transform version, hyperparameters and random seeds, and stores the training data as literal query text, a SELECT over the events table bounded by a hard-coded cutoff date. Rerunning the pipeline from that entry today, twice, produces two artifacts identical to each other, and each disagrees with the deployed model on 1 in 30 scored accounts. The events table now holds 4 percent more rows inside the training window, merged from upstream backfills since training. The warehouse keeps 30 days of table history. Which change makes this run reproducible?
MCQ-3A model card claims 'byte-identical rebuilds'. Rebuilds run on a shared cluster that assigns the node, so the GPU count varies. Two rebuilds on the same 8-GPU node produce byte-identical weights. A rebuild on 4 GPUs produces state dicts that differ in the last bits and an AUC of 0.9134 against 0.9131 on the frozen 50,000-row eval set. The promotion gate asserts element-wise equality of the state dicts and fails the 4-GPU rebuild. What should the team do?
MCQ-4During a March incident review nobody could say which weights had served a given prediction: the prediction log held only the alias `champion`, which pointed at v31, v33 and v34 that month, and the registry's promotion history had been edited during a rollback so it no longer matched what actually served. The serving path exports each registry artifact to a quantized build before deployment. The team wants the next incident review to name the exact artifact that served, from the prediction log alone. Which logging change is defensible?
MCQ-5Your model registry is vendor-managed: its upload API accepts a version that already exists and replaces the stored bytes, and it keeps no copy of the replaced artifact. Deployment records store the version number and the deploy timestamp, nothing else. Two versions were overwritten in place: v9 (digest 3f2a → 8b1c on 2026-01-05) and v12 (digest a91c → e77d on 2026-02-03). Four deployment records exist: #4402 (2025-12-20, v9), #4471 (2026-01-14, v12), #4502 (2026-02-10, v12), #4510 (2026-02-12, v9). Which records can no longer be reproduced from the registry?
MCQ-8A text-classification service is being promoted from image digest 55b1 to 90ff. The model artifact digest stays a91c, and the offline evaluation AUC stays 0.912. A replay of 10,000 fixed production requests through 90ff produces 410 outputs that differ from 55b1. The team's release policy says a serving image is evaluated as part of the release identity. What should the release gate do?
MCQ-58Model `fraud-detect` is at v8 in the registry. A nightly job joins the metrics table on `model_name` and writes one `model_card.md` per model to a shared bucket; the card's intended-use field is filled with the training dataset's name. The v8 card shows precision 0.94, v7's figure, while v8's held-out precision is 0.81, and its intended use reads `fraud_v3_dataset`. A downstream team read the card and shipped a rule change. Which change fixes the cause?
MCQ-59A lender's credit model logs the model version and the final score for every decision, but not the feature values. Features are computed by a nightly job that reads warehouse tables holding only their current values. An auditor asks for six decisions from last quarter to be reproduced. The team reruns the pipeline with the same logged model version and serving image on today's tables: two scores match exactly, four differ, and two of the four cross the 0.05 approval threshold. Which reading of the rerun is correct?
MCQ-60A lending service scores applications with model v6 and writes each decision to an audit log together with the feature vector it used. The explanation service reads features from the live feature store and loads whatever model the registry's current pointer names. After v7 was promoted, replaying decision #48213 (scored by v6 with logged vector tenure=6, balance=14200, region=EU) returned top factors [region, balance]. A second replay, run after a feature-store backfill, returned [tenure, balance]. A customer complaint cites two different top reasons for the same decision. What should the team do?
MCQ-25A fraud model ships as one bundle (weights, feature transform, decision threshold, and serving image) versioned together. v19 goes live at 09:00. By 11:00, precision on the manual review queue has fallen from 0.71 to 0.44. The on-call rolls the weights pointer back to v18 and leaves the v19 transform, threshold, and image live. Precision stays at 0.44. The v19 transform changed `amount_z` from a global mean/std to a per-card mean/std, and the v18 weights were trained on the global scaling. What should the on-call do next?
MCQ-26A pricing service writes quotes into the order system and tags each write with the model version that produced it. Model v12 went live at 09:00. At 14:00 an alert shows it quoting below cost; the team rolls back to v11 at 14:20 and the mispricing stops. The log shows 3,100 quotes written by v12, of which 900 were accepted by customers before the rollback. What should the team do before declaring the incident resolved?
MCQ-27A ranking model ramps from 5% to 50% of traffic between 22:00 and 03:30. The halt policy wired into the deployment controller is: the canary's schema-invalid rate above 0.5% over a rolling 10-minute window with at least 300 requests, or the canary's p99 above 2x the 120 ms baseline, halts the ramp. The metrics pipeline emits schema-invalid counts as 1-hour aggregates; the controller polls it every minute. Total traffic is 20,000 requests/min. At 03:00 the canary's schema-invalid rate jumps to 0.9% and holds; the ramp finishes at 03:30, and the first hourly aggregate above 0.5% is emitted at 04:00. Which statement about this rollout is correct?
MCQ-23A team runs a 10-day shadow deployment for a candidate fraud model v3 alongside the incumbent v2. Real requests are mirrored to v3, its outputs are discarded, and no user sees them. Over 1.2M requests: v3 p99 latency is 210 ms (serving SLO is p99 ≤ 100 ms), error rate 0.02%, schema-valid responses 100%, overall disagreement with v2 1.8%. Disagreement by slice: routine requests are 97% of traffic, 0.3% disagreement, 31% of fraud value; high-value requests are 3% of traffic, 22% disagreement, 69% of fraud value. Which conclusion is justified by this shadow run?
MCQ-40A ranking service caches each prediction in Redis under the key `user:{id}` with a 24-hour TTL. Model v6 was replaced by v7 at 09:00 today, and the deploy script does not touch Redis. Five minutes later, a sample of 1,000 production responses contains 400 that are byte-identical to v6's output for the same users, while the v7 canary, running on a separate fleet with an empty cache, reports no ranking-metric change. What is the cause and the fix?
MCQ-86An ML gateway splits prediction traffic between model-v1 and model-v2 by a weight it reads from the config service on every request. Per-model dashboards show 5xx rates of 0.2% for model-v1 and 5.0% for model-v2, unchanged all morning. The gateway's aggregate 5xx rate has been 0.68% since 08:00 and sat there for 80 minutes after gateway 4.7.2, a logging change, was deployed at 10:04. At 11:30 an engineer set the model-v2 weight in the config console from 0.10 to 0.40; since 11:32 the aggregate has been 2.1% and the alert is firing. Which action fits the evidence?
MCQ-66Your registry runs a nightly cleanup that deletes any model version with zero serving requests in the last 30 days. Last night it removed nine versions, including v31. This morning the batch scoring job failed with 'artifact v31 not found', and the on-call found v31 is also the pinned rollback target for v34, the version currently serving. Artifacts live in a versioned object store, so deleted objects can be restored. What is the correct remediation?
MCQ-99Your team is standing up an ML platform for a fraud model that scores 1.2 M card authorizations/day (50 k/hour). The training job writes every run to `s3://models/latest/`, and the deploy job serves whatever is there. Last Tuesday the 03:00 run promoted a model with a feature-encoding bug that mis-scored one merchant category in 1 of 10,000 authorizations. The write replaced the previous 380 GB artifact; recovery took 9 h of retraining before service returned to normal. Compliance requires restoring the exact previously serving model version, and the team has two sprints before it automates retraining. Which change should ship first to make the next bad promotion recoverable in under 30 minutes?
MCQ-85A team's disaster-recovery plan for its model platform: 12 TB of model artifacts in an S3 bucket in us-east-1, replicated cross-region to us-west-2 with measured lag under 15 minutes; the model registry on a single RDS instance in us-east-1, with a nightly pg_dump written to an S3 bucket that also lives in us-east-1. The plan claims an RTO of 4 hours and an RPO of 24 hours. A drill restored the artifacts and the registry dump into a clean us-west-2 account, served the current model, and retrained successfully. Which finding should the review record?
MCQ-84A fraud model serves from two regions behind a latency-based router. A canary promotion of v52 started at 09:00; by 15:00 eu serves v52 with features 4 minutes old, while us still serves v50 with features 3 hours old. Support reports the same cardholder approved in eu and declined in us, and no alert fired. Replaying 1,000 recent requests through both regions with a shared feature snapshot, the two versions agree on 99.8% of decisions; replaying with each region's live features, they disagree on 7%. Which action addresses the cause?
MCQ-83A team is moving 30 ranking models from an on-prem scoring service to a managed platform. The migration gate is 99.5% output agreement per model over at least 100,000 mirrored requests. After a week of shadow traffic, 28 models agree on 100% of 1.4M requests; two agree on 96.2%, and every disagreement is a request with a missing categorical feature: the old service substitutes the literal "UNKNOWN", the new platform treats missing as its own category. The on-prem cluster is decommissioned in six weeks. What should the team do next?
MCQ-51An ML platform team rebuilt staging from the same Git repo that is meant to define production. The repo contains Terraform for the cluster and a KServe InferenceService manifest. The rebuilt staging endpoint came up with minReplicas 2 and timeout 30 s. Production runs minReplicas 8 and timeout 5 s. In production, `terraform plan` reports no changes, and `kubectl diff -f model/` reports no differences. Which explanation accounts for both clean checks and the drift?
MCQ-53A model registry stores artifacts as `torch.save` output. The serving container runs as UID 1000 and mounts a feature-store token at `/var/run/secrets/fs/token` that is readable by UID 1000 and grants read and write on the feature store. The loader fetches `ranker.pt` by tag, does not verify a digest, and calls `torch.load(path, map_location='cpu')` on PyTorch 2.4, where `weights_only` defaults to `False`. A contractor with write access to the registry bucket overwrites that tag with a crafted artifact. What does the next deploy do?
MCQ-52A warehouse service-account key was baked into a training image that has sat in a shared registry for three weeks. The key is one of several active keys on the account, carries read access to 11 schemas, and expires in 152 days; the image has been pulled 340 times. The warehouse issues a new key on rotation but leaves existing keys valid until their own expiry. Which action closes the exposure?
MCQ-54An ML platform retrains and redeploys 200 times a day. Every training job runs as one shared service account, sa-training, which can register a model version and set the registry's production alias that the inference service reads. On Tuesday two alias writes landed 27 seconds apart, both logged to sa-training with no job identifier: one promoted the run that had just finished, v412, and the other wrote the version it had read at start, v377, a three-week-old checkpoint. Production served v377 for three days; the team noticed only when v412's eval dashboard did not match online metrics, and no log showed which job made either write. A human reviewer on every promotion is off the table at this volume. Which change stops a training job from putting a version of its own choosing into production while keeping all 200 daily retrains automatic?
MCQ-55An MLOps engineer reviews a prediction-logging pipeline. The service writes one row per request to an S3 bucket: `user_id` (SHA-256 of the account email, salted with a per-day salt), `age`, `postcode`, `diagnosis_code`, `model_version`, `score`. Each day's salt is written to the same bucket at `salt/YYYY-MM-DD.txt`. The bucket's lifecycle rule expires objects after 24 months, and its bucket policy grants read to the `data-org` group (1,400 engineers). The source table these features come from is readable by 9 analysts and expires rows after 90 days. A reviewer signs the log off as de-identified. Which assessment of the sign-off is correct?
MCQ-56Hypothetical audit case. A recommender system's erasure pipeline deletes a user from the warehouse and the online feature store, then stops. User 418 first generated events on 2025-02-15. On 2025-06-10 an erasure request for 418 arrives. The remaining stores are: - fs_user_features: offline feature table built by an upsert job that never deletes keys missing from the warehouse. It still contains 418. - S3 training snapshots: 2025-01-01 (no 418), 2025-03-01, 2025-04-01, 2025-05-01 (the latter three contain 418). Snapshots are immutable, shared by several models, and can be copied and redacted but not edited in place. - prediction_log: append-only, 90-day retention, rows contain user_id and the raw feature vector. It contains 418 from 2025-02-15 through 2025-06-10. - model registry: v16 trained on 2025-01-01 and frozen for an audit until 2025-07-01; v17 trained on 2025-04-01; v18 trained on 2025-05-01. v17 and v18 serve production. Retraining takes 6 hours. The audit is scheduled for 2025-07-15. Which action set satisfies the erasure request for 418 with the least disruption to the audit?
MCQ-93A safety filter model sits in front of a generative endpoint. It was retrained and rolled out behind a gate that alerts when the aggregate block rate moves more than ±2 percentage points from its pre-rollout value. Post-rollout telemetry: aggregate block rate 3.4% → 4.1%; the Portuguese slice 3.0% → 27.0%, and Portuguese is 0.6% of traffic; filter p99 latency 45 ms → 190 ms against a 250 ms request-path budget; false-negative rate on the held-out eval set unchanged at 1.8%. No alert fired. What should the review require before this filter stays in production?
MCQ-97A benefits-eligibility team is deciding whether to promote model v7, a gradient-boosted classifier, to replace the four-condition rule that decides every application today. The rule reads a config table updated quarterly, is fully deterministic, and every denial must cite which of the four conditions failed to satisfy the regulator's adverse-action notice. v7 was trained on 18 months of the rule's own decisions, and the review was called to see whether it captures patterns the four conditions miss. On a 60,000-application holdout, v7 agrees with the rule on 99.4% of cases. Two adjudicators, blinded to both decisions, reviewed the 360 disagreements and determined the outcome the policy actually required: the rule matched it in 341 cases, the model in 19. v7 retrains weekly on the previous 90 days of decisions. What should the promotion review decide?
MCQ-41A credit model auto-approves applications scoring ≥ 0.62 and declines the rest. When the inference service returns 5xx, the serving layer substitutes a constant score of 0.5. Last quarter, outages sent 12,000 applications down the fallback path; replaying those applications through the model shows 8,400 would have scored ≥ 0.62. What did the fallback do, and what does it imply?
MCQ-20A risk model scores loan applications with logit = 1.2 − 0.06 × months_since_last_delinquency and a decision threshold of 0.5. Training imputes the training median of 26 months when the delinquency lookup is missing. After a serving-path rewrite, a missing lookup fills 0 instead. Requests with a missing lookup, 6% of traffic, now score 0.77 instead of 0.41; requests with a present value score as before, and the offline AUC is unchanged. The model is frozen for a regulatory review for 90 days, so retraining is not available. Which action is correct?
MCQ-35A ranking service must hold p99 end-to-end latency under 120 ms before launch. Live-traffic profiling of the current build shows a 240 ms p99, broken down as: feature retrieval 190 ms, queueing 26 ms, inference 8 ms, serialisation and other 16 ms. Feature retrieval issues four independent lookups sequentially, each about 45 ms at p99, plus about 10 ms of client-side overhead. The team has one sprint. Which change is most likely to meet the 120 ms target?
MCQ-36A serving team must choose the dynamic batching configuration for a new ranking model before it takes live traffic. The model is called inside a page with a hard 40 ms p99 budget, the GPU sits at 12% utilisation, and the production sequence-length mix is wide and stable. Which approach to choosing the configuration is defensible?
MCQ-37A GPU inference service runs vLLM pods behind a Kubernetes HPA that scales on CPU utilisation with a 70% target. In a load test the pods hold 22% CPU while GPU utilisation sits at 98%, the request queue reaches 400, and p99 latency passes 9 s against a 2 s SLO. Loading model weights into a new pod takes about 90 s. Which change should the review accept?
MCQ-38A model server loads a 12 GB artifact from object storage and warms up on start; measured start-to-ready is 90 s. One replica sustains 20 req/s at the 500 ms p99 SLO. Traffic is idle for ~20 min, then ramps from 0 to 40 req/s over 8 min and holds for 30 min. HPA scales on request concurrency, reacting within ~60 s. The team runs minReplicas 0; after each idle gap the first requests take ~90 s. Which configuration is the cheapest that holds the 500 ms p99 through the ramp?
MCQ-34An insurer scores each of its 1.4 million policyholders once a day to decide who gets a renewal-upsell email. A warehouse job rebuilds every policyholder's 30-day aggregates at 01:00; the campaign job reads the snapshot table at 06:00 and sends inside the next hour. The model is served today by a real-time HTTP endpoint that the autoscaler takes to 400 requests/s for the 06:00 window, at p99 85 ms and 290 dollars a day. The same model scores the whole table inside the 01:00 warehouse slot in 9 minutes for 3 dollars of compute. A score is usable as long as it matches the 01:00 aggregates; nothing in the email depends on events after that. Which change should the team make?
MCQ-49A team serves nine models, each on its own dedicated GPU endpoint costing $1,100 a month. Each model needs 4 GB of GPU memory and each endpoint's GPU has 24 GB. Fleet-wide utilisation averages 6 percent, request traffic is steady around the clock with no idle windows, and cost per thousand predictions is $0.38. Which change cuts cost per prediction the most?
MCQ-50A 16 GB inference host serves nine models behind a single multi-model endpoint. Eight models have a 1.5 GB working set; a ninth, added last week, has a 4 GB working set. The serving runtime itself holds 2 GB. Since the ninth model was deployed, p99 latency for three of the older models has gone from 30 ms to 900 ms, while the endpoint's mean latency has moved only from 32 ms to 61 ms. The serving log shows repeated load and unload events for those three models. What is the correct remediation?
MCQ-46A shared training cluster has one 8-GPU node and gang-schedules every job: a job gets all 8 GPUs or none. A hyperparameter sweep has held all 8 GPUs for 40 hours and does not checkpoint. A production retrain triggered by a data incident needs all 8 GPUs, the model does not fit on 4, and takes 6 hours. The hardware budget is frozen this quarter, so no new GPUs can be bought. Which change gets the retrain running within minutes?
MCQ-47A 30-hour training job runs on spot instances that are preempted on average once every 15 hours; preemptions are memoryless. Writing a checkpoint pauses training for 2 minutes, and resuming from one is instant. The team wants the checkpoint interval that minimizes expected wall-clock time. Which interval should they use?
MCQ-48A training job needs 12 hours of compute. Preemptible instances cost $0.45 per instance-hour and are reclaimed every 6 hours; on-demand instances cost $1.20 per instance-hour. The training script checkpoints every 3 hours. What is the effective cost per completed run on preemptible instances?
MCQ-43A nightly pipeline runs as a single Kubernetes Job with no mounted volumes: each stage holds its output in memory and passes it to the next. The stages are ingest (20 min of CPU), transform (40 min of CPU), train (4 h on 8 GPUs), evaluate (10 min of CPU), and register (1 min of CPU). Registration authenticates with a token the Job reads from its environment at start; the security team revoked the previous token that afternoon, so runs that began before the secret was updated get a 401 at the final step, after training has already finished. Which proposal keeps a failed registration from repeating the training, and what does that failure then cost?
MCQ-42A nightly scoring job reads 2.1M entities and writes one row per (entity, run) into `scores`. The table is append-only: a compliance rule requires every attempt to be retained, so no upsert, delete or partition swap is allowed, and the job runs as three parallel tasks that commit independently. The orchestrator retries the whole job on failure. Last night attempt 1 wrote 1.4M rows and died; attempt 2 wrote all 2.1M rows. Both are in the table under run_id='2024-06-01', tagged attempt=1 and attempt=2, and the model artifact is reloaded per attempt, so attempt 2 scored with v13 while attempt 1 used v12. The ranking service runs `SELECT entity_id, AVG(score) FROM scores WHERE run_id='2024-06-01' GROUP BY entity_id`. What is the correct remediation?
MCQ-78A nightly batch scoring job reads 2.4M rows, starts at 01:00, and must finish before a 06:00 campaign send. A 300-minute window. It currently takes 190 minutes. Input volume is growing 20% per quarter and runtime scales linearly with rows. The team's only alert fires when the job misses 06:00. Which alerting policy should replace it?
MCQ-94An ML platform calls a model provider with a quota of 600 requests/min and 10M tokens/min. A nightly batch job fans out 40 workers and offers 900 requests/min for 5 minutes; each request averages 1,200 tokens. On any 429 the client retries immediately, up to 3 more times, with no jitter. The provider rejects over-quota requests with 429 and never queues them. The quota stays saturated for 30 minutes, and the client sends about 18,000 requests in that window. Which diagnosis fits these measurements?
MCQ-95Four ML teams share one GPU cluster. Finance treats the cloud billing export as complete per-team attribution because it groups spend by Kubernetes namespace and each team has its own namespace. Last month's bill was $412k, up from $190k, and the `platform` namespace alone was $180k of it. The three largest training jobs, 61% of the increase, ran in `platform`: teams submit through a shared Argo workflow that must run there for RBAC reasons, and every pod it creates carries a `team` label that the export never reads. Which change makes next month's bill attribute that $180k to the teams that caused it?
MCQ-82A four-engineer ML team must put a real-time fraud-scoring model behind an interactive checkout flow. A managed serving vendor quotes $120k/year, runs inference in its own region, offers no VPC peering on the tier the team can afford, and publishes a 120 ms p99. Self-hosting on the existing Kubernetes cluster is estimated at two engineers for six months (fully loaded $18.3k per engineer-month) plus 0.5 FTE ongoing ($110k/year fully loaded). The checkout flow requires p99 under 80 ms, and the processing agreement forbids fraud features leaving the VPC. Which path do you take?