Every question on one page: open one, choose an answer, and see why at once. Signed in, your latest answer to each question counts, and you can answer again after 24 hours.
Curated: · Written: · Reviewed:
Reviewed 100Review pending 0
Answers to reviewed questions count toward this role and the overall leaderboard; your latest answer to each one is the one that counts.
MCQ-32A credit-default model uses income as its only feature. The training set holds 1,000 applications. Income is missing for 200 of them, and those 200 include 16 defaults. Of the 800 with income recorded, 600 are ≥ $50k and include 12 defaults, while 200 are < $50k and include 20 defaults. The team adds an explicit missing indicator and trains a depth-2 tree that splits first on the indicator, then on income ≥ $50k within the recorded branch. What default probability does the model assign to an applicant whose income is missing?⌄
MCQ-41A support assistant answers a frozen 400-question eval set: 300 questions on policy text unchanged since the model's training cutoff, and 100 on SKU prices that changed after the 6,000 fine-tuning pairs were written. The price feed is regenerated nightly; a full fine-tune takes three days on the team's single A100 and the team can afford one per quarter. Measured on that eval set:
| Configuration | Stable 300 | Post-cutoff prices 100 | Cost / 1k requests | p95 |
| --- | --- | --- | --- | --- |
| Prompt only | 88% | 41% | $1.10 | 900 ms |
| Prompt + retrieval over the nightly feed | 90% | 86% | $1.90 | 1,400 ms |
| Fine-tuned, no retrieval | 93% | 38% | $0.80 | 850 ms |
| Fine-tuned + same retrieval | 91% | 44% | $1.60 | 1,350 ms |
Which configuration should the team ship?⌄
MCQ-21A retailer forecasts daily demand for two SKUs. Over the same 30-day holdout, model A's mean absolute error is 5 units on a niche SKU that sells 25 units/day and 1,000 units on a core SKU that sells 50,000 units/day. Model B's mean absolute error is 1 unit and 2,000 units on the same two SKUs. The planning team's cost model charges a flat rate per unit of forecast error, whichever SKU it falls on. The dashboard ranks B first, on a MAPE of 4% against A's 11% (MAPE averaged across the two SKUs). Which model should the review recommend, and on what basis?⌄
MCQ-12A fraud model's chargebacks land 30 days late, so labelled performance lags; the shifts that hurt arrive within hours; input drift also fires on marketing traffic-mix changes that leave accuracy unchanged; and label volume tracks traffic, not model health. Which trigger policy holds up?⌄
MCQ-17A team's tracker keys runs by `script_name + '-' + git_branch`; each launch replaces the key's metrics row and `model.txt`. The entry point is `train.py` on branch `exp`. Three launches run: (A) features v7, snapshot 2026-06-01, seed 0 -> AUC 0.844; (B) features v7, snapshot 2026-06-01, seed 1 -> AUC 0.838; (C) features v6, snapshot 2026-05-01, seed 0 -> AUC 0.831. The tracker UI now shows AUC 0.831, features v6, snapshot 2026-05-01. The board wants to promote the 0.844 model. What does the tracker actually hold for this key?⌄
MCQ-20A fraud model scores 1,000,000 card transactions; 5,000 are fraud. At the shipping threshold it flags 25,000 transactions, 2,000 of which are fraud. Offline ROC-AUC is 0.94 and PR-AUC is 0.31. The launch review asks which reading of these numbers is correct.⌄
MCQ-33A team scores loan applications with two models on the same features: `income` in dollars (range $18,000–$240,000) and `debt_ratio` in [0, 1]. Model A is a k-NN (k=5, Euclidean distance) and Model B is a gradient-boosted tree. Neither pipeline scales its features. Model A's validation accuracy is 0.64 against Model B's 0.91, and for a sample with income $200,000 and debt_ratio 0.05, all five of Model A's nearest neighbours have income within $2,000 of it, whatever their debt_ratio. What should the team do?⌄
MCQ-11A payments fraud model scores transactions. Each week you log the maximum population stability index (PSI) across features against the training reference and AUC on the labelled chargeback slice that arrives 30 days later. Use the common PSI bands: below 0.1 stable, 0.1–0.25 moderate, above 0.25 major. At week 8, max PSI is 0.29 on the `merchant_category` feature; labelled AUC is 0.91 against a training baseline of 0.92. At week 16, max PSI is 0.04 across all features; labelled AUC is 0.76. Which pair of diagnoses and actions fits these two readings?⌄
MCQ-13A team shadow-scores a candidate ranking model against the incumbent for two weeks. Both models score every request; only the incumbent's output is served. Over 1.2M paired requests: incumbent p99 41 ms, candidate p99 380 ms; incumbent error rate 0.02%, candidate 3.40%; disagreement at the serving threshold 7.1%. The promotion gate requires candidate p99 ≤ 60 ms and candidate error rate ≤ 0.1%; disagreement above 10% is flagged for manual review. What should the review decide?⌄
MCQ-14A team ramps a new ranking model with a pre-declared guardrail: roll back automatically if add-to-cart rate drops more than 0.5% relative to control at the 5% traffic step. At the 5% step, 120,000 sessions have been collected (the step's minimum is 100,000). The aggregate add-to-cart rate is down 0.3%, inside the bound, so the rollout proceeds to 25%. Two weeks later the aggregate is still within 0.3%, but the mobile slice is down 4.1% and the desktop slice is up 3.8%. What should the rollout policy have done?⌄
MCQ-15A fraud-scoring service logs the model registry id, the serving image digest, and the raw request payload with every prediction, but not the computed feature vector. The registry entry for `fraud-v9` pins the artifact sha256, the training snapshot id, and the training commit. An auditor replays 200 decisions from 90 days ago using the logged image digest and the logged payloads: all 200 return a different score, while the artifact hash and training commit both match what was logged. What is the missing pin?⌄
MCQ-16A team reports a new model improves validation AUC by 0.6 points over the incumbent (0.851 vs 0.845). Before shipping, they retrain the new model five times with identical configuration except the random seed, getting 0.842, 0.858, 0.849, 0.860, 0.845. What does this study support?⌄
MCQ-18A moderation classifier scores every post 0–1 and flags it when the score is above the deployed threshold. The review team clears 3,000 posts a day. At the deployed threshold of 0.5 the model flags 24,000 posts a day (precision 0.19, recall 0.94), the queue is 8 days deep, and the posts users reported are the ones waiting longest. A sweep on last month's labelled set gives: 0.80 → 7,400 flagged/day, precision 0.44, recall 0.81; 0.91 → 2,950 flagged/day, precision 0.68, recall 0.62; 0.97 → 900 flagged/day, precision 0.86, recall 0.31. What should the team do?⌄
MCQ-19A trust-and-safety team reviews flagged content with a hard ceiling of 3,000 human reviews a day for the next two quarters; headcount is frozen and there is no contractor budget. Policy requires a human to sign off on every removal, and the SLA is the total number of violations removed per day, not a severity-weighted score. Reviewer agreement with the current model has fallen to 31% and reviewers have started clearing the queue by pattern rather than reading each item. Reviewer time is the scarce resource and the 3,000 slots are the team's only lever on daily violations removed. Leadership wants a queue design to run next week and a defensible case for more reviewers next budget cycle. Which approach should the team adopt for the next two quarters?⌄
MCQ-22An e-commerce search team is choosing between the incumbent ranker and a candidate for the results page. The offline harness grades 100 documents per query and can report NDCG at any cut. The page renders 10 results, and 6% of sessions open page 2. On the same judged queries the candidate scores NDCG@100 0.482 against the incumbent's 0.470, NDCG@10 0.355 against 0.371, and pointwise accuracy over the judged documents 0.74 against 0.71; its NDCG@100 gain comes entirely from positions 11 through 100. What should the team do?⌄
MCQ-23A team has three weeks of interaction logs produced entirely by the incumbent ranker, which ranked a candidate pool of thousands of items and showed the top 10 on every request. No randomized traffic was ever served, and tail items had a propensity below 1e-3. They need an offline CTR estimate for a new ranker before committing to an A/B test. Which offline evaluation plan is sound?⌄
MCQ-24A ticket-escalation model scored 0.96 AUC in cross-validation and 0.77 on the next month's live traffic. The pipeline fits a target encoder for `queue` (40k levels) on the full training frame before splitting, and `resolution_code` is written when the ticket closes. The team retrains this week and needs the offline number to predict live performance. Which change closes the gap?⌄
MCQ-25A marketplace ranks items with matrix factorization over purchase co-occurrence. Of 8,100 items listed in the last 30 days, none reached 100 impressions, and sellers of those items churned at 3.4x the platform rate. Listing metadata (title, category, price, image embedding) is available at creation time, and a content-based score over it predicts first-week click-through at AUC 0.71 on held-out data; past 200 interactions the collaborative model is 0.09 AUC ahead of that score. New items see a median of 3 impressions/day, and the zero-impression rate is the same in every category. Which approach should the team adopt?⌄
MCQ-26A team is building a gradient-boosted model to predict whether a support ticket will breach its SLA. They fit the same hyperparameters on 10%, 25%, 50% and 100% of an 800,000-row training set:
| Train fraction | Train logloss | Validation logloss |
| --- | --- | --- |
| 10% | 0.61 | 0.68 |
| 25% | 0.60 | 0.655 |
| 50% | 0.595 | 0.648 |
| 100% | 0.59 | 0.645 |
Predicting the majority class scores 0.693 logloss. The 40 features are raw counts and timestamps. Which action is most likely to lower validation logloss?⌄
MCQ-27A credit-risk model has 340 features; 60 of them form 12 groups of near-duplicates (pairwise |r| ≥ 0.97) drawn from three vendor feeds. L1 at alpha 0.01 leaves 38 non-zero coefficients and validation AUC 0.831. Refit on five bootstrap samples, the surviving set has mean pairwise Jaccard 0.41 while the count stays at 38 ± 1. Which explanation fits?⌄
MCQ-28A churn model ships next week. On the held-out validation set it scores 0.84 AUC against a 0.80 launch bar, and its train AUC is 0.95. Broken out by tenure, validation AUC is 0.89 for accounts over a year old (54% of rows), 0.85 for 90–365 days (24%), and 0.61 for under 30 days (22%); on the under-30-day rows train AUC is 0.98. Which reading of this evidence is right?⌄
MCQ-29Review a 200 GPU-hour hyperparameter search for a model with eight hyperparameters. Prior sweeps show only learning rate and dropout move validation AUC by more than 0.001; the other six are fixed by the architecture. A pilot finds that at 0.05 GPU-hour per run, the eventual best configuration is in the top 5% for 90% of seeds, and that top 5% contains the eventual best 95% of the time when trained for 0.5 GPU-hour. A full run costs 0.5 GPU-hour. The proposal runs a 3^8 grid at 0.5 GPU-hour per run on 32 GPUs and calls it 102.5 wall-clock hours. Which correction should the review require?⌄
MCQ-30A team must choose one model for a 250,000-row, 180-feature tabular risk score. The features are numeric and categorical only: no text, no images, and no pretrained model to transfer from. The team has 20 wall-clock hours for model selection and tuning, and the chosen model is retrained weekly. On identical folds, LightGBM reached 0.847 AUC at 6 minutes per run; TabNet reached 0.844 AUC at 14 hours per run. Which plan is defensible?⌄
MCQ-31A fraud model is trained on 40M card transactions. merchant_id has 2.1M distinct values; 60% of merchants appear fewer than 5 times in the 3-month training window and together account for 30% of transactions. Per-merchant fraud rate ranges from 0.1% to 40%. The model is a GBDT, the feature vector is capped at 500 dimensions, and the same merchant can appear in both the training and holdout splits. Which encoding should the team use for merchant_id?⌄
MCQ-34A fraud model scores 900 features with a gradient-boosted tree. Serving p99 is 40 ms against a 35 ms budget, and the risk team needs per-feature importances for its model documentation. Three candidates were measured on the same folds: the unreduced model at AUC 0.851 / 40 ms p99; the top 120 features by importance at 0.849 / 31 ms; 50 principal components at 0.830 / 37 ms. Which change should ship?⌄
MCQ-36An 8 to 1 cost asymmetry applies: a missed fraud costs 8 dollars for every 1 dollar of false-alarm cost, and the model is served at one fixed decision threshold. Fraud runs at roughly 1 positive per 249 negatives, the holdout set is large enough to resolve single-digit-percent differences between candidates, and the nightly retrain already runs, so changing the training config costs nothing extra. Which plan should the team adopt?⌄
MCQ-37A team has 4,000 labelled rows and splits them 70/15/15 into train, validation and test. Early stopping monitors the validation split. They train 20 variants that differ only in random seed and feature subset, score each on the 600-row test split, and report the best test AUC, 0.861, as the expected production figure. Live traffic measures 0.834. Which diagnosis of the 0.861 figure is correct?⌄
MCQ-38A team's five-model ensemble scores 0.848 AUC against the best single model's 0.844 on the same folds, with a fold-to-fold standard deviation of 0.006. Serving p99 is 51 ms for the ensemble and 12 ms for the single model, against a 20 ms p99 budget, and the ensemble needs five retraining pipelines against the single model's one. Which policy is defensible?⌄
MCQ-39A team adapts a 25M-parameter pretrained vision backbone to a new task with 4,000 labelled images. Three runs on the same validation split:
| Trainable parameters | Learning rate | Validation accuracy | Wall time |
| --- | --- | --- | --- |
| Head only (0.5M) | 1e-3 | 84.1% | 12 min |
| Last block + head (4.2M) | 1e-4 | 85.6% | 41 min |
| All parameters (25M) | 1e-4 | 71.3% | 110 min |
The team has one 45-minute run left before the deadline and wants the highest validation accuracy it can defend. What should it run?⌄
MCQ-40Hypothetical scenario: A team must ship a model with p99 latency ≤ 25 ms. They can run one more training job before launch. Current candidates on the same test set:
| Model | Architecture | Training | Accuracy | p99 | Meets 25 ms |
| --- | --- | --- | --- | --- | --- |
| Teacher | 340M params | — | 91.2% | 980 ms | no |
| Student A | 4-layer | hard labels | 86.4% | 18 ms | yes |
| Student B | 6-layer | soft targets, T=4 | 89.6% | 30 ms | no |
| Baseline | 4-layer | — | 84.0% | 15 ms | yes |
They have not yet tried a 4-layer student trained on the teacher's soft targets. Which action is best supported?⌄
MCQ-42A churn model's split-gain importance ranks sessions_7d, sessions_14d and sessions_28d first, second and third, but the order changes every quarterly retrain; the three are pairwise correlated at 0.94. On the held-out set, permuting any one of them drops AUC by 0.002, while permuting all three drops it by 0.09. What should the team do?⌄
MCQ-43On 2024-06-12, a lender's credit model scorecard-v4 declined application APP-88231 with score 612. The prediction log stores model_version=scorecard-v4, the exact feature vector (missed_payments_12m=3, credit_age_months=7, income_verified=true, utilization=0.82), and the decision. The registry has the v4 artifact and training feature list. The team also has a global SHAP summary plot from v4 training, where missed_payments_12m is the top global feature. The applicant asks for the reason for this decision. Which action produces a correct local explanation for APP-88231?⌄
MCQ-44A daily demand model predicts day T's units at 06:00 on day T. The order system finalizes day D's units at 23:00 on day D+2. The current feature is `units.rolling(7).mean()` aligned to day T. Offline, with random 5-fold CV, MAPE is 4.1%; in production it is 23.8%. Which feature should the team ship?⌄
MCQ-45A grocery chain trains a daily demand model on 1,095 days (three years) of history and validates on a random 20% holdout, reporting 9.0% MAPE. A backtest on last year's Black Friday shows 61% error. The team argues the peak is one day in 365 and proposes shipping. Which statement should the review accept?⌄
MCQ-46A payments platform runs an unsupervised autoencoder that scores 900,000 events/day with a reconstruction-error score in [0,1]; no labels exist. The review team can adjudicate 1,500 alerts/day. A stratified sample of 100 scored events from each of three bands was adjudicated:
| Score band | Events/day | Sampled | Confirmed actionable |
| --- | --- | --- | --- |
| 0.95–1.00 | 200 | 100 | 74 |
| 0.80–0.95 | 900 | 100 | 31 |
| 0.50–0.80 | 12,000 | 100 | 4 |
Which operating point should the team ship?⌄
MCQ-47A retail team's k-means segmentation feeds a campaign platform that can run at most 4 campaigns per quarter. The pipeline currently returns 14 segments, and re-running it on a resampled week moves 48% of customers to a different segment. Silhouette is flat (0.41–0.46) from k=3 to k=14. Which justification for the cluster count and its validation is sound?⌄
MCQ-48A two-head model shares one trunk: head A predicts add-to-cart and head B predicts review submission. Trained alone, head A reaches AUC 0.81 and head B 0.66. Trained jointly with equal loss weights, head A falls to 0.74 while head B rises to 0.72, and the cosine similarity between the two tasks' gradients on the shared trunk is -0.4. The team wants head A's AUC back without giving up head B's gain. Which change is most likely to achieve that?⌄
MCQ-49A churn model is trained on 400,000 legacy accounts (median tenure 4 years, 70% on annual contracts) and deployed to a self-serve cohort (median tenure 3 months, 85% on monthly plans) with 5,000 labelled rows. Pooled holdout AUC is 0.83; AUC on the 5,000 new-cohort rows is 0.71. The team wants the new-cohort number to improve and will not discard the legacy rows. What should it do?⌄
MCQ-50A ranking model is retrained monthly on a 5,000-label budget drawn from a 4M-row unlabelled pool by uncertainty sampling. Round 1 gained +0.021 held-out AUC, round 2 +0.006, round 3 +0.001, while production AUC on the top-decile traffic slice fell from 0.83 to 0.79. 62% of the rows picked in rounds 2 and 3 came from one segment that is 0.4% of production requests, and the held-out set is re-drawn each round from the same uncertainty-selected pool. What should the team change?⌄
MCQ-51A team is deciding whether to trust a new classifier's 88% accuracy on a 500-example evaluation set. Three annotators labelled the same 500 examples; 70% of the examples are positive, and the annotators' labels follow the same 70/30 split. Pairwise raw agreement is 82%, and the team lead says that is high enough to proceed. Which conclusion does the agreement data support?⌄
MCQ-52A payments team is sizing an A/B test for a redesigned checkout before it ships. Baseline conversion is 3.2%; the smallest lift worth shipping is 1% relative (0.032 percentage points). They can randomize 20,000 users per arm per day and have budgeted a 14-day run, testing at alpha 0.05 with 80% power. Which conclusion is right?⌄
MCQ-53A growth team runs a 30-day A/B test on a checkout change. The dashboard computes a two-sided p-value every day. On day 12, p=0.028. The team lead wants to stop and ship. The original plan was a single fixed-horizon analysis at day 30 with alpha=0.05. No sequential boundary was pre-registered. What should the review conclude?⌄
MCQ-54A team runs a 14-day A/B test of a new ranker at 50/50 traffic. Before the test they pre-registered one primary metric and three guardrails, each with a bound; a guardrail is breached when its observed effect exceeds its bound, and a breach stops the rollout. Results: click-through +3.1% (95% CI +1.2% to +5.0%); p95 page latency +38 ms (95% CI +20 to +56), bound +50 ms; control completed-purchase rate 4.00%, treatment 3.96% (95% CI for the relative change -1.5% to -0.4%), bound -0.5% relative; complaint rate +0.05 pp (95% CI -0.02 to +0.12), bound +0.20 pp. What should the team do?⌄
MCQ-55A ranking model ships behind an A/B test. Week-one data shows a 9.1% lift in the primary metric, and the annualised business case has already been built on that number. Four weeks of traffic remain before the launch decision. Which measurement plan is sound?⌄
MCQ-56A grocery marketplace tests a 'free delivery over $35' banner for four weeks, splitting 2.4M users 50/50. Both arms order from the same 40-city courier pool. Orders per user rose 6.0% in the treatment arm against its pre-period baseline and fell 3.0% in the control arm, and courier on-time delivery dropped from 94% to 88% across both arms. What does this evidence support?⌄
MCQ-57A team testing a new checkout flow on existing users measures 14-day revenue per user. To shorten the test they apply CUPED using revenue in the first 3 days after exposure as the covariate, with theta = 0.58. Control 14-day revenue averages $40.00 and control first-3-day revenue averages $10.00; treatment first-3-day revenue averages $12.28. The unadjusted lift is +4.1% and the CUPED-adjusted lift is +0.8%. What should the team conclude?⌄
MCQ-58A team evaluates a new ranking model against the incumbent. They pre-registered one primary metric (NDCG@10 over all traffic) and also computed 240 exploratory cells (60 segments × 4 secondary metrics). At α = 0.05 uncorrected, 17 exploratory cells favor the new model. The primary metric is flat: Δ = +0.1%, 95% CI [−0.4%, +0.6%]. The launch decision is due tomorrow. What should the team do?⌄
MCQ-59A ranking model was A/B tested on 50/50 traffic for two weeks (4.2M sessions). Conversion rose within every device class: desktop 5.0%→5.2%, mobile 2.0%→2.1%, but overall conversion fell from 3.80% to 3.53%, because the model's reordering moved desktop's share of sessions from 60% to 46%. The shift held in both weeks, the shift toward mobile is itself a product goal, and the launch decision turns on overall conversion. Which handling should the team adopt?⌄
MCQ-60The team is deciding whether to replace incumbent ranker A with candidate B. On a held-out test set of 20,000 rows, B scores 0.812 AUC and A scores 0.806. Bootstrapping each model separately gives 95% intervals of 0.804–0.820 for B and 0.798–0.814 for A. A paired bootstrap of the per-row AUC difference over 2,000 resamples gives a 95% interval of 0.001–0.011. What does the evidence support?⌄
MCQ-61A ranking model shipped to 3 metro regions while the other 7 kept the old model, because a per-user A/B test was impossible. Checkout conversion per session, averaged over the 4 weeks before and after the launch:
| Group | Pre | Post |
| --- | --- | --- |
| Treated (3 metros) | 4.10% | 4.59% |
| Untreated (7 metros) | 4.05% | 4.48% |
Both groups moved together over the 4 pre-launch weeks, +0.02 pp per week. Which effect size should the review record as the model's contribution?⌄
MCQ-62Two classifiers, A and B, are compared on the same held-out test set of 20,000 rows. The discordance table is: both correct 16,000; A correct and B wrong 1,440; A wrong and B correct 1,560; both wrong 1,000. A two-proportion test that treats the two accuracy sets as independent gives z = 1.81, p = 0.07. The team is deciding whether to replace A with B in production. Which conclusion should the evaluation report state?⌄
MCQ-63A team ships twelve model updates over twelve months. Each ran a two-week A/B test against the then-current production model and reported a significant +1.2% on the primary metric, so the launch log sums to +14.4%. A 1% holdback, held out of every update for the full year, shows the treated population at +3.1% versus the holdback. What does the comparison establish?⌄
MCQ-64A churn model's features are rebuilt by a nightly job and the outreach decision is sent the following morning. A week-long comparison scored the same decisions both ways: 0.3% differed, and the online path added $6,400 a month plus a 24-hour on-call rotation. Which change should the team make?⌄
MCQ-65A model review board is setting latency budgets for a model in a request path. The model's forward pass measures 8 ms at p99 in isolation, but the end-to-end p99 is 214 ms against a 200 ms SLO; feature retrieval accounts for 184 ms of that. Which approach should the board adopt?⌄
MCQ-67A churn-scoring service caches predictions in Redis under the key `user_id | model:churn-v14` with a 24-hour TTL. At 11:00 the feature pipeline ships feature set `v8`, which redefines `sessions_7d` from a 7-day session count to a 7-day session rate; the model `churn-v14` is unchanged. Traffic is steady at 4,000 rps with a 92% cache hit rate. At 11:20 the p50 score for returning users has not moved from yesterday, and 92% of served responses were computed from `v7` features. What is the correct diagnosis and fix?⌄
MCQ-68The team must agree an approach to autoscaling a model service and cold starts before the model ships. A replica takes 96 s from scheduling to first successful prediction: 71 s to read 4.2 GB of weights from object store, 18 s of kernel-cache warmup, 3 s of readiness probing. Weekday traffic doubles within 60 s at 09:00 and is flat the rest of the day, peak is 4× off-peak, and the GPU budget is fixed. Which approach holds up?⌄
MCQ-69A team is choosing how to serve a model behind a synchronous API. The SLO is p99 ≤ 50 ms at 40 requests per second; traffic is flat and forecast to stay flat for the next year; the model has been validated at int8 with no measurable quality loss; and the GPU reservation is month-to-month. Their benchmark at 40 rps: one GPU instance, p99 22 ms, 12% utilisation, $9,400/month; eight CPU cores, p99 48 ms, 68% utilisation, $610/month; eight CPU cores with int8, p99 26 ms, 51% utilisation, $610/month. Which recommendation should the review accept?⌄
MCQ-71A fraud model reads `user_txn_count_1h` from a feature store. The Flink job that computes it reports 100% job success and 0-record consumer lag, and the feature's p99 age measured when the value is written to the store is 45 s against a 90 s freshness SLO. Over two days the model's precision fell 3.2%. The serving path caches each feature value for 600 s before re-reading the store. What explains the gap between the 45 s metric and the degraded predictions?⌄
MCQ-72A nightly training pipeline validates each snapshot before training with four gates: column presence and types against SCHEMA_V7; row count within ±15% of the previous run; null rate per column at most 2%; and category overlap with the prior snapshot at least 95%. Last night's snapshot has 3.1M rows against 4.2M the night before, an amount_usd null rate of 0.61, 17 distinct country values against 42 in the prior snapshot, and every column and type matching SCHEMA_V7. Which gates breached?⌄
MCQ-73A payments team rebuilds 24 months of training data for a fraud model. The feature `merchant_risk_tier` is joined from `merchant_dim`, a Type 1 dimension overwritten in place and re-tiered about quarterly. Offline AUC on the rebuilt set is 0.94, up from 0.88. Online, AUC over the first three weeks is 0.86. Training and serving both call the same `compute_features` function, and labels settle within 24 hours. In a sample of 4,000 training rows dated 2024-03, 29% carry a tier that differs from the tier the event log records for that merchant on that date. Which action fixes the model?⌄
MCQ-74An Airflow DAG runs one Spark task per day that writes s3://features/date=<ds>/ with df.write.mode('append'); the task is configured with retries=2 and retry_delay=5m. For 2024-03-07 the partition should hold 1,000,000 rows and holds 1,600,000, and the extra 600,000 are byte-identical copies of rows already in it. The task log shows attempt 1 starting 01:12:04 and being killed at 01:12:57 when its executor node was lost, then attempt 2 starting 01:18:30 and finishing successfully. Two consumers read that partition continuously: an hourly scoring job and a backfill tool that re-runs several days at once. Which change removes this failure mode?⌄
MCQ-75A team fine-tunes a model under a $6,000 hard cap with a checkpoint every 30 minutes. A pilot on 5% of the data cost $290 (9 GPU-hours) and reached validation AUC 0.812; linear extrapolation to the full dataset gives $5,800 and an estimated 0.84, the bar the approval required. The full run is launched. At 40% of the planned steps it has spent $4,900 and validation AUC is 0.821. The learning-rate schedule was configured for a dataset one quarter the size of the full one. What should the team do?⌄
MCQ-76A 7B-parameter model trains in bf16 on 8 A100s of 40 GB each. At global batch 32 and sequence length 4096 the per-device working set is: weights 14 GB, Adam optimizer state in fp32 56 GB, gradients 14 GB, activations 9 GB. Plain DDP OOMs on every rank. An earlier 8-stage pipeline run of the same model on this cluster took 1.6× the wall-clock per step of the data-parallel run at the same global batch. What should the team do?⌄
MCQ-77A feature pipeline emits a daily report of unique users per campaign. It processes 400M events a day across 500 campaigns, and 2.1M distinct users appear overall. The aggregation stage runs with a 4 MB memory budget, and each campaign's unique-user count must land within ±2% of the true value. Which aggregation meets both constraints?⌄
MCQ-78A retrieval service streams 2,000,000 scored candidates per request and returns the top 100 to the ranker. It keeps them in a min-heap of size 100 with this loop:
```python
for score in scores:
if len(heap) == 100:
heapq.heappop(heap)
heapq.heappush(heap, score)
```
p99 is 28 ms, but the output is wrong: on a query whose true 100th-best score is 0.91, the service returns items scoring as low as 0.12. The service must stay under 50 ms p99; the full sort it replaced measured 310 ms. Which change fixes the selection?⌄
MCQ-79A stream processor computes a 5-minute sliding count per key. Today, on each event, it scans the key's buffer and counts entries within 300 s. The buffer holds 2,000 events per key on average, and a scan costs 0.3 µs per buffered event. Traffic is 5,000 events/s. A rewrite keeps a deque of timestamps plus a running count: 0.6 µs to push an event and 0.3 µs to evict one. In steady state, every event that enters the window is eventually evicted. The consumer runs on a single core. What is the per-event CPU cost after the rewrite, and what fraction of one core does the consumer use at 5,000 events/s?⌄
MCQ-80A nightly abuse-flagging job scores 40 million events. Flagged volume is monotone decreasing in the score threshold. The team needs the threshold that flags 3,000 events per day, accurate to ±1e-4, and the job must finish in under 5 minutes. Each full scan of the 40M scores takes 3.1 seconds. The threshold must be stored with the model version for audit, and monotonicity must be verified on validation scores before use. The current sweep evaluates thresholds from 0.0 to 1.0 in steps of 0.001. Which change meets all requirements?⌄
MCQ-81A dedup pipeline over 2.4M customer records turns every pair scoring above threshold into an edge and takes connected components as entities. On a 50,000-pair labeled set the matcher scores precision 0.998 and recall 0.71. The component size distribution comes back as 214,000 components of size 2–4, 3,100 of size 5–20, 62 of size 21–500, and one of size 41,000. The pipeline splits any component above 50,000 records. What does the 41,000-record component most likely indicate?⌄
MCQ-82A dedup job over 2 million product titles ran all-pairs edit distance: about 2×10^12 comparisons, abandoned after 30 hours. Replacing it with 3-gram signature blocking gave 41 million comparisons in 18 minutes, with 0.97 recall against a hand-labelled sample of true duplicate pairs. The catalogue is about to grow to 8 million titles, and the batch window is 2 hours. Which statement about the two approaches at the new size is correct?⌄
MCQ-83A team is choosing the prefix structure for an autocomplete service over a vocabulary of 8 million distinct queries. Their trie stores the top-10 completions at every node; it has 40 million nodes, and each node costs 144 bytes (26 four-byte child pointers plus a ten-entry list of four-byte query IDs), so the structure is 5.8 GB. The service runs on 8 pods, each capped at 512 MB of resident memory, and the p99 lookup budget is 10 ms. A compressed prefix index (FST) over the same vocabulary measures 410 MB and 6 ms p99. The per-pod memory cap cannot be raised this quarter. Which structure should ship?⌄
MCQ-84A streaming job sessionizes clickstream events per user with a 30-minute inactivity gap: a new session starts only when the gap to the previous event for that user is strictly greater than 30 minutes. The batch below is one micro-batch; arrival order is shown, and event_time is UTC.
| arrival | user | event_time |
| --- | --- | --- |
| 1 | u1 | 10:00 |
| 2 | u2 | 10:05 |
| 3 | u1 | 10:50 |
| 4 | u2 | 10:40 |
| 5 | u1 | 10:25 |
| 6 | u2 | 11:00 |
| 7 | u1 | 11:10 |
Which session assignment must the job emit for this batch?⌄
MCQ-85A join pipeline reads two event-time-sorted streams: 2.4 billion ad impressions at 120 bytes each, arriving at 40,000 records/s, and a much smaller click stream. The join window is 5 minutes. A hash join that built a table on the impressions side was killed at 240 GB. The team replaces it with a two-pointer merge join that advances whichever cursor is behind and buffers only records still inside the window. Roughly how much memory does the impressions-side buffer need at peak?⌄
MCQ-86A feature pipeline runs three stages. Before deciding whether to triple the cluster to hit a deadline, the team measured wall-clock time at 10%, 25%, and 50% of the training data:
| Data | Stage A | Stage B | Stage C | Total |
| --- | --- | --- | --- | --- |
| 10% | 6 min | 3 min | 1 min | 10 min |
| 25% | 15 min | 18 min | 2.5 min | 35.5 min |
| 50% | 30 min | 75 min | 5 min | 110 min |
Which action is justified by these numbers?⌄
MCQ-87An ML team trains on 400 GB of Parquet data split across 40 files of 10 GB each on a 64 GB machine. Their map-style Dataset assigns each worker one shard and caches it as a pandas DataFrame; the DataLoader runs with num_workers=8, prefetch_factor=2 and batch_size=1024. The first epoch is OOM-killed at ~62 GB RSS after 25 minutes. Which change lets the epoch finish on this machine?⌄
MCQ-88A nightly job scores 12,000,000 rows on one shared worker it cannot move off this quarter, and every row has to be scored each night. The row-wise implementation takes 41 minutes. A profile puts each row at about 205 microseconds: 180 of those inside a single call into a compiled expression evaluator, the same whether the row carries one feature column or eight, and 25 in Python-level dispatch and arithmetic around it. An engineer vectorises the arithmetic over the frame's columns and the run drops to about 36 minutes. Which change removes the cost that is left?⌄
MCQ-89A training job decodes JPEGs with Pillow and augments them with numpy before feeding a GPU. On an 8-vCPU host running 8 loader threads, the loader process reports 760% CPU, the NVMe device reads at 120 MB/s (it sustains 500 MB/s), the input queue is almost always empty, and the GPU is fed 152 images/s at 38% utilisation. A synthetic in-memory source shows the GPU can consume 400 images/s. The team wants 400 images/s without changing the per-sample work. What should it do?⌄
MCQ-90A card-fraud model's training job builds txn_count_7d from an event table with ts <= label_ts and ts > label_ts - 7 days, so the transaction being scored sits inside the window. The online service builds the same feature name from Redis counters written strictly before the request, so that transaction sits outside it, and the counters are keyed with a 604800-second TTL. The model's first split is txn_count_7d > 4. Offline AUC is 0.91; in production the same model scores 0.84. A card has 5 transactions in its trailing 7 days, including the one being scored. What value does the service feed the model, and what repair closes the gap?⌄
MCQ-91A Python 3.11 training pipeline has 9 stages, each wrapped by one decorator (with the usual imports):
```python
def timed(fn):
@wraps(fn)
def wrapper(*a, **kw):
t0 = time.perf_counter()
out = fn(*a, **kw)
log.info("stage=%s ms=%.1f", fn.__name__, (time.perf_counter()-t0)*1000)
return out
return wrapper
```
Stage `batches` is a generator that yields minibatches. Its line reads 0.3 ms, while the training loop spends 11 min iterating it. Stage `validate` raises on malformed input, and on those runs the log has no `stage=validate` line. The other stage lines sum to 6.2 min, and a full run takes 18 min. Which change makes each stage's record match the time its work actually takes, and guarantees a line when a stage raises?⌄
MCQ-92A 50M-row pandas DataFrame has to fit in a 1.5 GB worker. Its measured footprint is 4.4 GB: six int64 columns holding values 0–500 (2.4 GB), four float32 columns holding whole numbers from 0 to 1000 (0.8 GB), and three object columns with eight distinct values each (1.2 GB). The model consumes every column and the values it receives must not change. Which plan fits the worker?⌄
MCQ-93You are reviewing a training pipeline proposal for serialising a model for production. The artefact goes to an internal Python batch scorer and to a partner's JVM service that will not run Python. The serving runtime is upgraded quarterly on a schedule the team does not control, the training environment is torn down after each run, and the model's pre- and post-processing is non-trivial and retrained alongside the weights. Which statement should the review accept?⌄
MCQ-94A training job shuffles its index array once per epoch in the main process and hands the fixed order to a DataLoader with `num_workers=8`. The root seed is recorded in the run metadata, and each epoch's seed is derived as `seed = hash(f"{root}:{epoch}") % 2**32`, then `np.random.default_rng(seed).permutation(n)` produces the order. Weight initialization is seeded from the same root and has been verified identical across runs. Two runs with the same root seed on the same machine produce different loss curves from epoch 0, and their first-epoch batch index lists differ; validation accuracy at epoch 40 differs by 0.9 points. What is the cause?⌄
MCQ-95A recidivism model is deployed at threshold 0.5. At that threshold, the per-group table is:
| Group | Selection rate | FPR | FNR |
| --- | --- | --- | --- |
| A | 11.0% | 4.1% | 18.0% |
| B | 11.0% | 9.6% | 41.0% |
Base rates are 8.9% for A and 2.8% for B. The review must decide what to do before sign-off. Which conclusion is supported by the table?⌄
MCQ-96A feature store keeps 14 monthly snapshots and three training sets built from the customer table so backtests stay point-in-time correct. A deletion request removed a customer from the source database, but an audit found the customer's date_of_birth and postcode still in the snapshots and training sets. The DPA requires erasure within 7 days, and the model does not need the raw values. Which change makes the next deletion request complete within the window while keeping point-in-time backtests?⌄
MCQ-97A support-triage ranking model has a registry card listing intended use (desktop web triage), training population (desktop web, EN and DE, 2025-01 to 2026-05), overall AUC 0.844, an operating threshold of 0.91 tuned to hold reviews at 3,000/day, and one slice figure: AUC 0.61 for accounts under 30 days old. A mobile team whose traffic is 55% accounts under 30 days old plans to adopt it at the same threshold. Which conclusion does the card support?⌄
MCQ-98A card-fraud model serves 4,000 req/s behind a feature store. Over 11 days it held 100% availability, p99 latency of 38 ms and a 0.00% 5xx rate, yet the mean predicted fraud probability fell from 0.031 to 0.004. The `merchant_risk_score` feature's null rate rose from 0.4% to 22% after an upstream schema change; the pipeline fills nulls and still returns 200. Labels arrive 14 days late. The only alerts today are on availability, latency and 5xx. Which alert would have caught this regression first?⌄
MCQ-99A fraud-scoring model went live 40 minutes ago and is approving transactions it should decline. The feature pipeline has not changed in three weeks and is serving the same vectors it served before the deploy. The model scores about 110 decisions per minute, and the approvals cannot be unwound once made. Which response order should the team use?⌄
MCQ-100A fraud model shipped in 2023 still scores 0.792 AUC on the trailing 90 days: above the 0.75 floor set at launch and within 0.01 of its launch score, and its input-drift monitor has been quiet for six months. A logistic regression re-fitted on the same window scores 0.804 and costs $120/month against the deployed model's $9,400. The team wants a retirement rule that fires on its own. Which one holds up?⌄