Real Interview Questions for the Machine Learning Engineer at NVIDIA
New
Practice MCQs
Answers count toward the Machine Learning Engineer and overall leaderboards; your latest answer to each question is the one that counts.
MCQ-1A ticket router scores 0.14 log-loss on train and 0.16 held-out; a 4× wider model reaches 0.158, and held-out ECE is 0.11. In Q3 `plan_tier` moved from a 0–100 score to a 0–5 bucket rank, not a linear rescale, and cannot be recomputed for the 900k pre-change rows; post-change held-out AUC fell 0.91 → 0.72, and 120k post-change rows are labelled across 40 queues, the smallest holding 60. Which training and calibration sets should it use?⌄
MCQ-2A 13B decoder (40 layers, 40 query heads, head dim 128) serves 8K-token sessions on 80GB GPUs with 50 GiB of HBM free for the KV cache, which this build pins to BF16: FP8 KV cache is not enabled. The target is 32 concurrent sessions per GPU. An ablation on this checkpoint shows MMLU down 0.0 points at 40 or 20 KV heads, 0.4 at 8 and 2.1 at 4, against a 1-point budget. Which layout fits?⌄
MCQ-3A conv block's input activations, calibrated over 512 batches, sit in `[-0.6, 2.4]` for 99.9% of values, with real signal reaching 4.5; 3 of the batches carry a sensor glitch that spikes to 9.0. The layer is per-tensor Int8, `q = clamp(round(x/scale) + zp, -128, 127)`. Ablations: clipping at 2.4 costs 0.9 top-1 points, clipping at 4.5 costs 0.05, covering 9.0 costs 0.3. Which calibration ships, and what scale and zero point does it give?⌄
MCQ-4A team replaces a loop-based 2D convolution with `np.lib.stride_tricks.sliding_window_view` (NumPy 1.26). The input is a C-contiguous float32 array of shape `(8, 3, 32, 32)`, zero-padded by 1 on each spatial side, then `w = sliding_window_view(xp, (5, 5), axis=(2, 3))[:, :, ::2, ::2]`. What is `w.shape`?⌄
MCQ-5A candidate implements causal attention with a fixed per-head decay bias `b[i,j] = -m·(i-j)` at `m = 1`, added to the logits, then applies the causal mask multiplicatively as `s = s * M` with `M[i,j] = 0` for `j > i`, then softmaxes over keys. At position `i = 2` of a length-6 sequence every raw logit is 0. Which account of that row, and which fix, is right?⌄
MCQ-6A 1T-parameter model trains on 1,024 H100s across 128 nodes with synchronous steps; any node failure kills the job, which restarts from the last checkpoint. Each checkpoint is 14 TB and the parallel filesystem sustains 350 GB/s aggregate, so a write stalls training for 40 s; reload and re-init after a failure add a further 3 minutes. The job sees one node failure every 90 minutes on average. Which checkpoint interval minimizes the expected fraction of wall-clock time lost?⌄
MCQ-7A 1B model trains in FP16 with a static loss scale, and the current scale of `2^16` overflows within a few hundred steps. A calibration pass records the largest gradient element at 5.0 and the smallest nonzero one at 1e-8. FP16's largest finite value is 65,504 and its smallest positive normal is 6.1e-5, with subnormals down to 6e-8. Which static scale keeps every gradient element finite and normal through the backward pass?⌄
MCQ-8A 1.3B transformer trains on one A100 80GB with PyTorch 2.3, BF16, per-GPU batch 16 and sequence length 2048. `nvidia-smi` reports 35% SM utilization. `nsys` shows a 310 ms step: 18,000 kernels totalling 105 ms of GPU time, with the other 205 ms in gaps of about 11 µs between consecutive kernels; the dataset is a preloaded tensor in pinned memory, and `ncu` on the layer norms reports 71% achieved occupancy and 38% DRAM throughput. Which change should the team make first?⌄
MCQ-9A team fits L2-regularized logistic regression (convex) on 2M rows, recomputing the training loss over all 2M rows each epoch. Full-batch gradient descent reaches 0.211 with gradient norm 3e-7. SGD at batch 256, constant `lr=0.5`, reaches 0.219 and oscillates in a ±0.004 band for 20 epochs; a probe at `lr=0.25` gives ±0.002 with mean 0.219, and one at batch 1024, `lr=0.5` gives ±0.002 with mean 0.219. Which account of the band, and the step that follows, is right?⌄
MCQ-10A 40M-parameter U-Net segments 3,000 microscopy frames. Training batches get random 512×512 crops, flips and ±30° rotation; validation resizes each 1024×1024 frame to 512 with no jitter. After 60 epochs augmented training batches score Dice 0.79, validation 0.88, and the same weights score 0.86 when training frames go through the validation pipeline. The lead proposes dropout and weight decay. Which reading and next step are right?⌄
MCQ-11A defect model is validated on 50,000 die images, 250 of them true defects (0.5%). At threshold 0.5 it flags 300 images, 150 of them defects; at 0.2 it flags 1,575, of which 225 are defects. The review queue holds 1,600 flags per 50,000 and the team wants to catch as many defects as the queue allows. Which operating point ships, and what does the move from 0.5 cost per extra defect caught?⌄
MCQ-12A 24-layer ReLU MLP ranks tabular data: 768 input features, 1024 hidden units, no normalization, weights drawn `N(0, 0.1^2)`. After 300 steps the loss is NaN, logged activation RMS grows about 5× per layer, and the gradient norm reaches 3e4. The team re-initializes every layer with one scheme; which std should the first layer's weights use?⌄
MCQ-13A 2M-item retrieval ranker is evaluated offline on a strictly temporal, point-in-time held-out week: AUC 0.87, up from 0.79 for production. In a two-week 5% A/B test the new model serves frozen, and its logged-impression AUC is 0.74. The offline eval draws 100 negatives per positive uniformly from the 2M-item catalog; online the ranker scores only the ~200 candidates retrieval returns. Feature parity is verified to 1e-6. Which diagnosis and next step fit?⌄
MCQ-14A card-fraud model scores AUC 0.96 on a random 80/20 split of 24 months of transactions but 0.68 on a time-based split (train months 1–18, test 19–24). Removing `balance_after_settlement` alone takes the random-split AUC to 0.69; removing any other feature, or refitting the scaler inside the training folds, moves it by under 0.01. Which diagnosis and fix does this support?⌄
MCQ-15A 13B FP16 decoder serves single-stream chat on one L40S (864 GB/s HBM): 1K-token prompts, 200-token answers, p99 inter-token budget 20 ms. Weights are 26 GB, the KV cache 0.8 GB at this context, and a decode step measures 33 ms with HBM at 88% of peak and SMs at 15%. A 1B draft model scores 0.55 acceptance at 4 speculative tokens. Which change brings the step under 20 ms?⌄
MCQ-16A team runs DDP on 32 A100s in 4 nodes of 8. The BF16 gradient buffer is 4 GB per rank, NVLink moves 300 GB/s per direction inside a node, and the inter-node fabric gives 25 GB/s per GPU. NCCL builds one flat ring across all 32 ranks and the all-reduce does not overlap the backward pass; forward plus backward takes 40 ms per GPU. What step time should the flat ring produce?⌄
MCQ-17A 175B decoder (80 layers, hidden 4096, BF16) trains on 64 H100s in 8 nodes of 8. NVLink gives 450 GB/s per direction inside a node, InfiniBand 50 GB/s per GPU between nodes. At micro-batch 1 and sequence 1024 a layer-boundary activation is 8.4 MB; a tensor-parallel all-reduce costs about 2× its size per rank, per-micro-batch compute is 30 ms, and DP gradient all-reduce is not overlapped. With 32 micro-batches per step, which layout gives the lowest steady-state step time?⌄
MCQ-18A 3×3 conv (512→256 channels, 56×56, batch 32, FP16) runs at 1.9 ms on an A100 and feeds a 1×1 conv. Zeroing 64 of its 256 output filters costs 0.2 top-1 points against a 0.3-point budget; forcing 2:4 along the 4,608-wide reduction costs 0.9, since the surviving weights hold 3–4 nonzeros in every group of four. Re-exporting the filter-pruned model still builds a 256×512 dense kernel at 1.9 ms. Which representation and dispatch should ship?⌄
MCQ-19A C++ service transposes 16,384×16,384 float32 matrices (1 GiB, row-major) on a CPU with 64-byte lines, 32 KiB 8-way L1D, 1 MiB L2 and 4 KiB pages. The naive `for i for j B[j*N+i]=A[i*N+j]` sustains 0.9 GB/s; a 32×32 tiled version sustains 7.1 GB/s, a 64×64 tiled version 6.8 GB/s, and adding `_mm_prefetch(&B[(j+1)*N+i], _MM_HINT_T0)` to the 32×32 inner loop drops it to 5.6 GB/s. Which account of the 64×64 and prefetch results is right?⌄
MCQ-20A pipeline service holds 200,000 stage nodes and 1,000,000 dependency edges; each node has at most 4 successors and the set reachable from any node averages 40 nodes. Each edit adds one edge `u → v` and must be rejected if it would close a cycle; edits arrive at 2,000/s with a 0.5 ms budget each, and the full topological order is rebuilt once per scheduling round, not per edit. Which per-edit check rejects exactly the cycle-closing edges inside the budget?⌄
MCQ-21A fraud scorer serves 12,000 req/s across 16 GPU replicas behind one batcher, p99 SLO 50 ms over all traffic. One feature service answers in 12 ms p99, but for 3% of requests blocks 180 ms on a legacy `account_age` lookup before returning the vector; which requests those are is unknown until it answers. The batcher waits for every member's features, 96% of accounts are first seen, and p99 is 210 ms with GPU at 55% and the queue near empty. Which change fixes the p99?⌄
MCQ-22A 7B decoder is LoRA-tuned on 8×H100 with per-GPU micro-batch 2, grad accumulation 4, seq length 4096, BF16, LoRA r=16. Adapter LR is `2e-3`, copied from a full FT run at effective batch 512; batch 64 is fixed by the step budget. Training loss falls 2.0→1.1 by step 500, validation perplexity rises 7.9→18.4. Gradient norm stays 0.6–0.9, the run is 0.4 epochs in, and a 5k held-out slice of the training corpus scores 17.8. Which check or change comes first, and what result confirms it?⌄
MCQ-23A resume bullet claims '3× throughput on the ranker by moving it to TensorRT INT8, GPU call 45 ms → 27 ms.' The deep dive's A100 ablation, preprocessing in the loop, gives: FP32 batch 1 45 ms/call, INT8 batch 1 32 ms/call, FP32 batch 8 60 ms/call, INT8 batch 8 27 ms/call. The tokenizer and feature stage runs once per request, serially before the GPU call, at 17 ms; the same release raised the serving batch from 1 to 8. Which account of the 3× is right?⌄