Skip to content
Tech Interview Prep
Technical interview guide

Incident Management & Postmortems

Running an incident from detection to resolution, and writing a blameless postmortem that actually prevents a repeat.

Read
30 min
Practice MCQs
25
Interview Q&A
25
Edition
v3
Editorial status
Reviewed

Scope: Google SRE, NIST SP 800-61 Revision 3, NIST CSF 2.0, and CISA guidance current 2026-08-31.

Curated: · Written: · Reviewed:

25 questions

MCQ-12A checkout service's postmortem policy requires a postmortem when any of these holds: user-visible error rate above 1% for more than 5 minutes; any data loss or corruption; any incident detected by a customer rather than by an alert; or any incident requiring manual data repair. On Tuesday a feature-flag rollout made 0.6% of checkout requests return 500 for 18 minutes. The error-rate alert, set at 1%, never fired; a customer posted about the failures and support escalated at minute 12. The on-call engineer turned the flag off and service recovered. No data was lost or repaired. What does the policy require?
MCQ-9At 02:14, checkout p99 latency jumps from 40 ms to 180 ms and error rate from 0.2% to 12% across all regions, about $4k/min in lost orders. The on-call declares Sev-1 and pages payments and platform. At 02:24, traces show every failure carries header X-Feature-Flag: new-tax-calc, which is on for 8% of traffic. Disabling that flag at 02:26 drops error rate to 0.2% and p99 to 45 ms. Your severity matrix: Sev-1 = >5% error rate or full outage; Sev-2 = 1–5% error rate; Sev-3 = <1% error rate with a workaround; Sev-4 = no customer impact. What should the incident commander do with the severity now?
MCQ-1Your team's incident policy declares Sev-1 when the 5-minute error-budget burn rate is at or above 100x, Sev-2 at or above 14.4x, and Sev-3 at or above 2x. The checkout API's SLO is 99.9% successful authorizations and it normally handles 4,000 authorizations per minute. At 10:04, the 5-minute success rate is 98.0%. The service runs active-active in us-east-1 and us-west-2 with a 50/50 traffic split, and the per-region 5-minute success rates are 98.0% in us-east-1 and 98.0% in us-west-2. A deploy at 09:55 changed the payment client's retry budget from 2 to 5. The payment provider's status page reports elevated latency in us-east-1 only, and their support has not confirmed a root cause. The deploy author is reachable. What should the on-call do next?
MCQ-18At 14:10 UTC an incident commander is drafting the next public status update for a checkout outage. The facts: the checkout API has returned 503s for 12% of requests since 14:02 UTC; the payment vendor's public status page confirms that a configuration change at 13:58 UTC caused the failures; a rollback is being validated on 10% of traffic; one customer has reported a duplicate charge, unverified, and no refund policy has been approved. Which line belongs in the update?
MCQ-14During a 42-minute window on 2024-03-12, a bad deploy to the checkout service caused 18,200 requests to the /pay endpoint to return HTTP 500. The service normally handles 1,200 requests/min. The deploy was rolled back at 14:22 UTC. Logs show 6,100 distinct authenticated user IDs hit the failing endpoint; 2,300 of those users retried successfully within 5 minutes. The remaining 3,800 users did not retry during the window. The incident was limited to the EU region; US traffic was unaffected. The team has no data on whether the 3,800 users completed checkout later. Which impact statement is strongest for the postmortem?
MCQ-15A checkout outage postmortem has three sources. The deploy log is NTP-synced and shows checkout v4.7 rolling out at 09:13:50. The pager alert 'checkout p99 > 2 s' is stamped 09:14:22, and the alerting host's NTP log shows its clock was 40 s fast at that moment. The first 'connection pool exhausted' line on the pod is stamped 09:15:03, and the pod's NTP log shows its clock was 25 s slow. Which corrected chronology should the timeline record, and what does it rule out?
MCQ-2A checkout service is failing 38% of requests. Fourteen people are on the incident bridge: three engineers are each editing the same feature-flag config, two are running the same query against the primary database, and the support lead must post a customer update every 15 minutes per the incident SLA. Six minutes in, no task has an assigned lead and no update has gone out. What should the incident commander do next?
MCQ-3A Sev-1 has been open 40 minutes. Checkout error rate is 30% and climbing. The regional failover is two-thirds done, two of three regions now serving from standby, and only the operations lead is tracking it. The incident commander is holding the bridge and a customer call is waiting with no comms lead paged. What should the operations lead do?
MCQ-22A payments team runs a game day for a primary database failover. They assign an incident commander (IC), an operations lead, and a communications lead. At 10:00 the primary DB fails. The on-call engineer, who is not the operations lead, starts running failover commands directly. The IC sees this but says nothing. The comms lead waits for the IC to provide a status update and sends nothing to stakeholders for 25 minutes. The failover completes at 10:18, but stakeholders escalate because they heard nothing. What is the most useful change for the next game day?
MCQ-4A Sev-1 checkout outage has run 40 minutes. The incident commander has spent 25 of those minutes on six executive calls and eleven customer-thread replies, and the replica failover that would restore service has not started. Support has told two customers "no data loss," the status page still reads "investigating," and the account team has promised a fix "within 15 minutes." Which action best restores progress?
MCQ-10A SEV-2 on the checkout API has run since 01:40. The leak in release 4821 exhausts the connection pool at roughly 1,800 concurrent requests. Raising the pool from 20 to 60 connections at 02:10 stopped the errors; overnight traffic is about 900 concurrent requests and peaks near 2,400 at 08:00. At the 06:00 shift change the outgoing commander hands off: "Pool leak in 4821, we scaled the pool to 60, errors stopped, monitoring." The incoming commander closes the incident at 06:05. Which fact, had the handoff carried it, would have shown that closing was wrong?
MCQ-8During a checkout incident, a bad config deploy made 30% of `POST /orders` calls return 500 for 40 minutes (13:40–14:20). The deploy was rolled back at 14:05 and the error rate was back to baseline by 14:20. The client retries a failed call up to 3 times with the same idempotency key, then shows the user an error. Of the 12,000 calls that failed, 8,600 eventually succeeded on retry and the retry queue drained at 14:18. At 14:25 the incident commander wants to close the incident and stand the team down. What should happen first?
MCQ-7At 14:03, checkout p99 latency jumps from 180 ms to 2.4 s and errors hit 12%. The on-call sees pool wait 1.9 s with active connections pinned at 20/20. At 14:11 an engineer restarts checkout pods; p99 falls to 220 ms. By 14:19 p99 is 2.1 s again. The incident channel contains one message: 'Checkout is down because of a connection leak in the new retry code.' Which entry should the incident commander require next in the working log?
MCQ-5At 14:02 a deploy went to the checkout service. By 14:07 the checkout error rate is 12% (baseline 0.3%) and p99 is 4.2 s (baseline 180 ms). Checkout logs show connection-pool acquisition timeouts. The deploy changed the connection-pool limit from 100 to 20; that limit is read from a hot-reloadable config flag. The deploy also changed application code, and rolling the binary back takes about 25 minutes because the previous image was evicted from the node cache. Checkout runs 40 pods, each with its own pool. Traffic is 3× normal from a flash sale. What should the on-call do first?
MCQ-6Checkout runs 60 pods behind a 5% canary (3 pods). At 14:02 a deploy ships two changes: the new pricing path, gated by the `pricing_v2` flag and enabled only on the canary, and a dependency upgrade that closes a known auth bypass. By 14:07 the canary's error rate is 4.3% against a 0.2% baseline, and p99 is 940 ms against 180 ms; the other 57 pods are unchanged. Canary CPU is 22%, the same as baseline. Checkout serves ~2,000 rps, so the canary carries ~100 rps. Flipping `pricing_v2` off takes effect in about 20 s and affects only requests on the new pricing path. Rolling the deploy back needs an image rebuild and rolling restart, about 12 minutes, and also reverts the auth-bypass fix. What should the on-call engineer do next?
MCQ-16A checkout service calls a payments API through an internal load balancer. During a 14-minute incident, the payments API returned 500 within 5 ms for every call: its fraud-check dependency had started erroring, and payments failed fast rather than waiting on it. The load balancer retries any 5xx up to 3 times, so retry traffic peaked at 4× the normal request rate and saturated the payments API's 200-thread pool, which then failed calls that would otherwise have succeeded. Checkout's overall error rate went from 0.1% to 18%. The checkout client itself does not retry and uses a 5 s timeout. Which action item, had it been in place before the incident, would have kept the payments API's thread pool from saturating?
MCQ-13A checkout service's postmortem template has a single 'Root Cause' field. Timeline: 09:12 a deploy ships a retry policy (3 retries, no jitter); 09:40 a flash sale pushes traffic to 600 rps; 09:41 the connection pool (max 20) saturates. Each request holds a connection for 50 ms; normal traffic is 200 rps. The circuit breaker trips at 50% errors over 10 s but counts only 5xx responses, so the timeouts it saw never tripped it. The error-rate dashboard also counts only 5xx and stayed green, so the on-call followed the runbook's first step, saw green, and escalated to the database team. Which entry should the postmortem record?
MCQ-11A schema migration locked the orders table for 11 minutes and pushed p99 checkout latency from 220 ms to 9 s. The timeline shows the engineer ran the script against production because the runbook's staging step had been deleted two releases earlier and the change tool listed no staging target. The draft postmortem's only action item is 'Retrain the engineer on migration procedure.' Which revision best follows blameless postmortem practice?
MCQ-24A payments team runs a postmortem for a 47-minute checkout outage. The write-up names the trigger, a config push that dropped the connection-pool cap, and lists 11 action items; all 11 are marked Done, each linked to a merged PR. Six weeks later the same trigger causes a second outage: the guardrail PR was merged but never deployed to the prod config pipeline. Which conclusion does this evidence support?
MCQ-17A payments team has 90 open postmortem action items. Every item has a named owner and a due date. An audit of on-time closure by owner role found: - 30 items owned by the incident commander (IC): 10% closed on time. - 30 items owned by the on-call engineer who was paged: 80% closed on time. - 30 items owned by a volunteer from the team: 45% closed on time. Seven of the last ten Sev-2 incidents repeated a failure mode whose action item was owned by the IC and was overdue. Which change is most likely to cut repeat incidents next quarter?
MCQ-25A payments team holds a postmortem for a 47-minute checkout outage. The draft lists 34 candidate action items. Over the last four quarters the team closed 2, 3, 1 and 4 action items respectively, and the tracker shows 6% of all items closed within 90 days. The review is scheduled for 60 minutes with 12 attendees. What should the review lead do?
MCQ-23A director sets a quarterly goal: every team must cut its mean time to resolve (MTTR) by 30%, and per-team incident counts go on a shared dashboard. Two weeks later, declared Sev-2 incidents are down 40%, while paging volume, Sev-3 counts and error-budget burn are all flat. Which explanation best fits?
MCQ-20At 11:00 a production host is confirmed compromised. Its memory was captured to an off-host store at 10:50, and its EDR agent streams the in-memory connection buffer to the SIEM every 5 minutes, so the C2 domain is already recorded. The disk has not been imaged. Capturing memory takes 25 minutes, imaging the disk takes 40 minutes, and reimaging takes 10 minutes; the change window closes at 12:00, after which the host must be reimaged to restore service. Which action should the responder take first?
MCQ-19At 09:00 Monday an SRE confirms that a misconfigured storage bucket has exposed 12,000 customer records, including EU residents, to the public internet. The company's incident policy names the privacy counsel as the only role authorized to notify regulators and affected customers. The German customer's contract requires notice within 24 hours of the company becoming aware of a personal data breach. Privacy counsel is unreachable until 08:00 Tuesday. What should the incident commander do?
MCQ-21During a 4-hour outage, an on-call engineer assumed the `break-glass-admin` role through the SSO JIT flow. The permission set grants the role for 24 hours; the role's max session duration is 12 hours. To let two other responders run commands, the engineer pasted the role's temporary STS credentials into the incident Slack channel, which includes two contract engineers. The incident is resolved. What must the cleanup include?