Skip to content
Tech Interview Prep
Technical interview guide

Scalability Fundamentals

Production scalability fundamentals for technical interviews: bottlenecks, scaling, load balancing, autoscaling, capacity, overload control, and failure behavior.

Read
18 min
Practice MCQs
25
Interview Q&A
25
Edition
v7
Editorial status
Reviewed

Scope: Vendor-neutral fundamentals; Google SRE, AWS Well-Architected, Google Cloud, and IETF references accessed 2026-08-30..

Curated: · Written: · Reviewed:

25 questions

MCQ-14A nightly batch job takes 100 s on one worker and 55 s on two. It currently runs on 16 workers. The team proposes doubling to 32. Assuming the serial fraction is fixed and the parallel part scales perfectly, what runtime should they expect at 32 workers?
MCQ-15A request path runs gateway → auth → orders in series, and all three are mandatory. Over the same 30-day window (43,200 minutes) each service recorded 432 minutes of downtime, so the team reports path availability as 0.99³ ≈ 97.03%. Gateway and auth share a rack whose 400-minute outage is counted in both of their downtime figures; no other downtime overlaps. What is the path's actual availability for that window?
MCQ-2A stateless HTTP API runs on one 16-vCPU node and serves 4,000 req/s with node CPU at 65%. The team replaces it with four identical 16-vCPU nodes behind a load balancer. Total throughput rises only to 5,200 req/s, each app node reports 20% CPU, and the p99 write latency of the single Postgres primary they all share has risen from 3 ms to 38 ms. What best explains the result?
MCQ-1A service's CPU is 35%, its database connection pool is saturated (all 20 connections busy, 40 requests waiting), and p99 latency is climbing. The database server's own CPU is 30% and its slow-query log is empty. What should the team scale first?
MCQ-13A payments API serves a steady 2,400 rps with a mean latency of 18 ms and a p99 of 940 ms; its SLO is p99 ≤ 300 ms. Error rate is flat at 0.01% and CPU sits at 40%. Which explanation fits these numbers?
MCQ-4Your team load-tests a checkout API with a p99 latency SLO of 200 ms. The test ramps in 100 RPS steps and records: 900 RPS: p99 180 ms, CPU 70%, DB pool 18/20. 1,000 RPS: p99 210 ms, CPU 78%, DB pool 20/20 with waiters. 1,100 RPS: p99 450 ms, 2% 5xx, DB pool 20/20 with waiters. 2,000 RPS: p99 1.2 s, 18% 5xx. You define the capacity ceiling as the first load step where the SLO degrades and a specific resource saturates. Which RPS should be recorded as the capacity ceiling?
MCQ-3A checkout service runs 8 identical instances behind a round-robin load balancer with no sticky sessions. Each instance keeps the in-flight cart in a `ConcurrentHashMap` keyed by cart id, and the client sends the same cart id on both requests. In load tests, a client that POSTs `/cart/items` and then GETs `/cart` 200 ms later gets an empty cart about 7 times out of 8. Which explanation accounts for that rate?
MCQ-21A checkout service runs three instances behind a load balancer with cookie-based sticky sessions. Carts are stored in application memory, and nothing is written to Redis or a database. What is the main failover risk?
MCQ-5A write path moves from one INSERT per request to batches of up to 100 rows, flushed when the batch fills or after 5 ms. Throughput rises from 8,000 to 40,000 rows/s and p50 latency rises from 2 ms to 12 ms. Which explanation accounts for both numbers?
MCQ-18An e-commerce site runs behind a CDN. Product pages are served from `/product/<id>` with `Cache-Control: public, max-age=300` and `Vary: Accept-Encoding, Accept-Language`. The catalog has 800 product IDs, 3 languages, and 2 encodings. Account pages are served from `/account` with `Cache-Control: private, max-age=0` and `Set-Cookie: session=...`. CDN logs show one cache entry per product ID and a cache hit for `/account`. After launch, users report seeing product pages in the wrong language, and one user's account page was served to another user. Which diagnosis explains both symptoms?
MCQ-6A service runs on four backends, each with a pool of 8 request-handling workers. An L4 load balancer round-robins TCP connections across them. Client X holds one long-lived connection and sends 40 req/s, each occupying a worker for 200 ms. Client Y holds one long-lived connection and sends 400 req/s, each occupying a worker for 1 ms. Backend 1's workers are pegged and its p99 is 3 s; backend 2 sits at 5% utilization and backends 3 and 4 are idle. X's traffic is within contract and must not be throttled. What is the right fix?
MCQ-7A 12-instance API fleet sits behind an L7 load balancer. The health check is GET /healthz every 2 s, timeout 1 s, unhealthy threshold 3, healthy threshold 2. /healthz runs SELECT 1 against the primary database and returns 200 only if the query completes within 500 ms. The database starts rejecting queries at t=0 and recovers at t=8 s. The first failed probe occurs at t=0. How many instances are in rotation at t=5 s, and what is the load balancer's response to clients?
MCQ-23A Kubernetes Deployment runs an API whose liveness and readiness probes call the same handler, which queries the primary database. During a 40-second primary failover the query fails, kubelet restarts every pod, and the Deployment never stabilizes: replacement pods come up, fail the same query, and are killed again. Failovers of up to a few minutes are expected. Which change fixes the restart loop?
MCQ-19An API tier runs 8 replicas, each with a HikariCP pool of maximumPoolSize=20, against a Postgres primary configured with max_connections=200 and superuser_reserved_connections=10. During a latency spike the autoscaler takes the tier to 24 replicas. API p99 rises from 40 ms to 3 s and the primary's CPU stays pinned at 100%. Which explanation fits these numbers?
MCQ-12A service receives 2,000 requests per second. Its retry policy allows up to 3 attempts per request and enforces a retry budget: it will not send more retry attempts than 10% of the requests it receives in a 10-second window. During a 10-second dependency outage, 40% of the requests fail. How many retry attempts does the service send in that window?
MCQ-10A service runs 64 worker threads; each request needs 25 ms of CPU and the client deadline is 300 ms. The gateway in front of it has an unbounded queue and no admission control. During a flash sale, offered load reaches 5000 rps, p99 latency climbs past 8 s, and clients that hit the 300 ms deadline retry, pushing offered load to 7000 rps. What should the team do first?
MCQ-22A payments API is provisioned for a 400 rps peak. A ramp test offers 200 → 1200 rps over 15 minutes. From 200 to 450 rps, p99 holds at 60 ms with no errors. At 450 rps p99 starts climbing; by 700 rps it reaches 350 ms and the per-pod connection pool (20 connections × 6 pods) is fully checked out. From 700 to 1200 rps, completed throughput stays at ~720 rps, p99 flattens at 1.4 s, and the error rate holds at 0.2%. Which conclusion does the run support?
MCQ-17A service publishes events at 12,000/s to a bounded queue of 50,000. A consumer reads at 4,000/s and writes each event to a database through a fixed pool of 20 connections; each write takes 5 ms. Queue depth is rising by 8,000/s and p99 enqueue latency has gone from 40 ms to 180 ms. Which change provides real backpressure for this system?
MCQ-11An API gateway gives each tenant a 300 rps token bucket. The backend fleet is sized for 1,200 rps at p99 50 ms. During an incident one tenant sends 270 rps, inside its allowance, while a downstream dependency slows to 2 s per call. Backend in-flight requests climb to 900 and p99 goes from 50 ms to 2 s. What should the team do?
MCQ-24An API gateway fronts a pool of 8 workers with a combined capacity of 4,000 requests/min. Each API key has a documented allowance of 600 requests/min, enforced by the gateway. During a spike the pool is handling 4,600 requests/min, so the gateway sheds new requests at admission, before any upstream call. A client that has sent 40 requests this minute, well under its 600 allowance, is rejected. What should the gateway return to that client?
MCQ-16A checkout service and a recommendations service share a 20-connection database pool. Normal checkout p99 is 40 ms. During a flash sale the pool saturates: checkout p99 rises to 180 ms and recommendations p99 to 2.1 s. The database CPU is already at 95%. Recommendations traffic is 500 rps and its precomputed cache has a 92% hit rate with 15-minute staleness; checkout inventory must never be stale. Which action best preserves checkout while keeping recommendations available?
MCQ-9A service runs in three availability zones, and the load balancer spreads traffic evenly across whichever zones are up. At peak the service handles 9,000 rps, and one instance sustains 300 rps at the p99 latency SLO. The autoscaler needs about 90 seconds to add capacity, and the SLO is evaluated over one-minute windows. The fleet must be the smallest that still meets the SLO when any single zone is lost at peak. How many instances should be provisioned?
MCQ-25A payments API runs six identical stateless replicas, two in each of three availability zones in one region. Each replica sustains 400 req/s at p99 under 250 ms, and peak traffic is 1,800 req/s. When one zone goes down, the load balancer keeps routing to the four surviving replicas, yet the service returns 503s for the whole outage. What does this show?
MCQ-20A service autoscales on a 60% CPU target, and each instance serves 50 rps at that target. Reactive scale-out takes about 3 minutes end to end (1-minute metric window plus 2-minute instance boot). Traffic sits at 300 rps overnight, ramps from 300 to 1,800 rps between 08:59 and 09:00, and holds until 11:00. The team runs reactive scaling only with a minimum of 6 instances, and p99 latency breaches its SLO for the first few minutes of every morning peak. Which change fixes the breach at the lowest cost?
MCQ-8A service autoscales on CPU. A sudden spike pushes CPU past the threshold, new instances take two minutes to become ready, and requests queue and time out during that window. Which change closes the gap?