Skip to content
Tech Interview Prep
Technical interview guideNew

FastAPI Web Services

How FastAPI routes, validates, injects dependencies and serves requests on Starlette and Uvicorn, and what interviewers probe about async handlers, auth, sessions and production workers.

Read
14 min
Practice MCQs
25
Interview Q&A
25
Edition
v1
Editorial status
Reviewed

Scope: FastAPI 0.100 or later (Pydantic v2 support), Python 3.10 or later, current Starlette and Uvicorn releases. Behaviour of response_model, Depends caching and yield dependency teardown is stated for these versions; re-check the official docs if a newer FastAPI major changes them..

Interview Q&A

Treat each question like a live interview question: answer out loud first (structure, assumptions, tradeoffs), then open the model answer to spot gaps and rehearse a tighter follow-up.

Curated: · Written: · Reviewed:

25 questions · Flow follows Relevance until this set has a learning order.

Q&A-1

A teammate says 'FastAPI is a Python web framework that validates requests with Pydantic.' Which parts of a FastAPI service come from FastAPI itself, which from Starlette and which from Pydantic, and why does the split matter when you debug a 422 response?

Answer

The 60-second answer. FastAPI is a layer on top of two libraries. Starlette is the ASGI toolkit underneath: it owns the application object, routing, middleware, requests and responses, and the ASGI server interface that Uvicorn or Hypercorn drives. Pydantic owns data modelling and validation: it turns a JSON body or a typed value into a model instance, or raises a structured error. FastAPI itself adds what sits between them: it reads your function signature and type hints at startup, decides which parameter comes from the path, query, header, cookie or body, builds the Pydantic models for them, calls your endpoint with the validated values, serialises the return value through a response model, and generates the OpenAPI schema from all of that. So when a candidate says FastAPI is a framework, the precise version is: a signature-driven composition layer over Starlette routing and Pydantic validation.

The split matters because a 422 Unprocessable Entity is produced by FastAPI, from a Pydantic ValidationError, for a request that never reached your function body.

# FastAPI 0.115 on Pydantic 2.x
from fastapi import FastAPI
from pydantic import BaseModel, Field

app = FastAPI()

class Order(BaseModel):
    sku: str
    qty: int = Field(gt=0)

@app.post("/orders")
async def create(order: Order, limit: int = 10) -> Order:
    return order

Posting {"sku": "A1", "qty": 0} gives a 422 whose body lists loc: ["body", "qty"], type: "greater_than". The body prefix comes from FastAPI, which knows where the value was read from. The constraint text comes from Pydantic. Your handler never runs. Posting ?limit=abc gives loc: ["query", "limit"], and that is FastAPI's parameter analysis, not Pydantic seeing a body.

Where the layers fail, and how you tell.

  1. Response shape drifts from the declared model. If the handler returns an ORM object without a response model, FastAPI serialises whatever attributes the encoder can reach. Detect: the OpenAPI schema and the actual JSON disagree, and a response_model declared on the decorator is what makes the output filtered and validated.
  2. Blocking code inside async def. This is a Starlette and asyncio problem, not a FastAPI one. The event loop stalls for every request on the worker. Detect: p99 rises across unrelated routes at the same moment, and a plain def endpoint on the same app does not show the stall, since FastAPI runs those in a threadpool.
  3. Middleware ordering. Middleware is Starlette's; FastAPI exposes it unchanged. A 500 that skips your logging middleware was raised by a layer outside it.

When not to lean on the framework. For a service whose inputs are untyped or arrive as a stream, the signature-driven binding gives you less than hand-written parsing with explicit limits. Say so when it matters.

Curated: · Written: · Reviewed:

Q&A-2

In FastAPI, when a query parameter cannot be converted to its declared type, what status code does the client receive, and does the endpoint function run?

Answer

The 30-second answer. The client receives a 422 Unprocessable Entity response, and the endpoint function does not run. FastAPI resolves each declared parameter (path, query, header, cookie, body) and validates it against its type annotation through Pydantic before calling the endpoint. If any parameter fails, FastAPI builds a JSON body listing each error with its location and message, and returns it. The endpoint body is never entered, so it cannot write to a database or call another service.

# FastAPI 0.115, Python 3.12, Pydantic v2
from fastapi import FastAPI

app = FastAPI()

@app.get("/items")
def list_items(limit: int = 10):
    print("endpoint ran")
    return {"limit": limit}

GET /items?limit=abc returns 422 with a body like {"detail":[{"type":"int_parsing","loc":["query","limit"],"msg":"Input should be a valid integer...","input":"abc"}]}, and endpoint ran is never printed. The same applies to a Depends callable: if its declared parameters fail validation, the 422 is raised while that dependency is being resolved, before the endpoint is called.

The status code is part of the framework's contract, not a choice the endpoint makes. A handler that returns its own 400 for a bad query value will only do so if it reads the raw string itself, which bypasses the declared annotation.

How to check it. Send a request with a deliberately bad value and confirm two things: the response status is 422, and the endpoint's own log line is absent. If the endpoint log appears, the parameter is not declared with a type annotation FastAPI can validate, and the value is reaching the function as a raw string.

Curated: · Written: · Reviewed:

Q&A-3

A PUT handler takes shop_id and sku from the path, a PriceIn body, and dry_run and currency (Query, max_length=3) from the query string. The client sends ?dry_run=maybe&currency=EURO, path sku ABC-1, body sku XYZ-9 and amount -3. What does the client receive, does the handler run, and what stops a write to the wrong SKU?

Answer

The 90-second answer. FastAPI decides each source from the signature at startup. shop_id and sku are path parameters because both names appear in the template. body: PriceIn is the body because it is a Pydantic model. dry_run and currency are query parameters because they are scalars that match no path segment. currency is declared with Query(...), so it is required and has a length limit. Validation runs per source and collects every failure, so the client gets one 422 and the handler is not entered. The 422 does not check that the body's sku equals the path's sku. Nothing in FastAPI compares them, and XYZ-9 against ABC-1 passes if the other fields are valid. The handler has to enforce that invariant itself.

For this request, the expected failures are:

[
  {"loc": ["query", "dry_run"], "type": "bool_parsing"},
  {"loc": ["query", "currency"], "type": "string_too_long"},
  {"loc": ["body", "amount"], "type": "greater_than"}
]

The type strings are Pydantic v2 codes. Check them against the pinned version before quoting them. loc is the only reliable way to tell which source failed. shop_id converts to 12 and sku stays a string, so neither produces an error. The sku mismatch produces no error at all.

Why the sources are independent. Each source is validated separately, and the errors are merged. Fixing the query string does not change how the body is read, so a client that fixes one error per round trip will need several attempts. Because the sources are independent, a model that repeats a path field is a design smell. Either take the SKU from the path and drop it from PriceIn, or let the model own it and have the handler compare the two. The first option keeps the contract in one place.

Failure mode. A handler that writes prices[body.sku] instead of prices[sku] stores a price under the wrong item. The request passes validation, returns 200, and the wrong row changes. Detect it with a test that sends mismatched path and body values and expects a 4xx. Without that test, nothing in the stack catches it.

When not to rely on inference. For a public contract, declare Path(), Query() and Body() explicitly and document the SKU rule in the OpenAPI description. A rename of the path segment would otherwise change the source of a parameter without any change to the handler.

Curated: · Written: · Reviewed:

Q&A-4

A PATCH handler takes `patch: UserPatch`, where every field of UserPatch is Optional with default None, and `notify: bool = True` is a query parameter. One client sends no body and no query string. A second sends `{}` and no query string. What does each receive, and why?

Answer

Clarify first: FastAPI 0.100 or later on Pydantic v2. I assume the body parameter has no default of its own.

The two clients differ at the body parameter, not the model. A parameter annotated with a Pydantic model is required unless the parameter itself has a default. Making every field optional only controls what validates once a body exists. So the first client gets a 422 before the handler runs, and the second client gets a 200.

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class UserPatch(BaseModel):
    email: str | None = None
    display_name: str | None = None

@app.patch('/users/{user_id}')
def patch_user(user_id: int, patch: UserPatch, notify: bool = True):
    changes = patch.model_dump(exclude_unset=True)
    return {'user_id': user_id, 'changes': changes, 'notify': notify}
PATCH /users/7 (no body)
422 {"detail":[{"type":"missing","loc":["body"],"msg":"Field required","input":null}]}

PATCH /users/7 with body {}
200 {"user_id":7,"changes":{},"notify":true}

PATCH /users/7 with body {"display_name":null}
200 {"user_id":7,"changes":{"display_name":null},"notify":true}

The third request is the one that matters for a PATCH. Plain model_dump() would return display_name as None, and the handler could not tell an explicit null from an omitted field. exclude_unset=True keeps only the keys the client sent, so {} produces no changes and {"display_name":null} clears the name. Without it, every PATCH would overwrite fields the client never mentioned.

The alternative is patch: UserPatch | None = None. That lets the no-body request reach the handler with patch set to None, and the handler has to decide that None means no changes. The mistake is easy to make, because a bug that treats None as an empty update looks like a success. I would keep the body required, so an empty request fails loudly, and reserve the optional body for endpoints where no body is a valid request.

Notify is the contrast. Omitting it does not fail; it takes the default True, and nothing in the response tells the client the default was applied. If a client that always sends notify=false forgets it after a refactor, emails go out silently. Detection is a metric on the notify value per client version, not an error log.

Curated: · Written: · Reviewed:

Q&A-5

A handler returns a User ORM object with id, email and password_hash. Explain how response_model keeps password_hash out of the response, and why writing -> UserOut on the function is not an equivalent guarantee.

Answer

The 60-second answer. response_model declares the shape of the response. FastAPI takes the handler's return value, validates it against that model, and serializes only the fields the model declares. password_hash is never read into the output, so it never reaches the client. The filter happens after the handler runs, so the secret still exists in memory and in anything the handler logs. A return annotation looks like the same contract, but FastAPI only uses it when response_model is omitted, and it does not tell the reader or the tests what goes on the wire.

# FastAPI with Pydantic v2
from fastapi import FastAPI
from pydantic import BaseModel, ConfigDict

app = FastAPI()

class UserOut(BaseModel):
    model_config = ConfigDict(from_attributes=True)
    id: int
    email: str

@app.get('/users/{user_id}', response_model=UserOut)
def get_user(user_id: int):
    return fetch_user(user_id)  # ORM object: id, email, password_hash

The mechanism. A Pydantic model is validated from the returned object. Pydantic v2 reads an ORM object's attributes only when from_attributes=True is set, so the flag is required here. Without it the response fails validation and the client gets a 500 with a ResponseValidationError in the logs. With it, validation reads id and email by name and ignores everything else, because Pydantic's default is extra='ignore'. The serialized output is built from the validated model, so password_hash is dropped.

Why the annotation is not the same guarantee. In current FastAPI, -> UserOut acts as the response model only when no response_model is passed. Three consequences follow:

  1. Someone can add response_model= later and change the public schema without touching the signature, or the reverse.
  2. The annotation is also what mypy checks against. If the team annotates the function with the ORM class -> User to satisfy the type checker, the annotation is no longer a public contract at all, so the decorator must carry UserOut.
  3. Reviewers read the decorator first when they check what is exposed. A missing response_model on a route that returns a model invites a reviewer to assume the annotation is doing the work, which is only true by default.

Failure mode and detection. The leak to look for is a route that returns an ORM object with no response_model, where the annotation is absent or does not match the field list. Check it with a contract test that calls the route and asserts the key set equals {id, email}. The same test catches the 500 caused by a missing from_attributes flag.

Trade-off. Explicit response_model is easier to audit and keeps the public schema separate from the internal type. It costs a second class to maintain. Annotation-only is shorter, but the schema then depends on the signature, which the next refactor may change. For a route that serves a database object to the public, prefer the decorator.

When it does not apply. A handler that returns a Response object bypasses response_model. For fields that must never be loaded, select only the columns you need in the query.

Curated: · Written: · Reviewed:

Q&A-6

What does Pydantic v2 validation of a model give you that annotating a dict with type hints does not, and what does strict mode change?

Answer

Start with the fact that a type hint is metadata. Python does not check annotations at runtime. user: dict[str, int] = {'age': 'x'} runs without complaint, and so does def f(age: int) called with a string. A Pydantic v2 model turns the annotation into a validator. With class User(BaseModel): age: int, User(age='42') returns an instance whose age is the int 42, and User(age='x') raises ValidationError naming the field, the error type and the input.

Coercion. In the default lax mode, Pydantic converts inputs the target type can represent without loss: '42' becomes 42, 42.0 becomes 42, 'true' becomes True. It rejects what it cannot convert exactly. 42.5 for an int fails with a ValidationError instead of being truncated to 42. The guarantee is that every attribute of a validated model has its declared type, which a bare dict cannot promise.

from pydantic import BaseModel, ConfigDict, ValidationError

class User(BaseModel):
    age: int

User(age='42').age          # 42, an int
User(age=42.0).age          # 42
try:
    User(age=42.5)
except ValidationError:
    pass                    # rejected, not truncated

Strict mode. User.model_validate({'age': '42'}, strict=True) raises, because strict mode accepts only the exact Python type. You can set it per model with model_config = ConfigDict(strict=True) or per field with Field(strict=True). Use strict when a string where an int belongs is a client bug you want surfaced, not a value you want repaired.

Validators. Type hints describe shape only. A field validator enforces a rule: @field_validator('age') can raise when the value is under 18. A model validator checks relations between fields, such as end_date being after start_date. Neither can be written as an annotation, and neither runs on a dict that never passed through the model.

Where it shows up in FastAPI. A body parameter annotated with a BaseModel is parsed from JSON and validated before the handler runs. A failure produces a 422 response with the error list, and the function body is never entered. A parameter annotated as dict[str, int] gets no such check, so the handler receives whatever the client sent.

Failure modes. First, bypassing the model while trusting its output: model_construct skips validation, so its result can carry invalid data. Second, assuming lax coercion is harmless for fields where '1' and 1 mean different things, such as a text-encoded flag or an identifier with leading zeros. Detect it with a test that posts the string form and asserts a 422 under strict mode. Third, expecting validators to see raw input: an after field validator receives the already-coerced value, so an int field's validator gets an int.

Curated: · Written: · Reviewed:

Q&A-7

A route depends on get_db directly, and on get_current_user, which itself depends on get_db. Explain how FastAPI resolves this graph, what it caches within one request, and when you would set use_cache=False on one of these Depends.

Answer

The 90-second answer. FastAPI reads each dependency's signature when the route is registered, not per request. Every Depends(callable) becomes a node, and that callable's own parameters become its children, so the route owns a tree. On each request, solve_dependencies walks that tree depth first, resolves path, query, header, cookie and body parameters at each node, calls the callable, and stores the result in a per-request cache keyed by the callable and its security scopes. A later node that asks for the same callable gets the stored value. use_cache=False skips that lookup, so the callable runs again each time it is injected. I would set it only when a second execution is the point, and I would treat any other reason as a bug.

from typing import Annotated
from fastapi import Depends, FastAPI

app = FastAPI()
calls = []

def get_db():
    calls.append('db')
    yield object()  # stands in for a session

def get_current_user(db=Depends(get_db)):
    return {'id': 7}

@app.get('/orders')
def orders(user=Depends(get_current_user), db=Depends(get_db)):
    return {'calls': calls}

For GET /orders the trace is:

stepnodecacheeffect
1get_db (child of get_current_user)missopens S1
2get_current_user(db=S1)missreturns user
3get_db (route parameter)hitreuses S1

One connection, one call to get_db. Now set Depends(get_db, use_cache=False) on the route parameter. Step 3 becomes a miss, so the request holds S1 and S2 at once and calls ends with two entries.

Where the cache stops. It lives for one request. Two different function objects with identical bodies are two keys, so a factory that returns a fresh closure on every call never shares. Security(get_user, scopes=['a']) and Security(get_user, scopes=['b']) are resolved separately because the scopes are part of the key.

When use_cache=False is justified. Three cases hold up. A dependency that yields a unit of work which must be separate per use, such as an audit write that has to commit even if the main transaction rolls back. A dependency that produces a per-injection value, such as a nonce or a timer, where the caller needs a distinct value at each site. A test or instrumentation hook that needs to count invocations. Reloading data for freshness is not a case: the same request now reads the row twice, and the two reads can disagree.

The failure mode. Hypothetical figures: the SQLAlchemy defaults are pool_size=5, max_overflow=10 and pool_timeout=30 s. A route with use_cache=False on a session dependency checks out two connections per request. At eight concurrent requests that is 16 connections against a ceiling of 15, so the sixteenth waits for up to 30 seconds. Latency looks like a slow query, and the database shows idle sessions. You detect it by counting pool checkouts per request, or by logging the dependency's entry, not by reading the endpoint code.

What is guaranteed and what is observed. The per-request cache and the startup-time graph come from the framework's design and are stable across versions. Teardown timing for yield dependencies changed across FastAPI releases, so check the version you ship before you rely on the order of cleanup relative to the response.

Curated: · Written: · Reviewed:

Q&A-8

A FastAPI endpoint gets its database session from a dependency that uses yield, and the handler raises HTTPException after a flush. Walk me through what runs before the handler, what runs after it, and what the client receives.

Answer

The 60-second answer. FastAPI resolves the dependency graph before it calls the handler. For a dependency written as a generator, FastAPI runs the code above yield and injects the yielded value. The handler then runs with that value. When the handler returns, FastAPI resumes the generator, which runs the code after yield. If the handler raises, the exception is no longer simply propagated past the dependency. In current FastAPI it is thrown into the generator at the yield point, so the dependency's except and finally blocks see it. The client receives whatever the exception handler produces: for HTTPException(409), a 409 response. The exact moment teardown runs relative to sending that response has changed across releases, so the answer depends on the version you pin.

# FastAPI and Python version pinned in the project; check both before relying on this
async def get_session():
    session = SessionLocal()      # before the handler
    try:
        yield session             # handler receives this value
        session.commit()          # reached only if the handler returned normally
    except Exception:
        session.rollback()        # sees HTTPException raised in the handler
        raise
    finally:
        session.close()           # always runs as part of teardown

The trace for the scenario, on a version where the exception is thrown into the generator:

1. get_session() runs to yield; session is opened
2. handler flushes, then raises HTTPException(409)
3. FastAPI throws the exception into get_session at yield
4. except: rollback(), re-raise
5. finally: close()
6. exception handler turns the HTTPException into a 409 response

The commit() line never runs in this trace, which is the point of the pattern. Had the handler returned normally, step 3 would not happen, commit() would run, and close() would follow.

Why the version matters. In older releases, code after yield ran after the response had already been sent. Exception handlers could therefore not see exceptions raised in the handler from inside the dependency, and a rollback written in the except branch could be skipped entirely. FastAPI 0.106.0 changed this so the exception is delivered to the dependency. Later releases have changed the ordering again. I am not certain of the exact version boundaries from memory, so the honest move is to check the release notes for the pinned version instead of trusting a summary. Name that uncertainty out loud in the interview.

Failure modes and detection.

  1. Session leak. A dependency that opens a session and has no finally leaks the connection when the handler raises. Detect it by watching pool checkout counts under a forced error path in a test.
  2. Work after the response. Code after yield that does slow I/O delays connection release on some versions, so latency looks like handler latency. Detect it by timing the handler span and the dependency teardown span separately.
  3. Background tasks holding a yielded resource. A background task that uses the session after teardown reads a closed session. Detect it with a test that runs the task after the response.

Alternative. A context manager around the unit of work inside the handler avoids depending on framework teardown ordering, at the cost of making the handler responsible for the transaction. Choose it when the transaction boundary is smaller than the request.

Curated: · Written: · Reviewed:

Q&A-9

A FastAPI service has one async def route and one def route, and both call a synchronous database driver. Explain where each one runs, what blocks, and what other requests experience under load.

Answer

The 60-second answer. An async def path operation runs as a coroutine on the event loop, so it has to yield whenever it waits. A plain def path operation is sent to a worker thread: FastAPI calls Starlette's run_in_threadpool, which runs the function through AnyIO's thread limiter, 40 tokens by default. The loop keeps serving other connections while that thread waits. So async def is safe only if every call inside it yields to the loop. def tolerates blocking I/O, but each in-flight call holds a thread and any shared state needs guarding.

A synchronous driver such as psycopg2 or a sync SQLAlchemy session inside async def does socket reads that never yield to the loop. The coroutine simply stops, the loop thread is parked until the query returns, and every other request on that worker waits, including requests that never touch the database.

# FastAPI on Python 3.12, timings hypothetical
@app.get('/a')
async def a(db: Session = Depends(get_db)):
    db.execute(text('select pg_sleep(0.5)'))  # loop blocked for 0.5 s
    return {}

@app.get('/b')
def b(db: Session = Depends(get_db)):
    db.execute(text('select pg_sleep(0.5)'))  # runs in a worker thread
    return {}

With one worker and 20 concurrent calls to /a, the last response arrives after about 10 s, and a /health probe sent during that window waits in the same queue. The same 20 calls to /b run in parallel up to the 40-thread limit, so they finish in about 0.5 s and /health answers immediately.

Failure modes and how you catch them.

  1. Event-loop stall from sync I/O in async def. Latency for unrelated routes spikes in step with one route's query time. Detect: event-loop lag metrics, or a probe that measures scheduling delay, and per-route p99 that doesn't match the handler's own timing.
  2. Threadpool saturation from def routes. Latency rises with a flat CPU profile and requests queue before they start. Detect: time spent waiting for a thread token, or in-flight counts that pin at 40. The fix is a smaller pool, an async driver, or fewer blocking calls per request, not a bigger pool on its own.
  3. Shared session across threads. A session created once at startup and used by def routes corrupts transaction state under concurrency. Detect: intermittent wrong rows or errors that only appear under load. Use a per-request session from a dependency.

Trade-offs. An async driver (asyncpg, or SQLAlchemy's async engine) lets async def stay non-blocking end to end, but every helper on the path must also be async. def with a sync driver is the simpler choice for CRUD code, and it bounds concurrency by the thread limit. Neither helps CPU-bound work: under the GIL, a def route that computes still competes with everything else on the process.

Curated: · Written: · Reviewed:

Q&A-10

In FastAPI, a path operation is declared with plain def. How many of these can run at the same time in one worker process by default, and which library enforces that number?

Answer

The 30-second answer. 40. A path operation declared with def is blocking code, so FastAPI runs it in a worker thread instead of on the event loop. Starlette's run_in_threadpool hands the call to AnyIO's thread pool, and AnyIO's default thread limiter allows 40 concurrent threads. Each worker process has its own pool, so 40 is per process. A 41st concurrent def call waits for a free thread.

import anyio
from fastapi import FastAPI

app = FastAPI()

@app.get('/sync')
def handler():          # runs in a worker thread
    return {'ok': True}

print(anyio.to_thread.current_default_thread_limiter().total_tokens)  # 40, inside a running event loop

The async def routes do not use this pool. They run on the event loop and are not capped by this number.

The limit shows up as latency, not errors. Requests beyond 40 queue before the handler starts, so p99 rises while CPU may look idle. Detection: compare total request time with handler time. A growing gap with flat handler time points to thread queueing.

To change the cap, set total_tokens on the default limiter at startup. Raising it costs memory per thread and moves pressure to the database pool or other downstream systems. Adding worker processes multiplies the capacity.

Curated: · Written: · Reviewed:

Q&A-11

Compare running a FastAPI service on one event loop, on a thread pool, and on several worker processes. For an I/O-bound API and a CPU-bound API, which bottleneck does each option remove, and which one does it leave in place?

Answer

The 90-second answer. First establish where the time goes: waiting on a database or a socket, or executing Python bytecode. Each option removes a different bottleneck, and none removes both.

One event loop (one process, asyncio) overlaps waiting. While a coroutine awaits a socket, the loop serves other requests, so an I/O-bound API is limited by how many awaits it can keep in flight, not by core count. It does nothing for CPU work: a coroutine that spends 50 ms in bytecode blocks every other request on that loop for 50 ms.

Threads overlap waiting for synchronous code. Starlette runs def endpoints in AnyIO's worker thread pool, whose default limit is 40 tokens. That helps blocking I/O only. CPython's GIL lets one thread execute Python bytecode at a time and is released during blocking calls, so threads give no CPU parallelism for pure Python. The free-threaded build in CPython 3.13 is opt-in, not the default.

Processes give CPU parallelism. uvicorn --workers 4, or Gunicorn with uvicorn workers, starts four interpreters, each with its own event loop and GIL. The cost: memory per worker, a connection pool per worker, and no shared in-memory state.

Hypothetical: CPU handler 50 ms, 4 vCPUs, one process
  event loop only      ~20 req/s  (3 cores idle)
  40 threads           ~20 req/s  (GIL serialises bytecode)
  4 processes          ~80 req/s  (minus IPC and overhead)

Hypothetical: I/O handler 200 ms waiting on the DB, async driver
  1 process, pool 20   ceiling 20 / 0.2 = 100 req/s  (pool is the limit)
  4 processes, pool 20 ceiling 400 req/s, but 80 DB connections
OptionOverlaps waitingParallel CPUCost
One event loopyes, async I/Onoa blocking call stalls everything
Thread poolyes, sync I/O, capped at 40no, GILqueueing at the cap
Worker processesyes, per workeryesmemory, connections, no shared state

For an I/O-bound API, one process with a true async driver and a correctly sized pool beats threads, and workers only help once the pool is no longer the limit. For a CPU-bound API, the loop cannot help at all and threads cannot help under the GIL, so either add workers sized to cores or move the work to a separate process or service.

Failure modes and detection.

  1. Adding workers to lift throughput when the database is the limit. Connections go from 20 to 80, Postgres max_connections is exhausted, and clients see timeouts rather than a clear error. Detect: pg_stat_activity grouped by client address.
  2. A process-local cache. Four workers hold four caches with different contents, so the same key returns different answers depending on which worker is hit. Detect: a cache hit rate that differs per worker, or a stale read right after a write.
  3. Thread pool saturation from def routes waiting on I/O. At 40 threads all blocked for 200 ms, the queue grows while CPU stays near idle. Detect: rising latency with low CPU and a growing AnyIO limiter wait.

Curated: · Written: · Reviewed:

Q&A-12

Explain the ASGI interface: what the scope, receive and send callables are, and why ASGI replaced WSGI for websockets and streaming responses.

Answer

The 90-second answer. An ASGI application is one awaitable callable, async def app(scope, receive, send), that the server invokes once per connection. scope is a dict describing the connection (type is http, websocket or lifespan, plus path, method, headers as byte pairs). receive and send are coroutines that move events in each direction. The app keeps the connection open for as long as it awaits. WSGI, by contrast, is app(environ, start_response): a synchronous call that returns an iterable. One call is one request and one response, the worker thread is occupied until the body is exhausted, and nothing in the interface can express a client sending data after the response has started. That is why websockets could not be built on WSGI, and why streaming worked only one way and tied up a thread per connection.

# Raw ASGI, Python 3.11, runnable with any ASGI server
async def app(scope, receive, send):
    if scope['type'] == 'lifespan':
        msg = await receive()  # {'type': 'lifespan.startup'}
        await send({'type': 'lifespan.startup.complete'})
        msg = await receive()  # {'type': 'lifespan.shutdown'}
        await send({'type': 'lifespan.shutdown.complete'})
        return
    if scope['type'] == 'websocket':
        await receive()  # {'type': 'websocket.connect'}
        await send({'type': 'websocket.accept'})
        while True:
            event = await receive()
            if event['type'] == 'websocket.disconnect':
                return
            await send({'type': 'websocket.send', 'text': event['text']})

A websocket session is a trace of events on one coroutine:

StepDirectionEvent
1server to appreceive() returns websocket.connect
2app to serversend(websocket.accept)
3client to server, to appreceive() returns websocket.receive with text
4app to serversend(websocket.send)
5client closesreceive() returns websocket.disconnect, the loop exits

The interface itself is the argument. Between steps 3 and 4 the coroutine is suspended, not a thread, so one event loop can hold thousands of idle sockets. Hypothetical figures: 5,000 idle websockets on one worker cost 5,000 suspended coroutines, where a thread-per-connection WSGI design would need 5,000 threads.

Trade-offs. WSGI remains simpler and is fine for short CPU-bound or synchronous request handlers. Under an ASGI server a sync WSGI app can still run, through an adapter that calls it in a worker thread, but it gets no benefit from the long-lived model. FastAPI is built on Starlette, which implements ASGI routing and responses, and adds dependency injection and Pydantic validation above it.

Failure modes. (1) A blocking call inside an async handler, such as time.sleep or a synchronous HTTP client, stops the event loop, so every other websocket on that worker stalls at its next receive. Detect it with event loop lag metrics, or with a p99 that jumps for unrelated routes. (2) Calling send after a disconnect raises, so write loops must catch websocket.disconnect from receive and stop. (3) Skipping the lifespan scope means startup resources such as connection pools are created lazily or never closed, which shows up as a slow first request or leaked connections at shutdown.

Curated: · Written: · Reviewed:

Q&A-13

Explain what Uvicorn, Gunicorn and a process manager each do for a FastAPI service in production, and why one Uvicorn process cannot substitute for running across CPU cores.

Answer

The 90-second answer. Uvicorn is an ASGI server. It accepts HTTP connections, parses them, and runs the FastAPI application on one asyncio event loop inside one OS process. It is the server, not the scaler. Gunicorn is a process supervisor: its master forks worker processes, restarts workers that die, and reloads them gracefully on SIGHUP. For FastAPI the workers are Uvicorn workers, selected with -k uvicorn.workers.UvicornWorker. Recent Uvicorn releases point to the separate uvicorn-worker package for that class, so check the version you pin. A process manager such as systemd, or the orchestrator in Kubernetes, sits above all of this: it keeps the Gunicorn master alive, applies memory and CPU limits, and restarts the whole unit. Scaling across cores comes from several OS processes. One event loop runs on one thread, and the GIL stops threads from executing Python bytecode in parallel, so coroutines inside one process let more requests wait on I/O at once but add no CPU.

Mechanism, with hypothetical figures. Assume a handler does 20 ms of pure-Python CPU work and no I/O. One process keeps one core busy, so its ceiling is about 1 / 0.020 = 50 requests per second, and p99 grows as the queue builds once arrivals exceed that rate. Four processes on a four-core VM raise the ceiling to roughly 200 requests per second, before the database, the network and the proxy take their share.

# hypothetical: 4-core VM, 20 ms pure-Python CPU per request
1 Uvicorn process : 1 loop, 1 core busy   -> ~50 req/s ceiling
4 processes       : 4 loops, 4 cores      -> ~200 req/s ceiling
gunicorn app.main:app -k uvicorn.workers.UvicornWorker -w 4 -b 0.0.0.0:8000

The -k flag matters. Gunicorn's default worker class is sync, which would run the ASGI app in a way it was not built for. Do not size -w from the older 2n+1 rule, which was written for sync workers. Start near the core count, then measure.

Trade-offs. uvicorn --workers N gives you a supervisor with fewer moving parts. Gunicorn adds graceful reload, max_requests recycling against slow memory growth, and worker timeouts, which is why many teams keep it. Alternatively, run one process per container and scale replicas in Kubernetes. The orchestrator then handles restarts and placement, and an in-container supervisor adds little. Each replica carries its own memory footprint, so the choice is about where you want the supervision logic to live, not about throughput.

Failure modes and how you catch them.

  1. Per-process state. In-memory caches, rate limiters, clients created in lifespan and background tasks exist once per worker. A limit of 100 requests per minute per user becomes up to 400 with four workers that each count separately. Detect it with a load test that sends one user's requests across workers, or move the counter to Redis.
  2. Oversubscription. Four workers each with a database pool of 20 open 80 connections. Against a Postgres max_connections of 100 shared with other services, the symptom is connection errors or pool-wait spikes in the database's connection metrics.
  3. Memory multiplied by worker count. Copy-on-write sharing of the imported app erodes as refcounts touch pages. Measure resident memory per worker and multiply by the count before you pick -w.
  4. A crashed worker drops its in-flight requests. The master replaces it, but clients see resets, so only idempotent calls should be retried automatically.

Curated: · Written: · Reviewed:

Q&A-14

With app.add_middleware(A) followed by app.add_middleware(B), which middleware sees an incoming request first, and why can a middleware that reads the request body leave the route's body parameter empty?

Answer

Ordering. Each add_middleware call wraps the application built so far, so the last middleware added is the outermost. With app.add_middleware(A) followed by app.add_middleware(B), a request reaches B first, then A, then the router and the route. The response unwinds in reverse, so A's code after its await runs before B's.

from fastapi import FastAPI, Request

app = FastAPI()

@app.middleware('http')
async def outer(request: Request, call_next):
    print('outer in')
    response = await call_next(request)
    print('outer out')
    return response

@app.middleware('http')
async def inner(request: Request, call_next):
    print('inner in')
    response = await call_next(request)
    print('inner out')
    return response

inner is registered second, so it is outermost. For a GET request the output is inner in, outer in, outer out, inner out. Print it in your own environment to confirm the order on your Starlette version.

Why a body can go missing. In ASGI the request body is not stored anywhere. It arrives as http.request messages from receive, and each message can be consumed once. Code that calls receive() and discards the messages leaves the route with nothing to read. A raw ASGI middleware that drains receive without handing the bytes back produces exactly this, and a Pydantic body parameter then fails validation with a 422 reporting a missing body.

The decorator form behaves differently. Starlette's BaseHTTPMiddleware, which backs @app.middleware('http'), caches a body read through request.body() and replays it to the downstream app, so reading the body there does not normally empty the route's parameter. This has changed across Starlette releases, so verify it against the version you pin rather than relying on this description.

Fixes. Read the body in a dependency or in the route when the logic belongs there. If a pure ASGI middleware must inspect the body, collect every http.request chunk once, keep the bytes, and wrap receive so downstream code gets the same content. Cap the buffered size, because the whole body sits in memory until the request finishes.

Trade-off. The 'http' decorator wraps every HTTP response, which makes it a poor fit for streaming, and it never sees websocket scopes. A pure ASGI middleware receives every scope type and can pass streaming responses through untouched, at the cost of more code to write and test.

Curated: · Written: · Reviewed:

Q&A-15

Explain why an HTTPException and a custom exception handler leave different traces in the logs, and what an unhandled exception does to the response when CORSMiddleware is installed?

Answer

The 90-second answer. HTTPException and a custom domain exception are both control-flow signals. Neither reaches the client as a Python exception: a handler turns each into a response. What differs is which layer catches it and whether anything gets logged. A handled exception is silent. Only an exception that escapes every handler is logged with a traceback, and the client then gets a bare 500 that browsers may report as a CORS failure. A catch-all Exception handler does not remove that logging, so it changes the response without making the failure any less visible on the server.

Mechanism. FastAPI builds on Starlette's middleware stack, outermost first: ServerErrorMiddleware, your user middleware (CORSMiddleware, for example), ExceptionMiddleware, then the router. ExceptionMiddleware looks up a handler by walking the raised class's MRO. FastAPI's HTTPException subclasses Starlette's, and the default handlers cover it and RequestValidationError. A handler's return value becomes the response and nothing is logged. ServerErrorMiddleware handles only exceptions registered for Exception or the 500 key. It sends a response, then re-raises, so Uvicorn logs 'Exception in ASGI application' with the full traceback.

from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

class OrderNotFound(Exception):
    def __init__(self, order_id: str):
        self.order_id = order_id

app = FastAPI()

@app.exception_handler(OrderNotFound)
async def order_not_found(request: Request, exc: OrderNotFound):
    return JSONResponse(status_code=404, content={'detail': 'order not found'})
RaisedCaught byClient receivesTraceback logged
HTTPException(404)ExceptionMiddleware, default handler404, JSON with detailno
OrderNotFound, handler registeredExceptionMiddleware, custom handler404 from handlerno
OrderNotFound, no handlerServerErrorMiddleware500, text/plain 'Internal Server Error'yes
ValueError in a routeServerErrorMiddleware500, text/plainyes

The catch-all trap. Registering a handler for Exception moves the 500 path into ServerErrorMiddleware with a custom body, which is useful. But that middleware sits outside CORSMiddleware, so the 500 carries no Access-Control-Allow-Origin header. The browser blocks the response and shows a CORS error, which hides the server failure from the frontend team. The exception is still re-raised, so the traceback appears in the logs too. If you want the request id in that log line, you log it yourself inside the handler.

Streaming. If a StreamingResponse has already sent its headers, ServerErrorMiddleware cannot send a 500. It re-raises and the connection is cut, so the client sees a truncated body and no status it can act on.

Detection. Search logs for 'Exception in ASGI application'. Compare 500 counts in the access log with the traceback count. Check CORS on errors with curl -i -H 'Origin: https://app.example.com' against a route that raises.

Curated: · Written: · Reviewed:

Q&A-16

A POST /orders handler saves the order, then adds a BackgroundTasks job that charges the card and sends a receipt. When does the client get its response relative to that job, what happens to the job if the worker is killed mid-run, and why is this not a durable queue?

Answer

The 60-second answer. BackgroundTasks runs its callables after the response has been sent, inside the same worker process and the same request's ASGI call. The client gets its status code and body first. The job then runs in-process, with no record that it was accepted, no retry, and no way to resume if the process dies. So it is fine for work the system can afford to lose or redo (cache warm-up, a log line, a best-effort webhook), and wrong for a charge that must happen exactly once.

The ordering comes from how Starlette sends a response. Response.__call__ writes the headers and body through send, then awaits self.background(). FastAPI wires your bg.add_task calls into that object. Tasks run sequentially in the order added. A def task is pushed to the threadpool and an async def task runs on the event loop, so a slow synchronous task occupies a threadpool slot while the client has already gone.

# Python 3.12, FastAPI 0.115, Starlette underneath
from fastapi import BackgroundTasks, Depends, FastAPI

app = FastAPI()

def charge_and_receipt(order_id: int, card_token: str) -> None:
    # Calls the payment API (about 900 ms in this hypothetical trace), then the mailer.
    ...

@app.post('/orders', status_code=201)
async def create_order(body: OrderIn, bg: BackgroundTasks, db=Depends(get_db)):
    order = await db.insert_order(body)
    bg.add_task(charge_and_receipt, order.id, body.card_token)
    return {'id': order.id}

A trace with hypothetical figures:

t (ms)Event
0Handler returns; the order row is committed
12201 and body are written to the socket; client sees success
12 to 912charge_and_receipt runs in the worker
300Pod receives SIGKILL (deploy or OOM)

At 300 ms the client already holds a 201. The charge request may or may not have reached the processor, and nothing in your system records which. The task list lived in memory, so there is nothing to replay. A graceful shutdown with a long enough timeout can let the task finish, but you cannot count on that for SIGKILL, OOM or a node loss.

Two other failure modes matter in review. If the task raises, the client still got 201. The error appears only as a server log line, so nobody retries it. And the task shares the worker's CPU and threadpool with live traffic, so a burst of tasks slows the routes that the job was meant to stay out of the way of, with no queue depth to watch.

Why it is not a durable queue. A durable queue gives you persistence before acknowledgement, a visibility timeout or lease, retry with a bound, a dead-letter destination, and a depth metric. BackgroundTasks has none of these. The fix for money-moving work is to write an outbox row in the same transaction as the order, then have a separate worker (Celery, RQ, SQS consumer, or a polling loop) claim rows, call the processor with an idempotency key, and mark them done. The HTTP handler then returns a job or order id, and the client or a webhook learns the outcome later.

Curated: · Written: · Reviewed:

Q&A-17

Walk me through the OAuth2 password flow in a FastAPI service using OAuth2PasswordBearer: what the token endpoint receives and returns, and how a protected route gets the token.

Answer

The 90-second answer. The password flow has two halves that FastAPI wires together but does not perform. The token endpoint receives a form-encoded username and password, checks them, and returns a JSON body with access_token and token_type. A protected route declares OAuth2PasswordBearer as a dependency. That dependency reads the Authorization header, requires the scheme Bearer, and hands the raw token string to the handler. It does not decode, verify or look up the token. Validating the token is the route's job, through a function you write. The tokenUrl argument only tells the OpenAPI docs where to send login requests.

from fastapi import Depends, FastAPI, HTTPException
from fastapi.security import OAuth2PasswordBearer, OAuth2PasswordRequestForm

app = FastAPI()
oauth2_scheme = OAuth2PasswordBearer(tokenUrl='token')

@app.post('/token')
def login(form: OAuth2PasswordRequestForm = Depends()):
    user = authenticate(form.username, form.password)
    if user is None:
        raise HTTPException(status_code=400, detail='Incorrect username or password')
    return {'access_token': issue_token(user), 'token_type': 'bearer'}

@app.get('/me')
def me(token: str = Depends(oauth2_scheme)):
    return decode_token(token)  # raises 401 on bad signature or expiry

What each piece does. OAuth2PasswordRequestForm reads username, password and optional scope, grant_type, client_id and client_secret from an application/x-www-form-urlencoded body. It needs python-multipart installed, or FastAPI refuses to start the route. The /token response is not a framework default. You build the dict, and token_type should be bearer so clients know to send Authorization: Bearer <token>.

Extraction. On each request to /me, FastAPI runs oauth2_scheme before the handler. If the header is missing, or the scheme is not Bearer, the dependency raises 401 with a WWW-Authenticate: Bearer header, and the handler never runs. If the header is present, the handler receives the string after Bearer .

Failure modes.

  1. Treating the dependency as authentication. A route that uses oauth2_scheme but never decodes the token accepts any non-empty string. Detect it with a test that sends Bearer garbage and expects 401.
  2. Expired or forged tokens passing silently. decode_token must check the signature and the exp claim. PyJWT's jwt.decode does both when given an algorithm list, but the list must be pinned, or an attacker-chosen alg can be accepted.
  3. Sending credentials over plain HTTP. The password is in the request body in clear text. TLS is the only protection, and the server must not log the form.
  4. Assuming tokenUrl is a route. It is a string for the docs UI. Rename /token without updating it and the Authorize button breaks, but the API still works.

Two things I would check before relying on this: whether the login endpoint is rate limited, and whether the 400 versus 401 choice for bad credentials matches what your clients expect.

Curated: · Written: · Reviewed:

Q&A-18

Which claim in a JWT's payload is the standard one a server checks to reject a token after its lifetime has passed?

Answer

The 60-second answer. The exp claim (expiration time) is a NumericDate, the number of seconds since the Unix epoch, after which the token must not be accepted. The server compares it with its own clock on every request and returns 401 once the current time is at or past exp. A token with no exp never expires on its own, which is why libraries that accept one by default are a finding in review.

import time
import jwt  # PyJWT 2.x

claims = {"sub": "user-42", "exp": int(time.time()) + 900}  # 15 minutes
token = jwt.encode(claims, PRIVATE_KEY, algorithm="RS256")

# Verification raises jwt.ExpiredSignatureError once exp has passed.
jwt.decode(token, PUBLIC_KEY, algorithms=["RS256"])

The signature does not change this. A token can be correctly signed and still expired, so exp is a separate check the server performs after the signature is verified. Clock skew between issuer and verifier is the usual reason a fresh token is rejected, and most libraries accept a small leeway for it, which you should set deliberately rather than leave large.

Failure mode and detection. A service that decodes with options={"verify_exp": False}, or that never sets exp at issue time, keeps accepting a stolen token indefinitely. Detect it by minting a token, waiting past the lifetime, and expecting a 401. Note that exp only bounds how long a token is valid; it does not make a token revocable before that time.

Curated: · Written: · Reviewed:

Q&A-19

A FastAPI service creates one SQLAlchemy session at module import and every route uses it. Explain how you would give each request its own session, where it opens and closes, and what breaks under concurrent load.

Answer

The 90-second answer. A session is a unit of work: it holds a transaction, an identity map of loaded objects, and at most one connection while it is in use. It is not thread-safe, and it is not a global. So I create the engine once at import, and create a session per request through a dependency that uses yield. The code before yield opens the session, the handler runs with it, and the code after yield closes it. FastAPI caches a dependency within one request, so the route and get_current_user share one session and therefore one transaction.

# FastAPI 0.115, SQLAlchemy 2.0, Python 3.12
engine = create_engine(DSN, pool_size=5, max_overflow=10, pool_timeout=30)
SessionLocal = sessionmaker(bind=engine, expire_on_commit=False)

def get_db():
    db = SessionLocal()          # cheap: no connection is checked out yet
    try:
        yield db
    finally:
        db.close()              # returns the connection to the pool, rolls back any open transaction

The engine is shared and safe to share; it owns the pool. The session is cheap and per request. Pool arithmetic matters: with the settings above, 15 concurrent checkouts is the ceiling per process, and the 16th waits up to 30 s and then raises TimeoutError. A def route runs in Starlette's threadpool, which defaults to 40 threads, so the pool, not the thread count, is usually the first limit you hit.

The import-time failure.

db = SessionLocal()   # created once, at import

@app.post('/orders')
def create_order(o: OrderIn):
    db.add(Order(**o.model_dump()))
    raise HTTPException(409, 'sold out')   # no rollback, no close

Trace with two concurrent requests. Request A adds an order to the shared session and then raises. Nothing rolls back, so the pending insert stays in db. Request B, a read, flushes the session for its own query and writes A's order. The client of A received a 409 and the row exists. If A's failure happened during flush instead, the session is left in a state where every later call raises PendingRollbackError until someone calls rollback(), so one bad request poisons unrelated requests. Identity-map state leaks too: B reads an entity A already loaded and modified, and sees A's uncommitted values.

Two more failures appear with more processes. uvicorn --workers 4 imports the app in each worker, so each gets its own session, which hides the bug in a single-worker test. Gunicorn with --preload imports before forking, so the children inherit one connection opened in the parent, and two processes write to the same socket. That corrupts the protocol stream and shows up as garbled results or dropped connections, not a clean exception.

Detection. Run a barrier test: two requests released together, one that writes then fails, one that reads. Assert the reader never sees the writer's row. In production, alert on PendingRollbackError and on pool TimeoutError rates, and log the session's in_transaction() state at request end.

Version note. When the exit code of a yield dependency runs relative to response sending changed across FastAPI releases. Check the version you run before claiming a timing; the rule that the code after yield runs on success and on exception holds in the versions I checked.

Curated: · Written: · Reviewed:

Q&A-20

Your team's pytest suite hits a FastAPI route that depends on get_db, and the tests write to the real database. How would you point that route at a test session with app.dependency_overrides, and why is overriding get_db better than monkeypatching the ORM module?

Answer

The 90-second answer. A route that declares db: Session = Depends(get_db) gets its session from whatever callable FastAPI resolves for get_db at request time. app.dependency_overrides is a dict keyed by the original callable. When the router resolves a dependency, it checks that dict first and calls the replacement instead. So the test does not change the route, the ORM, or the import graph; it swaps one callable for another at the one seam the framework already exposes.

# FastAPI 0.115, Pydantic v2, SQLAlchemy 2.0
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from app.main import app
from app.deps import get_db

test_engine = create_engine('sqlite:///./test.db')
TestingSession = sessionmaker(bind=test_engine)

def override_get_db():
    db = TestingSession()
    try:
        yield db
    finally:
        db.close()

@pytest.fixture
def client():
    app.dependency_overrides[get_db] = override_get_db
    with TestClient(app) as c:
        yield c
    app.dependency_overrides.clear()

The override must be the same function object as the one the route passes to Depends. Import it from the module that defines it; a copy or a wrapper will silently not match, and the real database is hit with no error. clear() in teardown stops one test's override leaking into the next.

Why not patch the ORM module. monkeypatch.setattr('app.db.SessionLocal', ...) replaces a name in one module, but the route reaches the session through get_db, which captured SessionLocal by name at import, or through a second module that did from app.db import SessionLocal before the patch ran. The patch works for some call sites and misses others, and which ones miss depends on import order. The failure is quiet: the test passes against the wrong database.

Patching the ORM also couples tests to internals. Rename SessionLocal, move the engine, or add a connection pool argument, and every patch breaks, even though the HTTP contract did not change. Overriding get_db tests the contract the route actually declares: "this endpoint needs a session with this interface."

Trade-offs and when not to use it.

  • Overriding a dependency that you do not control in the test (a third-party client) is still the right seam, but you own the fake's behaviour, so keep it thin.
  • If the code under test runs queries outside a request (a background task, a CLI), the override does not apply; those paths need their own injection.
  • An override that returns a mock hides SQL errors. Use a real database (SQLite or a Postgres container) for query logic and reserve mocks for dependencies whose behaviour you do not need to exercise.

Failure modes.

  1. Override key mismatch: the test passes, but assertions show rows from the dev database. Detect by asserting that the test database file or schema is the one written, or by failing fast in a conftest check that get_db is overridden.
  2. Stale override across tests: a test that forgets clear() makes later tests use a closed session. Detect by ordering failures, DetachedInstanceError or Session is closed.
  3. Dependency cache: within one request FastAPI caches a dependency by default, so an override applied mid-request has no effect. Set overrides before the client issues the request.

Curated: · Written: · Reviewed:

Q&A-21

Your FastAPI test suite calls an async def route through TestClient and it passes, yet the same route stalls under load after a deploy. Which parts of the request path did the test run, which did it skip, and what one check would you add before trusting the suite for a release?

Answer

The 90-second answer. TestClient does not open a socket. It hands the ASGI app a scope, a receive callable and a send callable, and the request runs through middleware, routing, dependency resolution, validation and response serialization just as it would in production. What it skips is everything below the ASGI boundary: Uvicorn's HTTP parsing, keep-alive, TLS, proxy headers, server timeouts and the worker process model. A stall under load is a property of the running event loop, and one sequential test request cannot show it. So I would keep TestClient for application behaviour and add one concurrent check against a real Uvicorn process before the release.

Mechanism. Starlette's TestClient runs the app on its own event loop in a background thread, and each client.get submits the request to that loop through an anyio portal and waits for the result. The test function itself stays synchronous. Lifespan runs only inside with TestClient(app) as client:. Without the context manager, startup code never runs, so the test exercises an app with no database pool and no warm caches.

# Python 3.12, pytest, FastAPI with Starlette's TestClient
import time
from fastapi import FastAPI
from fastapi.testclient import TestClient

app = FastAPI()

@app.get('/partner/{sku}')
async def partner(sku: str):
    time.sleep(0.8)  # blocking call inside async def
    return {'sku': sku}

def test_partner():
    with TestClient(app) as client:
        assert client.get('/partner/abc').status_code == 200

The test passes in about 0.8 s. Now take a hypothetical load of 50 concurrent requests on one Uvicorn worker. The sleep holds the event loop, so the requests run one after another and the last one finishes after about 50 × 0.8 s = 40 s. The sequential test sends one request at a time and never sees the queue.

What the single check is. Run the same route under Uvicorn with a fixed concurrency, for example 50 connections for 30 seconds, and compare p99 with the single-request time. If p99 grows roughly with concurrency, the handler is blocking the loop. The fix is either an awaitable client or a plain def route, which FastAPI runs in its thread pool.

Other things the live run catches. TestClient reports the client address as 'testclient' and the scheme as http. Code that reads the client IP or scheme behaves differently behind a proxy that Uvicorn trusts, and only a run with forwarded headers shows that. Graceful shutdown and signal handling also run only in the real server.

When to choose which. Use TestClient for routing, validation and dependency behaviour, which is most of the suite. Use httpx.AsyncClient with ASGITransport when a test must share an event loop with other coroutines. ASGITransport does not run lifespan, so start it yourself in that case.

Curated: · Written: · Reviewed:

Q&A-22

In FastAPI, where does the OpenAPI document that /docs displays come from?

Answer

Opening answer. The document is built from the route declarations, not from the function bodies. When /openapi.json is first requested, FastAPI calls app.openapi(), which walks the registered routes and turns each decorator, parameter annotation, Pydantic model and response declaration into OpenAPI objects. It caches the result on the app object, and Swagger UI at /docs fetches that JSON and renders it. The document describes what each route promises. It does not record what a particular call returns.

The mapping works like this. The decorator gives the path and method. Function parameter annotations give path and query parameter types. A Pydantic model parameter becomes the request body schema. response_model, or the return annotation when response_model is omitted, becomes the response schema.

# FastAPI and Pydantic v2
from fastapi import FastAPI
from pydantic import BaseModel

class ItemOut(BaseModel):
    id: int
    name: str

app = FastAPI()

@app.get("/items/{item_id}", response_model=ItemOut)
def read_item(item_id: int):
    return {"id": item_id, "name": "kettle"}

The document lists item_id as an integer path parameter and gives a 200 response whose schema is ItemOut. Nothing in the body of read_item is inspected. Two consequences follow. A return annotation of dict or Any produces no useful response schema, so put the shape in response_model. And a handler that returns data failing ItemOut does not contradict the docs. It fails at runtime: FastAPI validates the return value against response_model and raises a ResponseValidationError, which surfaces as a 500. Detect that with a test that calls the route through TestClient and checks the status code, not with a read of the schema.

Alternatives and when to pick them.

ApproachUse whenCost
Generated from annotations (default)The code is the source of truth and consumers can tolerate the generated namesSchema detail is limited to what annotations and Pydantic express
Per-route openapi_extraA few operations need extra keys such as examples, tags or vendor extensionsHand-written dicts can drift from the models they sit beside
Override app.openapi()Many operations need a shared transformation, such as a custom info block or a security scheme added globallyYou own the function, and it must be kept in step with FastAPI upgrades
Schema-first (write the spec, generate server stubs)Several teams or languages must agree on the contract before any code existsTwo artefacts to keep in sync, and the spec can describe behaviour the code does not implement

For most services the default is correct. Schema-first makes sense when the contract is a product in its own right and external consumers generate clients from it. Overriding app.openapi() is a reasonable middle ground when you need one global change, but it hides the schema from the route declarations that a reviewer reads first.

Checking it. Export the document in CI and diff it against the committed version, so an unintended model change shows up as a reviewable change. Calling app.openapi() directly in a test avoids starting a server. A diff catches shape drift; it does not catch a handler that returns the wrong data, which only a behavioural test will find.

Curated: · Written: · Reviewed:

Q&A-23

A 4-core host runs a CPU-light, I/O-bound FastAPI service under Gunicorn with Uvicorn workers. How many workers do you start, and what limits how far you can raise that number?

Answer

Start from the arithmetic, not the rule of thumb. The 2N+1 guidance comes from synchronous WSGI workers, where each worker holds a core for the whole request. An async worker runs one event loop, so while a request waits on the database, that same process serves others. For a CPU-light, I/O-bound service on 4 cores, the starting point is 4 workers, one per core. I would not go much above that until the memory and connection budgets are checked.

The first constraint is memory, and it is paid per process. Each worker imports the app, the Pydantic models, the ORM mappers and the HTTP client. For a mid-sized service that is commonly 100 to 250 MB resident (hypothetical figure for this example: 180 MB each). With 4 workers that is about 720 MB, and with 8 it is about 1.4 GB. --preload lets the Gunicorn master import the app once and then fork, so many pages start shared under copy-on-write. That saving erodes: CPython writes to an object's header when its reference count changes, so pages get dirtied as traffic arrives and each worker drifts toward its full footprint. Measure USS or PSS per worker, not RSS, because RSS counts shared pages in every process.

The second constraint usually bites first: connection pools are per process. Four workers, each with pool_size=5 and max_overflow=10, can hold 60 database connections. At 8 workers the ceiling is 120, which is above PostgreSQL's default max_connections of 100 before a migration job or an admin session connects. The number you tune is the per-worker pool; the number the database sees is the product of that and the worker count.

Third, anything held in process memory exists once per worker. An LRU cache, a JWKS cache or an in-process rate limiter gets a hit rate that falls as workers rise, and a limit of 100 requests per minute per process becomes 100 times the worker count across the fleet. Lifespan handlers also run once per worker. A scheduler started in lifespan fires four times unless it is guarded by a lock or moved into its own process.

The trade-off: extra workers isolate one stuck process, put idle cores to use when some request does CPU work such as serialising a large response, and contain a blocking library call that slips into an async route. They cost memory, connections and cache hit rate. Fewer workers with a busy event loop is the opposite bet, and it fails sooner if any handler blocks.

A sensible command is gunicorn app.main:app -k uvicorn.workers.UvicornWorker -w 4 --preload --max-requests 5000 --max-requests-jitter 500 --timeout 30 -b 0.0.0.0:8000. --max-requests with jitter recycles workers to bound slow leaks, at the cost of a brief capacity dip and a cache reset on each recycle.

To decide, load test at fixed concurrency and watch p99 and pool checkout wait, not CPU. For an I/O-bound service, throughput rises with concurrency until the pool or the database saturates, and SQLAlchemy's QueuePool limit reached timeout appears well before the cores are busy. Adding workers past that point only moves the wall.

What is exact here: the pool multiplication, the per-process memory model and the lifespan behaviour. What is an observation: the 180 MB figure, and how far copy-on-write survives in your workload, which depends on the CPython version and on what the app touches.

Curated: · Written: · Reviewed:

Q&A-24

A FastAPI service creates an httpx.AsyncClient inside each handler to call a partner API. Explain what that costs per request, and how you would restructure it so one client is created per process and closed cleanly on shutdown.

Answer

The 90-second answer. A client created per request pays for work that has nothing to do with the request: a new connection pool, a TCP handshake and a TLS handshake to the partner before the first byte of the real call. Connection reuse only helps if the pool survives between requests, so the fix is one client per process. Create it in the lifespan hook before the server accepts traffic, keep it on app.state, and close it after the server has drained in-flight requests.

# Python 3.12, current FastAPI and httpx
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
import httpx

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.partner = httpx.AsyncClient(
        base_url='https://partner.example.com',
        timeout=httpx.Timeout(5.0, connect=2.0),
        limits=httpx.Limits(max_connections=100, max_keepalive_connections=20),
    )
    try:
        yield
    finally:
        await app.state.partner.aclose()

app = FastAPI(lifespan=lifespan)

@app.get('/quotes/{sku}')
async def quote(sku: str, request: Request):
    r = await request.app.state.partner.get(f'/prices/{sku}')
    r.raise_for_status()
    return r.json()

Code before yield runs once per process, before Uvicorn reports startup complete. Code after yield runs at shutdown, after the server stops accepting connections and in-flight requests finish or hit the graceful-shutdown timeout. The try/finally matters: if the process receives a shutdown signal while the lifespan is open, the close still runs.

Cost, with hypothetical figures. Assume a same-region partner at 30 ms round trip over TLS 1.3. A fresh client needs one round trip for TCP and one for the TLS handshake, then the request itself: about 90 ms per call. A reused keep-alive connection pays only the 30 ms request. At 500 calls per second, the per-request version also opens and tears down up to 500 sockets per second, which fills the partner's connection limits and leaves TIME_WAIT sockets on your host.

Failure modes.

  1. Pool multiplied by workers. Each Uvicorn worker builds its own client. Four workers with max_connections=100 can hold 400 sockets to the partner. Size the limit as the partner's allowance divided by worker count, not as a per-process number taken from a tutorial.
  2. Missing close. A client that is never closed leaves sockets open until garbage collection, and a growing open-file count under steady load shows it. Check ls /proc/<pid>/fd | wc -l across a load run; it should plateau.
  3. Tests that skip lifespan. httpx.AsyncClient(transport=ASGITransport(app=app)) does not run lifespan, so app.state.partner is missing and each test fails with AttributeError. Use with TestClient(app) as client:, which runs startup and shutdown, or override the state explicitly.
  4. Bad configuration found late. If the startup code validates the base URL or credentials and raises, the process never starts, so the failure appears at deploy rather than on the first user request.

Alternatives. A module-level client works, but it is built at import time, before configuration is loaded, and it has no shutdown point. A dependency that yields a client per request keeps the lifecycle but reintroduces the handshake cost. Use lifespan for process-wide objects. If the partner requires per-user credentials, keep the client shared and pass auth headers per call, because a single client bakes headers in at construction.

Curated: · Written: · Reviewed:

Q&A-25

A FastAPI route is declared async def and calls requests.get on a partner API that takes about 800 ms. Under load, p99 for every route in that worker climbs past 2 s, including a route that only returns a constant. Explain the mechanism and the smallest change that stops it.

Answer

Opening answer. The constant route is the clue. It does no I/O, so if it slows, the delay is not in its own work and not in a queue for its own resources. Every coroutine in the worker shares one event loop, and a blocking call inside an async def holds that loop until it returns. The partner call runs inside the route, so while one request waits 800 ms for the partner, no other coroutine in that worker makes progress.

Mechanism. FastAPI runs async def path operations directly on the event loop. The await points are where the loop can switch to other work. requests.get is synchronous: it blocks the thread inside a socket read, and the loop cannot run until the call returns. With one such request in flight, the loop idles for 800 ms. With several, they serialise. A worker that receives 10 concurrent requests to the partner route handles them one after another, so the 10th waits roughly 8 seconds, and the constant route queued behind them waits the same.

# Blocks the event loop for ~800 ms per call
@app.get("/quote")
async def quote(sku: str):
    r = requests.get(f"https://partner.example/price/{sku}", timeout=2)
    return r.json()

# Same route as def: FastAPI runs it in a worker thread
@app.get("/quote")
def quote(sku: str):
    r = requests.get(f"https://partner.example/price/{sku}", timeout=2)
    return r.json()

The smallest change. Keep requests and change the declaration from async def to def. FastAPI then dispatches the handler to Starlette's threadpool, which allows 40 threads per process by default. A blocked call now ties up one thread, and the event loop keeps serving the constant route. The edit is one keyword, and the shared wrapper is untouched.

Alternatives and their costs. Replacing requests with httpx.AsyncClient or aiohttp removes the blocking entirely and is the better long-term design, but it changes the shared wrapper the team depends on. Wrapping the call in anyio.to_thread.run_sync inside an async def also works and keeps the route async, at the cost of one extra indirection. Both are valid; the question asks for the smallest edit.

Limits of the fix. Thread dispatch does not add capacity. At 40 concurrent partner calls, the 41st waits in the thread queue, and that wait is invisible to handler timing. Set a timeout on the call, which the example does, and watch the thread queue depth. If p99 still climbs under load, the constraint has moved from the loop to the threadpool.

Curated: · Written: · Reviewed: