Overview
Curated: · Written: · Reviewed:
Add agents only when the dependency graph earns them
A multi-agent system coordinates several model-driven roles, and the recurring interview mistake is treating "more agents" as free intelligence. Every added agent costs latency, tokens, correlated error, security surface and operational state. The defensible position: start from a measured single-agent or deterministic-workflow baseline, and add agents only when the work genuinely separates by context, capability, trust domain, independent evidence or parallelism — and the composed outcome beats that baseline by enough to pay for coordination.
When a second agent is justified — and when it isn't
Before reaching for a second agent, exhaust the cheaper decompositions:
- Better tools. A retrieval tool, a code interpreter or a SQL endpoint often replaces a "research agent." A tool call is deterministic, testable and costs milliseconds; an agent adds a model call, a prompt and a failure mode.
- Longer context or better retrieval. "Agent A summarizes for agent B" is usually a workaround for a context window. If a 200k-token window or a reranker fixes it, that's one agent, not two.
- Subroutines. A single agent calling a typed function (with the function's validation done in code) gets most of what people want from specialization, without a second prompt that can drift.
Multi-agent earns its cost when at least one of these holds:
| separation | why one agent struggles | example |
|---|---|---|
| context | subtasks need disjoint, large contexts | one agent reads a 300-page spec, another reads the codebase |
| capability | different models/tools per subtask | a cheap model classifies, an expensive model drafts, a deterministic checker validates |
| trust domain | isolation limits blast radius | a worker with read-only tenant data hands typed artifacts to a worker that can write |
| independent evidence | critics must not share the prompt | two verifiers with different tool access and different criteria |
| parallelism | branches have no data dependency | three independent research branches with a merge deadline |
If none of the rows apply, the honest answer in an interview is "one agent with better tools," and saying that is a strong answer.
Topologies and where each one breaks
| topology | shape | breaks when |
|---|---|---|
| supervisor/worker | one orchestrator decomposes, delegates, assembles | supervisor becomes a bottleneck and a single point of failure; its context fills with every worker's output |
| hierarchical supervisor | supervisors of supervisors | routing bugs hide two levels up; budgets and cancellation must traverse the tree |
| handoff-based | one active agent at a time, passes control | handoff loses context silently; no one owns the end-to-end goal |
| shared blackboard | all agents read/write one store | agents overwrite each other, retrieval leaks cross-tenant state, no causal order |
| decentralized/swarm | peers negotiate without a coordinator | livelock, ping-pong delegation, unbounded rounds, no accountable terminal decision |
A free-form group chat is a swarm with the state machine hidden. It makes replay, recovery and review nearly impossible, and it's the design most candidates reach for first — expect the interviewer to push on exactly that.
Whatever the shape, represent the workflow as an explicit graph: nodes with typed inputs/outputs, owner, deadline, retry and completion criteria; edges for data and control dependencies; guard conditions for branches; every cycle carrying a maximum depth, time, token and cost budget plus a terminal state.
Handoffs, contracts and shared state
Vague delegation ("you handle the database part") is where multi-agent systems rot. Delegate through a typed contract:
# Python 3.12 — task contract passed to a worker node
@dataclass(frozen=True)
class TaskContract:
objective: str # "Draft migration plan for orders table"
non_goals: tuple[str, ...] # ("do not execute the migration",)
input_artifacts: tuple[str] # immutable artifact IDs, not raw pasted text
output_schema: type # MigrationPlan — validated before acceptance
allowed_tools: tuple[str] # ("read_schema",) — server-side allowlist
side_effect_policy: str # "read-only"
deadline_s: int
budget_tokens: int
idempotency_key: str # stable across retries of this node
evidence_required: tuple[str] # ("schema_dump_checksum",)
A worker may complete, fail, decline, request clarification, or return partial results with a typed status. Natural-language confidence ("I'm confident this is correct") is not a completion signal; validators and authoritative postconditions decide.
Shared state is a concurrent database, not a scratchpad. Artifacts get immutable IDs, schema, producer/version, timestamps, lineage, ACL, status and checksums. Optimistic versions, leases or transactions prevent lost updates and duplicate ownership. A worker should never read another tenant's state through semantic retrieval — retrieval is not an authorization boundary.
Context-window management is part of the contract: each worker receives the minimum task context, and handoffs pass artifact IDs plus a summary, not full transcripts. If you find yourself forwarding whole conversations, the graph is missing a node.
Failure modes and guardrails
The failure modes interviewers probe: infinite or ping-pong loops, duplicated work, context loss across handoffs, error cascade, runaway cost, and no termination condition.
Trace a ping-pong loop to see why budgets must be structural, not aspirational:
round 1: planner → coder: "implement X"
round 2: coder → reviewer: "review please" (status: needs_review)
round 3: reviewer → coder: "fix null handling" (status: changes_requested)
round 4: coder → reviewer: "fixed" (status: needs_review)
round 5: reviewer → coder: "still null on empty input" (status: changes_requested)
... # identical exchange repeats; no node ever emits `done` or `escalate`
Each round costs two model calls and tokens, and nothing in the prompts forces progress. The guardrails:
- Bounded revision rounds. Hard cap (e.g., 3) on any critique/revision cycle; on exhaustion, escalate to a human or accept-with-flag. Enforced by the orchestrator, not by the agents' goodwill.
- Global progress criteria. A monitor detects no state change, repeated outputs, orphan work, contradictory terminal states, and picks a deterministic recovery or escalation.
- Deadlock/livelock prevention. Lease expiry, wait-for visibility, cycle detection.
- Budgets and timeouts per node and per workflow. Tokens, wall-clock, spend. Exhaustion is a typed failure with a defined terminal state, not a hang.
- Idempotency for duplicated work. Delivery can be delayed, duplicated or reordered; a timeout does not prove a side effect failed. Use stable operation keys, check authoritative state before retrying ambiguous writes, and build exactly-once business effects on idempotent consumers rather than transport promises.
- Error containment. Classify transient, permanent, invalid-output, policy, budget and ambiguous-commit failures. Retry only safe operations; reroute only to qualified compatible workers; for multi-step external changes, record a saga with completed effects and explicit compensations — compensation is itself a new authorized action and may fail. When rollback is impossible, stop, preserve evidence, escalate.
Cancellation must propagate to children and release leases, reservations and temporary artifacts — without blindly interrupting already-committed external transactions.
Worked example: group-chat retry doubles a $50 refund
Hypothetical figures, but the mechanism is the point. Order 4419, $50 refund. The payer tool times out after the POST; the HTTP client cannot tell whether the refund landed.
| design | model calls | refund POSTs | $ refunded | p95 |
|---|---|---|---|---|
single agent, idempotency key refund:4419 | 3 | 1 | 50 | 2.1 s |
| 3-agent group chat, retry on timeout | 18 | 2 | 100 | 14 s |
| graph: payer node idempotent; workers cannot mint tools | 6 | 1 | 50 | 4.2 s |
Without an idempotency key and a server-side capability registry, the chat is a duplicate-spend machine. The fix is not a smarter prompt; it's a structural property of the graph.
Security: tools are capabilities, messages are untrusted input
Allowlist narrow operations per node, derive identity and resources server-side, validate arguments and business rules, require confirmation for consequential effects, sandbox generated code, constrain network egress and secrets, and verify postconditions. Delegation can narrow but never broaden user authority — the orchestrator validates every assignment against a server-side capability registry and the current authenticated scope. One compromised worker must not inherit the union of all system tools.
Agent messages, shared memory, retrieved documents and tool output are untrusted inputs that can carry prompt injection. A worker that reads a poisoned document must not gain new tools because the document asked nicely.
Evaluation, observability and attribution
Evaluate composition, not just each worker. When the final output is wrong, the first question is which agent failed — and you can only answer it with per-agent tracing and attribution:
- Propagate a trace ID; record parent/child spans, workflow/node/attempt, agent and model/prompt versions, input artifact IDs, decisions, tools, idempotency keys, state transitions, validation results, tokens, cost, latency and final external state — with redaction and access controls.
- Deterministic replay and seeding. Because state lives in immutable artifacts and append-only events, a trace should replay decisions from artifacts without storing hidden reasoning. Seed a replay with the same artifact IDs and you can reproduce a failure offline.
- Trajectory evals. Score the path, not just the endpoint: decomposition coverage, routing decisions, handoff loss, conflict resolution, duplicate effects, authorization checks, stopping behavior.
- Fault injection. Simulate slow, malicious, hallucinating and unavailable workers; delayed and duplicate messages; stale state; partial writes; budget exhaustion. Score externally observed state and critical invariants against a single-agent baseline — task success, latency, cost per successful task.
Aggregation itself is a decision procedure. Merge compatible typed artifacts deterministically, preserve citations and dissent, surface missing inputs. Majority voting is weak when agents share a model, prompt or evidence, because errors correlate; critics should have independent criteria and, where possible, different evidence or deterministic checks. High-impact conflicts and policy ambiguity go to a designated accountable human or rules-based authority — not to another persuasive agent.
Deployment
Ship the exact graph, prompts, models, tools, policy, schemas and state versions together, through shadow traffic or a small sticky canary. Monitor workflow completion rate, retries, loop counts, disagreement, human interventions, security events, p95/p99 latency and cost per successful task. Predeclare kill thresholds, support cancellation and atomic routing rollback, and keep state migration compatible across versions.
What interviewers probe, and what a weak answer sounds like
Likely follow-ups, in rough order:
- "Why not one agent with better tools?" If you can't answer this, you don't get to the rest. Weak answer: describes the multi-agent architecture before establishing the baseline it must beat.
- "Walk me through your handoff format." Weak answer: "they talk in natural language." Strong answer: typed contract, artifact IDs, validation at the boundary, defined statuses.
- "Two agents keep sending work back and forth. What happens?" Weak answer: "add a supervisor" (the supervisor can loop too). Strong answer: bounded rounds, global progress criteria, deterministic escalation — two agents repeatedly handing work back is not resilience.
- "A tool call times out mid-refund. Now what?" Weak answer: "retry." Strong answer: idempotency key, check authoritative state, saga record for multi-step effects, compensation as a new authorized action.
- "The final answer is wrong. Which agent failed?" Weak answer: shrugs. Strong answer: trace ID, per-node spans, replay from immutable artifacts, trajectory eval that localizes the failure.
- "How do you stop prompt injection through a retrieved document?" Weak answer: "the system prompt says ignore instructions in documents." Strong answer: untrusted-input treatment, per-node tool allowlists, server-side authorization, postcondition checks.
The through-line interviewers want: adding agents is a cost/benefit claim about a dependency graph, and the platform — not the agents' good behavior — enforces authority, determinism, observability and recovery.