A proof of concept buys evidence, not production code

A proof of concept (PoC) is a bounded experiment that reduces a decision-critical uncertainty. It is justified when an unknown could reverse an architecture, vendor, integration or investment choice and cheaper evidence is insufficient. It is not a miniature implementation, an unscoped innovation sprint, a sales demonstration or a way to start delivery before requirements and governance exist. The primary output is a reproducible finding with stated confidence and limitations.

Write the decision and hypothesis first. Name the alternatives, what is unknown, why the uncertainty matters, and what action follows from each result. Good questions are narrow: can the candidate preserve a required ordering invariant during failover; can it meet p99 latency and unit-cost bounds under a representative workload; can the team operate recovery within the RTO? “Can this technology work?” is too broad and almost always produces an impressive but non-decisive demo.

Define entry, exit and stop criteria before building. Entry criteria cover approved use case, representative data/workload, environment, access, skills, funding, security and dependencies. Exit criteria specify measurable functional, quality, operational and business thresholds and evidence required to decide. Stop criteria bound safety, cost, time and diminishing learning. Failure to meet a threshold is a successful experiment when it reliably eliminates or reshapes an option; changing the threshold after observing results destroys decision integrity.

Design the smallest experiment that isolates the uncertainty while retaining the conditions that determine it. Include a baseline or competing option and use the same method for each. Control versions, configuration, dataset, workload generator, warm-up, duration and measurement point. Repeat enough to understand variance. Preserve scripts, infrastructure definitions, seeds, raw results and analysis so another engineer can reproduce the claim. A screenshot or best run is not evidence.

Representativeness is explicit, not binary. State which production dimensions the PoC includes and omits: traffic distribution and concurrency, data size/skew/sensitivity, network and dependency behavior, quotas, identity and policy, failure modes, geography, team operations and change. Synthetic data can protect privacy but may erase cardinality, skew or malformed cases that drive the decision. A smaller environment may still answer algorithmic scaling questions if the model and limits are defensible; it cannot silently stand in for full-scale behavior.

Measure outcomes at the relevant boundary. Use percentiles and attainment, not averages alone; include correctness, throughput, error, recovery, resource/unit cost and operator effort where the hypothesis needs them. Validate telemetry and clocks before trusting charts. Record unsuccessful runs, anomalies, discarded data and why. Test negative paths such as dependency timeout, throttling, retry, partial result, malformed input, credential expiry, node loss, backup/restore and rollback when they can invalidate the choice.

Keep the PoC safe. Use isolated accounts/projects and least-privilege identities, approved non-production data, spend/time quotas, egress and network controls, audit, secret rotation and cleanup ownership. Do not expose an unreviewed prototype to production users or sensitive data merely to make the test realistic. If limited real traffic is necessary, use a separately approved pilot with consent, feature flags, monitoring, abort thresholds and rollback; call it what it is.

Manage vendor participation without surrendering experimental control. Use requirements-derived tests, equivalent conditions, customer-controlled raw results and disclosed assistance. Record credits, special configurations, roadmap promises and services unavailable under the proposed commercial tier. Validate portability and exit assumptions. A vendor-run benchmark can inform discovery, but it cannot be the sole evidence for a buyer decision when workloads or incentives differ.

Prevent prototype leakage. Mark PoC artifacts unsupported and non-production, isolate credentials and environments, time-limit access, track licenses and generated data, and plan deletion before starting. Deliberately decide what may be reused. Exploration shortcuts—hard-coded identity, broad permissions, manual state, missing validation, single-node dependencies, no observability—create hidden liabilities if copied. Productionization is new engineering work with architecture, security, reliability, testing, operations, support, data migration and cost gates.

Conclude with a decision review, not a demo. Report hypothesis, method, environment, evidence, variability, limitations, risks, unexpected findings and confidence. Classify each criterion as met, unmet or inconclusive. Recommend proceed, stop, change the option or run a specific follow-up; do not stretch an inconclusive result to protect sunk effort. Update the ADR and risk register, archive reproducible evidence, clean up resources and communicate what the PoC did not establish.

If the option proceeds, create a productionization gap assessment and backlog. A successful PoC does not prove production readiness. Retest material claims in preproduction and progressively in production, compare actual outcomes with PoC estimates, and update the method when they diverge. The best PoC is often small and disposable: it saves a larger program from building confidently on an untested assumption.

Write the gap assessment as work, not as optimism. List which identity, observability, failure, capacity, data-handling and support properties were never in the experiment, who owns each follow-up, and what evidence will close it. If a stakeholder wants to "just harden the prototype," answer with that list: the remaining unknowns are usually the expensive ones. A PoC that cannot name what it did not prove has already started leaking into delivery.

Worked example: OpenSearch p95 under 200 ms at 50 QPS

Hypothesis written before the first query: 10 million product docs, 50 QPS, p95 query latency under 200 ms in our account, or we stop.

run	corpus	load	p95	vs the written gate
vendor laptop demo	10,000 docs	click-around	40 ms	inconclusive — not our data, not 50 QPS
our account	10 million	5 QPS	180 ms	inconclusive — load is 10× too small
our account	10 million	50 QPS	340 ms	unmet — stop or change the option

A 40 ms demo does not meet the gate. It also does not fail it. The interview is refusing to let an inconclusive run become “it worked.”