URL shortener design

A URL shortener is the system-design question that looks like a hash function and is actually a unique-key store, an HTTP redirect policy, and an abuse problem. This guide treats the product as a map from a short public key to a long URI: how keys are allocated without races, which 30x status you can cache, how clicks leave the GET path, and why an open redirect under your domain is the failure that gets you dropped from Safe Browsing. It assumes the vocabulary of scalability, caching, and REST, and it uses concrete QPS, key lengths, and TTLs rather than a nameless 'high scale' box.

the mapping, not the hash

A URL shortener is a durable map from a short key to a long URI, plus a redirect policy; hashing is one way to mint keys, not the product.

The read path is GET /{key} → 30x Location: {long}. The write path inserts (key, long, owner, expiry) under a unique constraint on key.

The product is two operations and one unique column.

verb	path	store	response
POST	/v1/links	insert unique key	201 {key, short_url}
GET	/{key}	lookup key	302 Location + Cache-Control
GET	/missing	miss	404, cache 10 s not 1 day

Interview trap. Designing the whole system as 'MD5 the URL and take six characters' treats collisions, custom aliases, and updates as afterthoughts.

Engineering practice. Draw the table and the two HTTP verbs first. Choose a key generator only after the uniqueness and update rules are stated.

base62 encoding of a counter

Encoding an autoincrement integer in base 62 ([0-9A-Za-z]) yields short keys whose length grows with the count, not with hash collisions.

Alphabet size 62: 62^6 ≈ 56.8 billion keys, 62^7 ≈ 3.5 trillion. A 64-bit counter encoded without padding is 11 characters at most.

Length vs capacity, and when a 6-char hash actually collides.

scheme	6-char space	first expected collision
base62(counter)	56.8e9, no collision	never, until the counter wraps
random 6-char	56.8e9	~2.4e5 inserts (birthday)
sha256[:6] of URL	56.8e9	same birthday, plus same-URL duplicates

Interview trap. Saying a 6-character hash of the long URL is unique because 62^6 is large ignores birthday collisions, which start mattering around sqrt(N) ≈ 238,000 keys.

Engineering practice. If the key may be sequential, encode a unique integer. If the key must not leak creation order, hash or permute the integer after it is allocated.

counters vs hashes vs UUIDs

Autoincrement is unique and short; a content hash is deterministic for the same URL; a UUID is unique without a central counter and is too long to put in a tweet unless you truncate it, which reintroduces collisions.

RFC 9562 UUIDs are 128 bits (36 hex chars with hyphens in the standard text form). Truncating to 48 bits to get a 9-char base62 key is a new identifier, not a UUID.

What the user sees versus what the unique index covers.

-- public_key is what GET /{key} looks up. It must be unique, not a prefix of uuid.
create table links (
  id           bigint generated always as identity primary key,
  public_key   varchar(12) not null unique,
  destination  text not null,
  created_at   timestamptz not null
);

Interview trap. Storing a UUID v4 and showing the first 7 characters as the public key, then looking up with LIKE, both collides and cannot use a unique index on the public key.

Engineering practice. Allocate a unique integer or a unique random key in the public alphabet, and store the UUID only if you need an internal opaque id separate from the public key.

collision handling on insert

Uniqueness is enforced by the store, not by a SELECT-then-INSERT. On conflict, retry with a new key or return the existing row for the same owner and destination.

PostgreSQL unique indexes reject the second insert. INSERT ... ON CONFLICT (public_key) DO NOTHING RETURNING * tells you whether to retry.

Two concurrent POSTs for different URLs that sampled the same random key.

t	writer A	writer B	store
1	SELECT miss	SELECT miss	empty
2	INSERT k	INSERT k	unique violation on B
3	201	retry with k2	A holds k

Interview trap. Checking existence in the application then inserting is a race: two writers both see a miss and one fails with a 500 the client cannot retry safely.

Engineering practice. Put a unique index on public_key, retry a bounded number of times on conflict for random keys, and treat same-owner same-destination as idempotent.

custom aliases

A user-chosen alias is still a unique public_key, with a reserved-word list and a character allowlist, not a free-form slug.

Allow [0-9A-Za-z_-], reject homoglyphs, reject aliases that match product paths (/api, /health, /v1), and cap length (e.g. 32).

Unicode TR36 mixed-script spoof of a reserved alias.

requested alias	bytes	decision
apple	61 70 70 6c 65	reject if reserved
аррle (Cyrillic а,р)	d0 b0 ...	reject mixed-script
my-launch	6d 79 2d ...	accept if unique

Interview trap. Accepting any Unicode string as an alias because 'the database is UTF-8' creates /аррle vs /apple phishing (Cyrillic а).

Engineering practice. Normalize to a documented alphabet, compare case-insensitively if you fold case, and keep a blocklist of routes and trademarks.

301 versus 302 versus 307

RFC 9110: 301 and 308 are permanent; 302 and 307 are temporary. 301/302 historically allowed method rewriting; 307/308 preserve the method.

Browsers and CDNs cache 301 aggressively. If the owner later changes the destination, a cached 301 keeps sending users to the old Location.

A destination change after a cached 301.

status	Cache-Control	owner updates destination at t=60s	user who cached at t=0
301	public, max-age=86400	DB has new URL	still old Location for up to 1 day
302	private, max-age=60	DB has new URL	new Location within 60 s

Interview trap. Always returning 301 because it is 'better for SEO' makes destination edits and takedowns ineffective for anyone who already resolved the key.

Engineering practice. Use 302 or 307 for a product that allows edits, expiry, and abuse takedowns. Use 301 only when the mapping is contractually immutable.

open redirects

A shortener that will redirect to any http(s) URL is an open redirect: attackers mint keys that 302 to phishing pages under your domain's trust.

OWASP: validate the destination scheme (https only), optionally allowlist hosts for enterprise shorteners, and refuse javascript: and data: URIs (RFC 3986 schemes).

Destinations a naive startsWith('https') check accepts.

https://good.example/ok          -> allow
javascript:alert(1)              -> reject (scheme)
https://evil.example             -> allow only if host allowlisted / scanned
https://good.example.evil.example -> reject if you meant suffix match on good.example

Interview trap. Percent-decoding once then substring-matching the trusted host misses https://trusted.example@evil.example, whose host is evil.example, and nested encodings.

Engineering practice. Parse with a URI library, require https, reject userinfo, and run Safe Browsing or an equivalent lookup before first redirect.

the read path latency budget

Redirect latency is a cache lookup plus a 30x, not a join. p99 should stay under a few tens of milliseconds at the edge.

Redis GET of the mapping is O(1). A miss loads the row, populates the cache, and coalesces stampedes (AWS builders library).

Budgets for 50k redirects/s.

hop	p50	p99
edge TLS + 302	8 ms	25 ms
Redis GET (same AZ)	0.4 ms	2 ms
Postgres primary on miss	3 ms	40 ms
join to click_facts on GET	12 ms	180 ms (do not)

Interview trap. Joining owner, tags, and click totals on every GET /{key} puts analytics on the user-visible path and blows p99 when the stats table is large.

Engineering practice. Serve Location from a key-value cache. Write clicks asynchronously. Keep the hot GET handler under 1 ms of application work plus one cache RTT.

click accounting off the GET

A click is an event, not a row update on the mapping. Counting with UPDATE links SET clicks = clicks + 1 on the redirect serializes the hot key.

Emit (key, ts, ua_hash, country) to a queue or log; aggregate later. The mapping row stays read-mostly.

One viral key at 8,000 rps.

design	lock / hotspot	lost clicks
UPDATE clicks+1 per GET	row lock on that key	0, p99 collapse
Redis INCR, flush 1 s	one Redis key	~0, still on GET
append-only event, 10 s rollup	none on GET	±0.3% at flush

Interview trap. Incrementing a counter on the same row you read for Location creates write contention on the most popular keys, which are exactly the ones you needed to be fast.

Engineering practice. At-least-once click events with idempotent aggregation windows. Accept ±1% on a dashboard, not a locked integer on the request path.

negative caching

Cache misses and 404s, but much more briefly than hits, so a just-created key is not hidden behind a long negative entry.

At an HTTP cache, RFC 5861 stale-if-error can keep a cached redirect usable when an origin dependency fails; it does not justify caching a 404 for 24 h.

Create then immediately GET from another POP.

t	event	US-east cache	EU-west cache
0	GET /abc 404	NX ttl=15s	empty
1	POST creates abc	DEL NX	empty
2	GET /abc from EU	302	miss → DB → 302

Interview trap. Caching every unknown key as NX for one hour means a user who creates a link then shares it immediately still 404s.

Engineering practice. Negative TTL on the order of 5–30 s. After a successful insert, delete the negative cache entry for that key in the same region.

idempotent create

Retrying POST /v1/links after a timeout must not mint a second key for the same owner and destination unless the client asked for uniqueness of the long URL.

Idempotency-Key header or a unique (owner_id, destination_hash) for the default 'reuse existing' mode.

Client timeout after the insert committed.

attempt	Idempotency-Key	DB	response
1	k-9f	insert abc	timeout, 201 lost
2	k-9f	see k-9f	201 abc again
2 without key	—	insert def	two keys, abc unused

Interview trap. Returning a new random key on every retry makes the first key an orphan and the client's 'copied URL' point at a 404 if the first write actually landed.

Engineering practice. Document two modes: reuse-if-exists (default) and always-new. Both need an idempotency key for the HTTP retry case.

read-your-writes after create

The creator must resolve their new key immediately, even if the replica or remote cache is behind.

Return the key and destination in the 201 body. Route GET /{key} for a few seconds to the primary, or write-through the local cache in the same AZ as the insert.

W=1 on the primary, GET hits a replica 400 ms behind.

read	creator sees
replica, R=1	404, retries POST
201 body, no extra GET	key abc, done
primary for 2 s	302

Interview trap. Reading the mapping only from a 400 ms lagging replica after insert looks like a failed save and produces duplicate POSTs.

Engineering practice. 201 body is source of truth for the creator. Sticky or primary reads until replica lag is below your SLA.

expiry and 410

Expired keys should stop redirecting. RFC 9110 410 Gone is more honest than 404 when you know the key existed.

Store expires_at. The GET path treats now >= expires_at as gone. Cache 410 briefly, bounded by the product's reactivation and key-reuse policy.

TTL 24 h, cache max-age 6 h, job runs hourly.

t	DB expires_at	cache	GET
23 h	future	302 cached	302
24.1 h	past	still 302 if not revalidated	wrong
24.1 h if value carries expiry	past	treat as gone	410

Interview trap. Leaving expired rows in the hot cache as 302 until LRU evicts them keeps phishing and stale campaigns alive.

Engineering practice. Include expiry in the cache value. A background job can delete rows; the GET path must still enforce expires_at even if the job lags.

capacity of the write path

Write QPS is usually tiny next to redirects. Size the unique-id allocator and the primary for creates, and the cache for reads.

1 million new links/day is ~12 inserts/s average, ~200/s peak. 10 billion redirects/day is ~116k/s average.

A launch-week vs a mature consumer shortener.

stage	creates/s p95	redirects/s p95	store
internal tool	2	40	one Postgres
public beta	80	2_000	Postgres + Redis
consumer app	400	80_000	partitioned KV + edge cache

Interview trap. Sharding the mapping table on day one because 'shorteners are high scale' when you have 80 creates/s and 2k redirects/s.

Engineering practice. Quote both numbers in the design. Shard or partition when a single primary cannot take the write peak plus compaction, not when the whiteboard looks sparse.

partitioning the map

When writes outgrow one primary, partition by public_key (hash or range), not by destination URL, so GET /{key} is a single partition lookup.

hash(key) % N or a directory. Range-partitioning sequential counters hotspots the latest partition.

Where a GET lands.

shard by	POST duplicate URL	GET /abc
hash(public_key)	may be another shard	one shard
hash(destination)	same shard	unknown shard, scatter
sequential range	latest shard hot	one shard, write hotspot

Interview trap. Sharding by hash(long_url) so duplicates collocate, then GET /{key} has to broadcast because the key no longer determines the shard.

Engineering practice. The public key is the partition key. Dedup of destinations is a secondary index or an async job, not the shard function.

preview and interstitial

A preview page such as GET /preview/{key} that shows the destination before redirect is an abuse control, not a performance feature.

The interstitial is HTML, noindex, and must not auto-redirect in 0 ms or phishing filters treat it as a 302 with extra steps.

What the interstitial must display.

field	example	why
registrable domain	paypal-secure.example	not the full 200-char path
scheme	https	http is a warning
continue	explicit button	no 0-delay meta refresh

Interview trap. Putting a 3-second countdown that still always continues trains users to click through and does not change Safe Browsing risk.

Engineering practice. Show the registrable domain prominently, block known-bad destinations, and skip the interstitial for allowlisted enterprise hosts.

malware scanning

Scan destinations at create and periodically after, because a clean URL can be parked then swapped to phishing.

Use Safe Browsing for a non-commercial service, or Web Risk/equivalent for commercial use, on insert and on a crawl interval. On a confirmed hit, stop 302 and serve an interstitial block.

Same key, destination path unchanged, content swapped.

t	destination	scan	GET
day 0	https://site.example/x	clean	302
day 12	same URL, new HTML	hit	410 / block page

Interview trap. Scanning only at create, once, because 'the URL string did not change' misses hosted content that changed behind the same path.

Engineering practice. Re-scan popular keys more often. Treat scanner downtime as fail-closed for new creates and fail-open-with-log for existing GET only if product accepts that risk explicitly.

rate limits on minting

Create is the expensive, abusable verb. Rate-limit by account and by IP; do not rate-limit GET /{key} the same way.

Token bucket per owner: e.g. 30 creates/min, 500/day. RFC 6585 defines 429 Too Many Requests and permits a Retry-After header.

Limits that survive a viral GET storm.

route	limiter	budget
POST /v1/links	owner_id	30 / min
POST /v1/links	ip if anonymous	5 / min
GET /{key}	none (cache)	edge capacity

Interview trap. A global 100 rps cap on the whole HTTP service, which 302 traffic blows through, so creates starve or you open the floodgates.

Engineering practice. Separate limiters. Redirects are cheap and cacheable; creates allocate unique keys and may call the scanner.

HTTPS and mixed destinations

Serve the shortener only on HTTPS. Redirecting an https short URL to http: is a downgrade; prefer to refuse http destinations.

HSTS on the short host. Destination scheme checked after RFC 3986 parse, not with a regex on the raw string.

What the browser sends after 302.

short URL	Location	Referer / cookies on next hop
https://s.example/a	https://app.example	HTTPS, Referrer-Policy applies
https://s.example/a	http://app.example	downgrade, path on the wire

Interview trap. Allowing http:// destinations 'for compatibility' turns your 302 into a plaintext leak of the path and cookies on the target.

Engineering practice. Default deny http. If you must support it, warn on preview and never silently rewrite https→http.

key alphabet and copy-paste

Avoid visually ambiguous characters in generated keys (0/O, 1/l/I) if humans will read them aloud; keep the decoder accepting the full base62 if you ever used it.

A 32-char Crockford-like alphabet reduces mis-reads. Changing alphabets later requires supporting old keys forever.

A printed key that used 0 vs O.

printed	typed	lookup
ab0cd	abOcd	miss if alphabet distinguishes
ab0cd	ab0cd	hit
Crockford: no O,I,L,U	fewer misreads	decoder maps o→0 if you allow

Interview trap. Switching from base62 to a smaller alphabet and re-encoding old ids changes public URLs already printed on packaging.

Engineering practice. Pick the public alphabet once. New encodings can be versioned with a prefix; never mutate existing keys.

analytics privacy

Click logs can be personal data when they include IP addresses or stable device identifiers. Store only what the product needs, pseudonymize where useful, and enforce a retention limit.

Keep day-level counts forever if they are not personal. Keep IP at most days, or hash with a rotating salt.

What to keep after 30 days.

field	24 h	30 d	1 y
public_key, ts bucket	yes	yes	daily count
IP	yes	hash or drop	drop
user-agent	coarse	coarse	drop

Interview trap. Logging full IP, User-Agent, and destination in one row 'for fraud' with no expiry recreates a surveillance table on a link-shortening company.

Engineering practice. Document retention. Fraud features can use short-lived hashes. Dashboards should run on aggregates.

CDN in front of 302

A CDN can cache 302s per key if Cache-Control allows it, but then destination edits and takedowns wait on TTL or purge.

Purge by URL on update/delete. Vary is unused if the response does not depend on cookies.

Takedown at t=0 with a 5-minute CDN TTL and no purge.

t	origin	CDN POP	user
0	410	still 302	phishing
0 with purge	410	410	blocked
0, max-age=60, no purge	410	302 ≤60 s	bounded

Interview trap. Setting max-age=31536000 on 302 at the CDN because 'the short URL never changes' after you already shipped editable links.

Engineering practice. Short shared cache (30–120 s) plus purge-on-write. Origin shield to protect Redis on a viral miss storm.

multi-region active-active

Redirects can be served from any region that has a replica of the map. Creates need a uniqueness story across regions: one writer, or CRDT-unfriendly unique keys generated per region with a prefix.

Key prefix e vs w (1 char) partitions the namespace. Or a global identity service (single region) for creates only.

Two regions minting independently.

design	unique?	GET anywhere
random 6-char both sides	no	last-write-wins clobber
prefix + local counter	yes	replicate map
global allocator, local cache	yes	create latency = RTT to allocator

Interview trap. Active-active inserts of random 6-char keys in two regions without a uniqueness protocol: both succeed, later replication conflicts.

Engineering practice. Either generate keys in a single allocator, or make them unique by construction (region prefix + local counter).

what to draw first in the interview

The whiteboard should start with clients, edge, cache, store of record, async analytics, and the uniqueness rule — not with a hash function.

State QPS, key length, editability, expiry, and abuse requirements as numbers and yes/no before choosing Postgres vs Dynamo vs Cassandra.

Questions that fork the design in the first five minutes.

question	if yes	if no
Editable destination?	302, purge, no 301	301 possible
Custom aliases?	allowlist, Unicode fold	generated keys only
Multi-region creates?	prefix or allocator	single primary

Interview trap. Opening with 'we MD5 and take 6 chars, then ZooKeeper for locks' spends the interview on a collision protocol you may not need.

Engineering practice. Ask whether aliases, edits, and enterprise host allowlists exist. Those three answers change the design more than the hash.

when not to build one

Most products need a link branded to their own host and a table of (key, url), not a platform shortener with global abuse, CDN, and identity.

An internal tool at 40 rps GET is a Postgres table and a 302 handler. The consumer-scale design is for a different QPS and threat model.

Same feature, two honest designs.

constraint	internal	public
creates	200 / week	400 / s
destinations	corporate allowlist	the internet
store	one table	partitioned KV
scanner	optional	required

Interview trap. Copying Bitly's public architecture into a sprint for a marketing team that will mint 200 links a week.

Engineering practice. Scale the uniqueness and abuse controls to the actual create rate and the trust of the destinations. Quote 200/week vs 80k/s before drawing Kafka.