pnclENGINEGet API access

CAPACITY NOTES

When every cached copy expires at once

Shared TTLs turn one cache miss into a burst of identical work. Jitter the lifetimes, guard the miss, and the upstream graph goes flat.

Your poller fetches once, your cache serves everyone, and the request count stays flat. Then one Tuesday the upstream calls spike for a minute, at the same minute every day, and the pattern is not the market. It is your cache expiring everywhere at once. That is a stampede, and it is a design bug you can remove with one line of randomness.

How a stampede forms

The REST integration guide recommends caching snapshots for downstream readers, so one poller feeds many consumers, checked 2026-11-08. The trap sits in the TTL. If every cached copy is written in the same second and given the same lifetime, every copy expires in the same second, and every reader that arrives just after the expiry misses together.

The misses do not queue politely. They recompute or refetch in parallel, multiplying a one-miss moment into a burst of identical work. Your upstream request graph shows a spike with a period exactly equal to your TTL, which is the signature to recognize.

Jitter the lifetime

The fix is to make the expiry time fuzzy. Add a random offset to every TTL, say plus or minus ten percent, so the copies that were written together stop expiring together. A cache of many entries becomes a stream of small expirations instead of one shared cliff, and the recomputation load spreads across the minute instead of stacking at its edge.

The same trick applies at every level of the stack: the poller's own retry schedule, the browser cache on the page, the CDN in front of the API consumer. Anywhere many copies share a deadline, a deadline shared exactly is a spike scheduled in advance.

The single-flight guard

Jitter spreads expirations; a single-flight guard caps the damage when a miss still happens. The first request to miss recomputes while the rest wait for its result, so one miss costs one upstream call instead of fifty. Most cache libraries and most frameworks have the pattern under names like single-flight, request coalescing or dogpile protection, and it is a line of configuration more often than code.

The two together are the whole design: jitter keeps the misses rare and spread, the guard keeps the survivors cheap. A cache with both stops being a load amplifier and goes back to being what the integration guide intended: the thing that keeps fifty internal readers costing exactly one upstream reader.

Measure it once

The verification is one graph: upstream requests per second, watched for a day. A healthy cache shows a flat line set by the poller's interval. A stampeding cache shows spikes at TTL intervals, and the spikes vanish the day the jitter ships.

Shared expiry is a silent scheduler. Add jitter, guard the miss, and the only minute that spikes is the one the market actually owns.