Read replicas that fall behind at peak
A prediction market moves its reads onto its failover replicas at the peak, the replicas fall behind and fail, and the site goes down.
Four people decide to build a prediction market.
- They don’t have a database administrator and can’t afford one, so they go with a managed Postgres service from their cloud provider.
- That’s a good choice. It’s what I’d pick too.
- The provider runs it for them. It’s Postgres, so the four of them can get on with the product.
Illustrative scenario. Timings show the shape of the problem, not a measurement of any one company. Story by Boyd Stowe.
Your stack today, and with the Tacnode Context Lake™
Today
Moving reads to replicas is the standard advice, and the founders made sensible choices at every step. A replica’s main job is to take over when the writer fails. Once the product reads from it, it has a second job. At the peak, held connections put most of the traffic on one replica, it falls behind, and the app shows a book that is seconds old while every health check stays green. When that replica gives out, its traffic moves to the next one. The only way to add read capacity was to add a copy, and the copies are what failed.
With the Tacnode Context Lake™
- Read capacity without a copyCompute scales out over one shared store, so there is no replica to fall behind and the book on screen is current to the second.
- Heavy reads off the order pathRisk, settlement and search run on the same tables as the orders without slowing confirmations.
- Durability without the failover trapData is erasure-coded across the storage pool, so no standby copy gets used up serving reads.
Questions to ask your team
- 1Do your production reads run on replicas that exist for failover?
- 2At your peak, how far behind the writer is your busiest replica, and does anyone see it?
- 3When an order is rejected for a moved price, can you tell whether the price was really gone?
Go deeper
More replays
All replaysCard issuing
False declines on newly issued virtual cards
A store card approved at checkout is declined on its first purchase, because the fraud check can’t yet tie it to the customer’s history.
AI customer support
Duplicate credits from concurrent AI agents
Three AI support agents each give one customer a goodwill credit, because each checks a synced copy of credits that hasn’t received the others yet.
Sports betting
Late market suspension in in-play betting
After a goal, a sportsbook accepts bets at the pre-goal price for seven seconds, because the suspension reaches bet acceptance through the odds pipeline.

