Skip to content
A hands-on workshop · 8 chapters · 2.5 hours

Caching, from first principles.

A cache is a copy of an answer, kept closer than the original. Learn everything that goes wrong with that copy, one real bug at a time, in one small bookshop.

Start chapter 01Read the source
real bugs to fix
8
real bugs to fix
dependencies in the lab
0
dependencies in the lab
experiments you can run
4
experiments you can run
Fig. 1 · One read, measured
  1. dict lookup34 ns
  2. functools.lru_cache hit53 ns
  3. SQLite query by key22.9 µs
  4. open and read a 4 KB file46.3 µs
  5. HTTP GET to localhost239.9 µs
inside your process outside itA hit in memory takes nanoseconds. Leaving the process costs thousands of times more. Python 3.13.7, Windows, 16 CPUs.
The catalogue

Eight chapters, eight bugs

Each chapter explains one idea in depth, then hands you a cache that is wrong in exactly that way. You fix it, and the checker proves the fix: right answers, and fewer trips.

  1. No. 0120 minHits and missesThe cache that never hitskeys, hit rate, latency
  2. No. 0220 minInvalidationThe price that would not changecache-aside, delete or update
  3. No. 0315 minWhose answer is it?Someone else's basketkeys, privacy, fragments
  4. No. 0420 minExpiryThe offer that never endedTTLs, sliding expiry, clocks
  5. No. 0520 minStampedesThe nine o'clock stampedestampedes, single-flight
  6. No. 0620 minEvictionThe bestseller that kept leavingLRU, LFU, FIFO, sizing
  7. No. 0715 minCaching nothingThe cache that forgot zeronegative caching, sentinels
  8. No. 0820 minHTTP cachingThe 304 that liedCache-Control, ETags, 304
Try it

The shelf

Four slots in front of a slow back room. Every book you ask for is a hit if it is on the shelf, and a miss if it has to be fetched.

  1. 1. With FIFO chosen, play the bestseller pattern. Dune is asked for every other visit, and is still evicted.
  2. 2. Switch to LRU and play it again. Dune is fetched once and stays, because every read counts as a use.
  3. 3. That one difference is case 06. Fix it in Python, in chapter 06.
Eviction policy

FIFO evicts the book shelved first.

The shelf: room for 4
  1. Empty
  2. Empty
  3. Empty
  4. Empty
The shelf is empty. Ask for a book, or play the bestseller pattern.
Ask for a book
Visits
0
Hits
0
Hit rate
none yet
Dune fetched
0
How a case works

Run it, fix it, prove it

Every case is a few dozen lines of plain Python with one bug in it, and no dependencies to install. Here is case 02 from start to finish.

  1. 1Run it
    uv run python -m shop run 2
    before the sale:   shop shows 12.00
the database says: 9.99
after the sale:    shop shows 12.00

    The database has the sale price. The shop does not.

  2. 2Fix it
    lab/cases/case02_invalidation.py
    self.db.write_price(book_id, price)
self.cache.pop(book_id, None)

    After the write, remove the copy. Only that book.

  3. 3Prove it
    uv run python -m shop check 2
    PASS  before the sale, the shop shows 12.00
PASS  after the sale, the shop shows 9.99
PASS  10 views of the page, at most 2 database reads (2)
PASS  changing one price leaves the other books in the cache

    Right prices, and the cache still saves trips.

Measured, not guessed

No policy wins everywhere

Fifty thousand visits that follow a real shop's popularity curve, replayed through four eviction policies. LFU wins while tastes stay the same, and collapses the moment they change.

Steady traffichit rate for shoppers
Cache sizeFIFOLRULFUrandom
1018.4%21.2%35.2%18.5%
5040.7%46.2%57.8%40.6%
10052.3%57.7%65.3%52.3%
Tastes changehit rate for shoppers
Cache sizeFIFOLRULFUrandom
1018.5%21.5%3.7%18.6%
5041.0%46.5%5.3%41.1%
10052.5%58.0%8.1%52.1%

From uv run python -m shop try eviction, with a fixed random seed, so you will see these exact numbers. Chapter 06 explains each one, and the third scenario: a crawler.

Keep this

The caching checklist

Ten rules, one for every way a cache went wrong in the workshop.

  1. 1List everything a cached answer depends on, and put all of it in the key.
  2. 2Leave out of the key whatever does not change the answer.
  3. 3When the truth changes, write it first, then delete the copy.
  4. 4Give every entry a TTL, even when you invalidate. It is the safety net.
  5. 5Set expiry once, when the value is stored, unless you mean sliding expiry.
  6. 6Let one caller refresh a hot key, and let the rest wait or take the stale copy.
  7. 7Bound the cache's size, and measure the hit rate on your own traffic.
  8. 8Store zero, empty and "not found" on purpose, with a shorter TTL for "not found".
  9. 9Never let a shared cache store a private page: private or no-store.
  10. 10Make ETags from the content, and fingerprint static files so they can be cached forever.
Start

Ten minutes to your first fix

You need uv and git. uv installs Python for you, and the lab uses the standard library only.

Shell
git clone https://github.com/jars-demo/caching-demo.git
cd caching-demo/lab
uv sync
uv run python -m shop run 1
HazelStuck on a case? Ask me in the corner of any page, or run shop hint 1. I know where every nut is buried. Mostly.