Why the cache exists

The expensive part is reading, not answering.

A transformer request has two phases. Token cache only helps the first one.

Phase 1 · Prefill

Turn the prompt into KV tensors

Every input token is run through attention. The model stores a key and a value vector per layer. Cost grows with prompt length. A 40k-token system + docs block is a large bill even before a word is generated.

Prefill on a long promptDecode
Phase 2 · Decode

Write the answer, token by token

Generation reuses the KV tensors just built. Token cache does not change the output. It only skips rebuilding those tensors when the prefix has been seen before.

Cached tokens are reused computation. The reply is still generated live.