A transformer request has two phases. Token cache only helps the first one.
Every input token is run through attention. The model stores a key and a value vector per layer. Cost grows with prompt length. A 40k-token system + docs block is a large bill even before a word is generated.
Generation reuses the KV tensors just built. Token cache does not change the output. It only skips rebuilding those tensors when the prefix has been seen before.