Anatomy

Same prefix, two requests later.

Request 1

Cache write
system tools docs Q1

Nothing to reuse. The whole prompt is prefilled. The provider stores the KV tensors for the prefix as the response starts.

cached_tokens = 0  ·  cache write billed at a small premium

Request 2

Cache hit
system tools docs Q2

The gold tokens match the stored prefix. Prefill resumes at Q2. Gold tokens are the hit. Q2 is still computed live.

cached_tokens = prefix length  ·  reads often 50–90% cheaper