The cache is consulted on eligible requests. A hit is not “similar enough.” All of these must be true.
Byte-for-byte, token-for-token, including punctuation, whitespace, tool JSON, and image blocks. One character in the first 1,024 tokens zeros the hit.
OpenAI typically 1,024+ tokens, then 128-token steps. Claude minima range from 512 to 4,096 by model. Shorter prefixes silently skip caching.
The first request writes. Hits start on the next request after that response begins. Parallel first calls all miss and cannot share a write that is not ready yet.
Claude: 5 minutes, refreshed on every hit, optional 1 hour. OpenAI: automatic and opportunistic; explicit 30-minute TTL on newer models.
Caches do not cross models, and usually not workspaces. Routing must land on a replica that still holds that prefix.
On Claude, writes happen at the cache marker. If you mark the changing user turn, you write a new hash every time and never read.