The brief

Three questions. One mechanism.

01

What does a cache hit mean?

The provider skipped recomputing attention for a prefix of your prompt. Those tokens were already turned into KV tensors. You still get a fresh answer.

Reuse of work, not reuse of text
02

When is it triggered?

Only when the new request starts with the same tokens as a recently cached prefix, long enough, still inside the TTL, on the same model path.

Exact prefix. Not “similar.”
03

How do we hit more often?

Put everything stable first. Keep that prefix byte-identical. Leave the changing user turn at the end. Stay inside the cache lifetime.

Structure beats cleverness