The lede
Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang
Kevin Flansburg and Michelle Chen, Cloudflare
For thirteen days in August and September, 155 Workers AI responses came back carrying text from other customers' sessions, and the mechanism is the kind you only find by reading the whole post. Cloudflare advertised GLM 5.3 with a 1,048,576-token context window while the KV cache pool on its prefill GPUs held only 681,280 tokens — a mismatch they knew about and assumed the engine would turn into a clean error. It did not: SGLang marked the over-length prompt as rejected, swapped in a one-token placeholder, generated a cache for that placeholder and told the decode node the transfer was complete, so decode generated a reply while attending over pages still holding the previous request's data. Two details are worth carrying around: pages are not zeroed between requests because zeroing them costs too much, and the pool size is not a config value at all — SGLang sizes it at startup from whatever GPU memory is left after the weights load, so nothing forces it to agree with the context length you published.