Caching
Browse all articles, tutorials, and guides about Caching
Posts
Prompt Caching on DigitalOcean Inference, Measured: What Breaks It and How Long It Lasts
We sent 867 requests with a long on-call runbook as the system prompt to three DigitalOcean-hosted models. With the runbook first and unchanged, the cache cut the input bill about 5 times on Llama 4 Maverick and more than 12 times on DeepSeek V4.1 Flash. One timestamp on the first line turned it off on all three. Here is what else turns it off, how long a cached prefix lasted, and what prompt_cache_key does.
Your Semantic Cache Answers the Question Next Door
We put a semantic cache in front of an ops assistant on DigitalOcean Serverless Inference and replayed 288 hand-labelled questions through six embedding models. At the threshold that cut the bill by about 30 percent, about one hit in three was the answer to a different question. Someone asking how to undo a pushed commit got the instructions for an unpushed one.
How Netflix Ships a Third of the Internet: The CDN They Had to Build
At its 2015 peak Netflix was 37% of North American downstream traffic, and almost none of it came from a commercial CDN. Open Connect is a cache hierarchy built on one asymmetry: Netflix knows tonight what people will watch tomorrow. Here is how the appliances, the nightly fill, the BGP steering and the 800 Gb/s FreeBSD boxes fit together, a runnable comparison of push fill against pull-through caching, and what it teaches anyone running a cache.
HTTP QUERY Shipped. Your Cache Did Not Get the Memo
RFC 10008 gave HTTP its first new method since 2010: QUERY, a request that is safe and idempotent like GET but carries a body like POST. The semantics are the easy part. The hard part is that its cache key includes the request body, which is not something your CDN or browser does by default yet, and the RFC quietly ships a workaround for exactly that.
Is Valkey Ready to Replace Redis in 2026?
Valkey forked from Redis after the 2024 license change and has matured fast. Here is whether it is production-ready, how the migration works, and whether the AGPL question even applies to you.