Your cache hit rate can be 99% and your database can still fall over. I got reminded of this during a load test recently. The dashboard showed a wall of cache hits, and yet the database briefly lit up with a burst of identical queries. Same key, same query, fifty times, all inside a couple hundred milliseconds. That burst has a name: cache stampede. And the annoying part is that the standard pattern we all write does nothing to stop it.
The Problem with IMemoryCache
Consider a minimal API with two endpoints and a fake database that counts every call and takes about 200 milliseconds. The app fires 50 concurrent requests at each endpoint with a cold cache.
The code reads like it should be safe. GetOrCreate sounds atomic: get it, or create it. Surely one or the other. But there is no lock in there. Every request that misses starts its own factory, they all run the query, and they all write the result. Last one wins. Fifty requests arriving before the first query finishes means fifty queries.
The HybridCache Solution
HybridCache dedupes concurrent callers per key. The first request runs the factory. The other forty-nine wait for that same task instead of starting their own.
Here is the output from a run, after a warmup pass so nobody is paying JIT tax:
What Changed, and What Didn't
Notice what did not change. Wall time was a tie. Both bursts finished in roughly 210 milliseconds, because HybridCache's waiting requests still have to wait for the one real query. Stampede protection does not make the cold burst faster for your users. What it does is send your database 2% of the work.
With a fake database that has a fixed delay, the compounding effect is not visible. But a real database does not behave like that. Fifty identical concurrent queries make each other slower, which stretches the miss window, which lets more requests pile in. That is the spiral this kills.
The warm rows are there for honesty too. Once the key is cached, both approaches are fine. Caching was never the problem. The 200 milliseconds after expiry were the problem.
A Cautionary Tale
My first version of the demo crashed. I used the same key string in both endpoints. Turns out HybridCache's L1 is the same instance that DI hands to IMemoryCache, and it stores its own wrapper objects in there. So my plain read found HybridCache's entry and died with a deserialization error. If you are migrating an app cache-by-cache and both APIs are alive at once, prefix your keys.
When Not to Use HybridCache
If the factory is cheap, a dictionary lookup or a sub-millisecond primary-key fetch, fifty duplicate calls cost less than the time you will spend thinking about them. IMemoryCache is also still the right tool when you are caching things that cannot or should not be serialized, since HybridCache wants serializable values the moment you add an L2.
And the protection is per process. Three pods still means three cold queries, one each, unless you wire up a distributed backplane behind it. One query per pod is usually a fine place to stop worrying.
When to Use HybridCache
For anything that is expensive to build and shared across requests, the hot product page, the tenant config, the dashboard aggregate, this is about as cheap as insurance gets. One service registration, one method swap.
Clone it, crank up to 500, watch the counter. Have you caught a stampede in production, or did you only find out from the database bill?