FC26: Not recommended Microsoft.Extensions.Caching.Memory.MemoryCache

.Net Core comes with a new MemoryCache in Microsoft.Extensions.Caching.Memory namespace. Here is a simple test case for it:

The code is allocating an empty MemoryCache and then put 100 key/value pairs into it with 5-minute timeout.

Frugal Cafe would not recommend this version of MemoryCache for the following reasons:

1. It’s non generic, both keys and values are objects. So you can put data of different types into the same cache, and you can’t specify a custom equality comparer. For example, if you use string as key types, you can’t use StringComparer.OrdinalIgnoreCase as comparer. Also, you need to implement Equals(object other) properly which normally involves casting.

2. This version of MemoryCache is a wrapper around ConcurrentDictionary. It exposes its Count property directly. We discussed this issue before, ConcurrentDictionary.Count property getter needs to take all the locks, expensive locking and lock contention.

3. Each ConcurrentDictionary Node object is 48 bytes in 64-bit, CacheEntry adds another 104 bytes. That is quite large overhead if you have large number of entries in such cache.

4. The trimming logic is implemented by building a few lists and then sort them. When the cache is large enough, those lists could easily go into large object heap, consuming more memory and making GC more expensive. A better approach would be to use a segmented list, avoiding LOH allocations.

5. MemoryCache is implemented using single ConcurrentDictionary, not partitioned as in System.Runtime.Caching.MemoryCache). Basically, there is just one single huge array. This is inefficient for garbage collection in large server processes running on high core count machines.

Here is the 104 byte CacheEntry object in cdb:

Lots of pointers, lots of 64-bit value fields, key is duplicated. There are 4 64-bit time related fields!

Notice this is already the recently improved version (7.0.0); older versions are even worse.

Here is caching trimming logic:

The code is building four lists. Element size for the first list is 8 bytes, 16 bytes for the other three lists. So it just takes more than 4,096 items to reach large object heap (4 K x 16 = 64 kb, the next resize is 132 kb in LOH).

If you really need a memory cache, here are my suggestions:

1. For simple usage patterns, you can just use ConcurrentDictionary.

2. For large cache, the best option is to clone source code of MemoryCache + ConcurrentDictionary, merge CacheEntry into ConcurrentDictionary’s Node object, and then reduce the size of merged object. In trimming logic, switch to use segmented list.