Tuesday, January 8, 2013

Part 5: Understanding Amazon ElastiCache Internals: Memory Allocation and Eviction

Amazon ElastiCache internally uses Memcached 1.4.5 engine. We are going to explore the memory allocation and eviction policies of Amazon ElastiCache through our understanding of memcached internals in this article. If you are a complex and extensive user of Amazon ElastiCache it is better to have detailed understanding of how the internals work, to avoid leakages and maintain overall optimum performance.

How is it organized internally?

Amazon ElastiCache node usually breaks the allocated memory into smaller parts called pages. Each page is usually 1 megabyte in size. Each page is then assigned to a slab class when necessary. Each slab class is in turn divided into chunks of a specific size. The chunks in each slab have the same size.
Following diagram illustrates the above divisions:

If you are using memcached on EC2 you can view the slab, chunk and byte details by using the following command $. /memcached -vv
$ ./memcached -vv
slab class   1: chunk size        80 perslab   13107
slab class   2: chunk size       104 perslab   10082
slab class   3: chunk size       136 perslab    7710
slab class   4: chunk size       176 perslab    5957
slab class   5: chunk size       224 perslab    4681
slab class   6: chunk size       280 perslab    3744
slab class   7: chunk size       352 perslab    2978
slab class   8: chunk size       440 perslab    2383
slab class   9: chunk size       552 perslab    1899
slab class  10: chunk size       696 perslab    1506

From the output of the above command you can observe that there is page with 80 byte chunks (slab class 1), a page with 104 byte chunks (slab class 2). In slab class 1, each chunk is 80 bytes and each page can then contain 13,107 chunks (1 MB / 80 bytes). This continues all the way up the slab chain till 1 megabyte (with growth factor of ~1.25). Note: There can be multiple pages assigned for each slab-class, but as soon as a page is assigned to a slab-class, it is permanent.
When you are storing items in Amazon ElastiCache, they are pushed into the slab class of the nearest fit. If your key + miscellaneous data + value is 70 bytes total, it will go into slab class 1, with an overhead loss of 10 bytes per item. If your data is 90 bytes total, it will go into Slab class 2, with an overhead of 14 bytes. If your cache access pattern ends up putting 90% of your pages in slab class 2, there will be less memory available for slab class 3. This simple model is followed by memcached engine to avoid memory defragmentation and get better speed in performance versus memory usage tradeoff.
When a page is full (meaning all chunks in the page are filled) and we still need to add another item, engine will fetch a new free page from the pool, assign it to the specified slab-class (depending upon item size), partitions it into chunks and gets the first available chunk to store the new item. Note: These 1024 byte pages are assigned on a FCFS basis (first come-first served), to the slab classes.
When there are no more pages left to assign to slab class, it will use LRU algorithm to evict items to reclaim memory. In the following section you can see in detail about the eviction mechanism of Amazon ElastiCache works.

When items are evicted?

Memory for an item is not actively reclaimed in Amazon ElastiCache. The memcached engine does not have in-built background threads that explicitly expires item and reclaims chunks. For instance, if you store an item and it expires, it still sits in the LRU cache at its position, until it falls to the end of the cache and is reused. However, if you fetch an expired item, memcached will find that the item is expired and free its memory for reuse.
To explain eviction mechanism in detail; Items are evicted from Amazon ElastiCache if they are expired or the slab class is completely out of free chunks and there are no free pages to assign to a slab class. In case there are no free chunks, or no free pages in the appropriate slab class, Amazon ElastiCache will look at the LRU in tail to reclaim an item. Basically, it will search the last few items in the “end” and identifies the ones that are already expired, makes it free for reuse. If it cannot find an expired item on the end, it will "evict" one which has not yet expired. Actually you could end up with one slab class constantly evicting recently used items, on the other hand another slab having a bunch of old items that just sit around. For example: When we need a 104 byte chunk, it will evict a 104 byte chunk, even though there might be a 280 byte chunk that is even older.  This explains the internal workings that “Each slab-class has its own LRU and statistical counter updates, it behaves like a separate cache itself, it is not global LRU, but slab class LRU in short”.

How to reduce memory overheads?

If you are running Amazon ElastiCache clusters of few MB/GB overheads will not matter much. But if you are running Amazon ElastiCache clusters spanning in Hundreds of GB- > TB, you will end up losing lots of allocated memory as overheads if the chunk size and growth factor is not planned properly.
Since memcached engine of Amazon ElastiCache does defrag, you need to plan for better compaction and avoid overheads to keep the chunk sizes closer to your item sizes. This is not a one-time activity, but a periodical activity aligned to your cache growth.
 A simple approach that can be followed is:
·         Identify the item size usage and cache access patterns periodically
·         Set the initial chunk size and growth factors accordingly in AWS console or API's for Amazon ElastiCache. You can also set the factor using (-f) and the initial chunk-size (-s) options if its memcached on EC2.
·         If item sizes are big and predictable it is recommended to have bigger chunks and growth factors, if the item sizes are varied it is better to have smaller initial size and growth factor. This will keep the wastage minimal.

Disclaimer: The article is entirely based on the premise that Amazon ElastiCache uses memcached 1.4.5 as the engine. If AWS team has customized some sections of memcached engine, the author will not be liable for misinforming readers.

No comments:

Need Consulting help ?


Email *

Message *

All posts, comments, views expressed in this blog are my own and does not represent the positions or views of my past, present or future employers. The intention of this blog is to share my experience and views. Content is subject to change without any notice. While I would do my best to quote the original author or copyright owners wherever I reference them, if you find any of the content / images violating copyright, please let me know and I will act upon it immediately. Lastly, I encourage you to share the content of this blog in general with other online communities for non-commercial and educational purposes.