Improving Key-Value Cache Performance With Heterogeneous Memory Tiering: A Case Study of Compute-Express-Link-Based Memory Expansion
KyungSoo Lee, Sohyun Kim, Joohee Lee, Donguk Moon, R. Kim, Honggyu Kim, Hyeongtak Ji, Yunjeong Mun, Youngpyo Joo
Abstract
CXL memory brings extra bandwidth and capacity via PCIe-based memory expansion beyond DDR-based DRAM. This paper introduces the CXL 2.0 memory expansion solution, which incorporates two parts: 1) a CXL memory expander prototype, and 2) the Heterogeneous Memory Software Development Kit (HMSDK). We demonstrate the feasibility of our CXL memory solution by implementing it on CacheLib, Meta’s general-purpose key-valye caching engine. We highlight how our application design and guidelines for CXL memory enable resolving the shortcomings of conventional memory system architectures. Our proposals enable 1) expanding memory bandwidth and capacity or 2) considerable DRAM savings. Evaluation results show that we can achieve a 25% increase in memory bandwidth, up to 15% throughput gain, and a 9% latency reduction. Furthermore, in Hybrid cache using NVM, expanding the RAM cache area with CXL memory, which is relatively cheaper than DRAM, enhances throughput and hit ratio due to reduced NVM I/O.