An Adaptive Erasure-Coded Storage Scheme with an Efficient Code-Switching Algorithm
Zizhong Wang, Haixia Wang, Airan Shao, Dongsheng Wang
Abstract
Many distributed storage systems use erasure codes rather than replication for higher reliability at significantly lower storage costs. However, using traditional erasure codes increases consumption of network traffic and disk I/O tremendously when systems recover data, resulting in high latency of degraded reads. In order to mitigate this problem, we present an adaptive storage scheme based on data access skew, a fact that most data accesses are applied in a small fraction of data. In this scheme, we use both a Local Reconstruction Code (LRC) to store frequently accessed data, and a Hitchhiker (HH) code to store infrequently accessed data. Besides, an efficient switching algorithm between LRC and HH code with low network and computation costs is provided. The whole system will benefit from low degraded read latency while keeping a low storage overhead, and code-switching will not become a bottleneck. Experimental evaluation shows that this adaptive storage scheme’s performance was in line with expectations.