Measuring the compressibility of metadata and small files for disk/NVRAM hybrid storage systems

Appeared in Proceedings of the 2004 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS '04).

Abstract

File systems combining disk storage with non-volatile RAM (NVRAM) promise large improvements in file system performance. However, current technology allows for a relatively limited amount of NVRAM, limiting the effectiveness of such an approach. We are examining in-memory compression techniques that allow for significantly more efficient utilization of this limited resource. We focus on small objects—metadata and small files—and we have measured the compressibility of these objects for a set of representative file systems. Our results show that inodes are compressible by at least 76–90% at a rate of 270–900 thousand inodes per second for the best algorithms. For files in the range of 4–128 KB, we achieved an average compressibility of 40–60% at rates of 20–40 megabytes per second. Based on these measurements, we believe that compression of both metadata and small files should be included in any disk/NVRAM hybrid file system.

Publication date:
July 2004

Authors:
Nate Edel
Karl Brandt
Ethan L. Miller
Scott A. Brandt

Projects:
Storage Class Memories

Available media

Full paper text: PDF

Bibtex entry

@inproceedings{edel-spects04,
  author       = {Nate Edel and Karl Brandt and Ethan L. Miller and Scott A. Brandt},
  title        = {Measuring the compressibility of metadata and small files for disk/{NVRAM} hybrid storage systems},
  booktitle    = {Proceedings of the 2004 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS '04)},
  month        = jul,
  year         = {2004},
}