Evaluation of Distributed Recovery in Large-Scale Storage Systems

Appeared in Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing (HPDC 2004).

Abstract

Storage clusters consisting of thousands of disk drives are now being used both for their large capacity and high throughput. However, their reliability is far worse than that of smaller storage systems due to the increased number of storage nodes. RAID technology is no longer sufficient to guarantee the necessary high data reliability for such systems, because disk rebuild time lengthens as disk capacity grows. In this paper, we present FAst Recovery Mechanism (FARM), a distributed recovery approach that exploits excess disk capacity and reduces data recovery time. FARM works in concert with replication and erasure-coding redundancy schemes to dramatically lower the probability of data loss in large-scale storage systems. We have examined essential factors that influence system reliability, performance, and costs, such as failure detections, disk band- width usage for recovery, disk space utilization, disk drive replacement, and system scales, by simulating system behavior under disk failures. Our results show the reliability improvement from FARM and demonstrate the impacts of various factors on system reliability. Using our techniques, system designers will be better able to build multi-petabyte storage systems with much higher reliability at lower cost than previously possible.

Publication date:
June 2004

Authors:
Qin Xin
Ethan L. Miller
Thomas Schwarz

Projects:
Reliable Storage
Ultra-Large Scale Storage

Available media

Full paper text: PDF

Bibtex entry

@inproceedings{xin-hpdc04,
  author       = {Qin Xin and Ethan L. Miller and Thomas Schwarz},
  title        = {Evaluation of Distributed Recovery in Large-Scale Storage Systems},
  booktitle    = {Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing (HPDC 2004)},
  pages        = {172-181},
  month        = jun,
  year         = {2004},
}