Disk Scrubbing in Large Archival Storage Systems

Appeared in Proceedings of the 12th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS '04). Won Best Paper award.

Abstract

Large archival storage systems experience long periods of idleness broken up by rare data accesses. In such systems, disks may remain powered off for long periods of time. These systems can lose data for a variety of reasons, including failures at both the device level and the block level. To deal with these failures, we must detect them early enough to be able to use the redundancy built into the storage system. We propose a process called “disk scrubbing” in a system in which drives are periodically accessed to detect drive failure. By scrubbing all of the data stored on all of the disks, we can detect block failures and compensate for them by rebuilding the affected blocks. Our research shows how the scheduling of disk scrubbing affects overall system reliability, and that “opportunistic” scrubbing, in which the system scrubs disks only when they are powered on for other reasons, performs very well without the need to power on disks solely to check them.

Publication date:
October 2004

Authors:
Thomas Schwarz
Qin Xin
Ethan L. Miller
Darrell D. E. Long
Andy Hospodor
Spencer Ng

Projects:
Archival Storage
Reliable Storage
Ultra-Large Scale Storage

Available media

Full paper text: PDF

Bibtex entry

@inproceedings{schwarz-mascots04,
  author       = {Thomas Schwarz and Qin Xin and Ethan L. Miller and Darrell D. E. Long and Andy Hospodor and Spencer Ng},
  title        = {Disk Scrubbing in Large Archival Storage Systems},
  booktitle    = {Proceedings of the 12th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS '04)},
  pages        = {409-418},
  month        = oct,
  year         = {2004},
}