Logan: Automatic Management for Evolvable, Large-Scale, Archival Storage

Appeared in Proceedings of the 2008 Petascale Data Storage Workshop (PDSW 08).

Abstract

Archival storage systems designed to preserve scientific data, business data, and consumer data must maintain and safeguard tens to hundreds of petabytes of data on tens of thousands of media for decades. Such systems are currently designed in the same way as higher-performance, shorter-term storage systems, which have a useful lifetime but must be replaced in their entirety via a “fork-lift” upgrade. Thus, while existing solutions can provide good energy efficiency and relatively low cost, they do not adapt well to continuous improvements in technology, becoming less efficient relative to current technology as they age. In an archival storage environment, this paradigm implies an endless series of wholesale migrations and upgrades to remain efficient and up to date.

Our approach, Logan, manages node addition, removal, and failure on a distributed network of intelligent storage appliances, allowing the system to gradually evolve as device technology advances. By automatically handling most of the common administration chores—integrating new devices into the system, managing groups of devices that work together to provide redundancy, and recovering from failed devices—Logan reduces management overhead and thus cost. Logan can also improve cost and space efficiency by identifying and decommissioning outdated devices, thus reducing space and power requirements for the archival storage system.

Publication date:
November 2008

Authors:
Mark W. Storer
Kevin Greenan
Ian Adams
Ethan L. Miller
Darrell D. E. Long
Kaladhar Voruganti

Projects:
Archival Storage

Available media

Full paper text: PDF

Bibtex entry

@inproceedings{storer-pdsw08,
  author       = {Mark W. Storer and Kevin Greenan and Ian Adams and Ethan L. Miller and Darrell D. E. Long and Kaladhar Voruganti},
  title        = {Logan: Automatic Management for Evolvable, Large-Scale, Archival Storage},
  booktitle    = {Proceedings of the 2008 Petascale Data Storage Workshop (PDSW 08)},
  month        = nov,
  year         = {2008},
}