Store, forget, and check: Using algebraic signatures to check remotely administered storage

Appeared in Proceedings of the IEEE Int'l Conference on Distributed Computing Systems (ICDCS '06).

Abstract

The emerging use of the Internet for remote storage and backup has led to the problem of verifying that storage sites in a distributed system indeed store the data; this must often be done in the absence of knowledge of what the data should be. We use m/n erasure-correcting coding to safeguard the stored data and use algebraic signatures—hash functions with algebraic properties—for verification. Our scheme primarily utilizes one such algebraic property: taking a signature of parity gives the same result as taking the parity of the signatures. To make our scheme collusion-resistant, we blind data and parity by XORing them with a pseudo-random stream. Our scheme has three advantages over existing techniques. First, it uses only small messages for verification, an attractive property in a P2P setting where the storing peers often only have a small upstream pipe. Second, it allows verification of challenges across random data without the need for the challenger to compare against the original data. Third, it is highly resistant to coordinated attempts to undetectably modify data. These signature techniques are very fast, running at tens to hundreds of megabytes per second. Because of these properties, the use of algebraic signatures will permit the construction of large-scale distributed storage systems in which large amounts of storage can be verified with minimal network bandwidth.

Publication date:
July 2006

Authors:
Thomas Schwarz
Ethan L. Miller

Projects:
Archival Storage
Secure File and Storage Systems
Reliable Storage

Available media

Full paper text: PDF
Presentation: slides

Bibtex entry

@inproceedings{schwarz-icdcs06,
  author       = {Thomas Schwarz and Ethan L. Miller},
  title        = {Store, forget, and check: Using algebraic signatures to check remotely administered storage},
  booktitle    = {Proceedings of the IEEE Int'l Conference on Distributed Computing Systems (ICDCS '06)},
  month        = jul,
  year         = {2006},
}