Easing the Burdens of HPC File Management

Appeared in Proceedings of the 6th Parallel Data Storage Workshop (PDSW '11).

Abstract

While the amount of data we can process and store grows, our ability to find data remains dependent upon our own memories more often than not. Manual metadata management is common among scientific users, consuming their time while not making use of the computing resources at hand. Our system design proposes to empower users with more powerful data finding tools, such as unified search spaces, provenance, and ranked file system search. By returning the responsibility of file management to the file system, we enable scientists to focus on their science without the need for a customized file organization scheme for their work.

Publication date:
November 2011

Authors:
Stephanie Jones
Christina Strong
Aleatha Parker-Wood
Alexandra Holloway
Darrell D. E. Long

Projects:
Scalable File System Indexing
Dynamic Non-Hierarchical File Systems

Available media

Full paper text: PDF

Bibtex entry

@inproceedings{jones-pdsw11,
  author       = {Stephanie Jones and Christina Strong and Aleatha Parker-Wood and Alexandra Holloway and Darrell D. E. Long},
  title        = {Easing the Burdens of {HPC} File Management},
  booktitle    = {Proceedings of the 6th Parallel Data Storage Workshop (PDSW '11)},
  month        = nov,
  year         = {2011},
}