Pagoda: A Hybrid Approach to Enable Efficient Real-time Provenance Based Intrusion Detection in Big Data Environments

Appeared in IEEE transactions on dependable and secure computing .

Abstract

Efficient intrusion detection and analysis of the security landscape in big data environments present challenge for today’s
users. Intrusion behavior can be described by provenance graphs that record the dependency relationships between intrusion
processes and the infected files. Existing intrusion detection methods typically analyze and identify the anomaly either in a single
provenance path or the whole provenance graph, neither of which can achieve the benefit on both detection accuracy and detection
time. We propose Pagoda, a hybrid approach that takes into account the anomaly degree of both a single provenance path and the
whole provenance graph. It can identify intrusion quickly if a serious compromise has been found on one path, and can further improve
the detection rate by considering the behavior representation in the whole provenance graph. Pagoda uses a persistent memory
database to store provenance and aggregates multiple similar items into one provenance record to maximumly reduce unnecessary
I/O during the detection analysis. In addition, it encodes duplicate items in the rule database and filters noise that does not contain
intrusion information. The experimental results on a wide variety of real-world applications demonstrate its performance and efficiency.

Publication date:
August 2018

Authors:
Yulai Xie
Dan Feng
Yuchong Hu
Yan Li
Staunton Sample
Darrell D. E. Long

Projects:

Available media

Full paper text: PDF

Bibtex entry

@article{xie-tdsc18,
  author       = {Yulai Xie and Dan Feng and Yuchong Hu and Yan Li and Staunton Sample and Darrell D. E. Long},
  title        = {Pagoda: A Hybrid Approach to Enable Efficient Real-time Provenance Based Intrusion Detection in Big Data Environments},
  journal      = {IEEE transactions on dependable and secure computing},
  volume       = {},
  month        = aug,
  year         = {2018},
}