High-Speed Security Analytics Powered by In-Memory Machine Learning Engine

Modern Security Information and Event Management systems should be capable to store and process high amount of events or log messages in different formats and from different sources. This requirement often prevents such systems from usage of computational-heavy algorithms for security analysis. To d...

Full description

Saved in:
Bibliographic Details
Main Authors: Sapegin, Andrey, Gawron, Marian, Jaeger, David, Feng Cheng, Meinel, Christoph
Format: Conference Proceeding
Language:English
Subjects:
Algorithm design and analysis
Computers
in-memory
intrusion detection
Libraries
machine learning
Machine learning algorithms
Prediction algorithms
SAP HANA
Security
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Modern Security Information and Event Management systems should be capable to store and process high amount of events or log messages in different formats and from different sources. This requirement often prevents such systems from usage of computational-heavy algorithms for security analysis. To deal with this issue, we built our system based on an in-memory data base with an integrated machine learning library, namely SAP HANA. Three approaches, i.e. (1) deep normalisation of log messages (2) storing data in the main memory and (3) running data analysis directly in the database, allow us to increase processing speed in such a way, that machine learning analysis of security events becomes possible nearly in real-time. To prove our concepts, we measured the processing speed for the developed system on the data generated using Active Directory tested and showed the efficiency of our approach for high-speed analysis of security events.
ISSN:2379-5352
DOI:10.1109/ISPDC.2015.16