Big Data Mining and Analytics  2020, Vol. 3 Issue (3): 181-195    DOI: 10.26599/BDMA.2020.9020003
Applying Big Data Based Deep Learning System to Intrusion Detection
Wei Zhong*, Ning Yu, Chunyu Ai
Wei Zhong and Chunyu Ai are with the Division of Math and Computer Science, University of South Carolina Upstate, Spartanburg, SC 29303, USA. E-mail: aic@uscupstate.edu.
Ning Yu is with the Department of Computing Sciences, State University of New York College at Brockport, Brockport, NY 14420, USA. E-mail: nyu@brockport.edu.

Abstract

With vast amounts of data being generated daily and the ever increasing interconnectivity of the world’s internet infrastructures, a machine learning based Intrusion Detection Systems (IDS) has become a vital component to protect our economic and national security. Previous shallow learning and deep learning strategies adopt the single learning model approach for intrusion detection. The single learning model approach may experience problems to understand increasingly complicated data distribution of intrusion patterns. Particularly, the single deep learning model may not be effective to capture unique patterns from intrusive attacks having a small number of samples. In order to further enhance the performance of machine learning based IDS, we propose the Big Data based Hierarchical Deep Learning System (BDHDLS). BDHDLS utilizes behavioral features and content features to understand both network traffic characteristics and information stored in the payload. Each deep learning model in the BDHDLS concentrates its efforts to learn the unique data distribution in one cluster. This strategy can increase the detection rate of intrusive attacks as compared to the previous single learning model approaches. Based on parallel training strategy and big data techniques, the model construction time of BDHDLS is reduced substantially when multiple machines are deployed.

Received: 08 March 2020      Published: 15 September 2020
Corresponding Authors: Wei Zhong