论文标题
侵入检测数据集上Apache Spark MLLIB算法的性能评估
Performance Evaluation of Apache Spark MLlib Algorithms on an Intrusion Detection Dataset
论文作者
论文摘要
Internet和Web服务使用的使用以及第五代网络技术（5G）以及不断增长的物联网（IoT）数据流量的出现将增长全球Internet使用情况。为了确保未来网络的安全性，必须实施基于机器的入侵检测和预防系统（IDP）来检测新攻击，并且可以使用大数据并行处理工具来处理这些系统中的大量培训数据。在本文中，Apache Spark，通用和快速集群计算平台用于处理和培训大量网络流量功能数据。在这项工作中，CSE-CIC-IDS2018数据集的最重要功能用于构建机器学习模型，然后是最受欢迎的机器学习方法，即逻辑回归，支持向量机（SVM），三个不同的决策树分类器和Naive Bayes算法，用于训练多达八个工作人员Nodes数量的模型。我们的火花群集包含七台用作工人节点的机器，一台机器既配置为主人又是工人。我们使用CSE-CIC-IDS2018数据集评估这些算法在僵尸网络攻击上的总体性能，并使用分布式超参数调整来找到最佳的单个决策树参数。我们在实验中使用学习方法的选定功能实现了多达100％的精度
The increase in the use of the Internet and web services and the advent of the fifth generation of cellular network technology (5G) along with ever-growing Internet of Things (IoT) data traffic will grow global internet usage. To ensure the security of future networks, machine learning-based intrusion detection and prevention systems (IDPS) must be implemented to detect new attacks, and big data parallel processing tools can be used to handle a huge collection of training data in these systems. In this paper Apache Spark, a general-purpose and fast cluster computing platform is used for processing and training a large volume of network traffic feature data. In this work, the most important features of the CSE-CIC-IDS2018 dataset are used for constructing machine learning models and then the most popular machine learning approaches, namely Logistic Regression, Support Vector Machine (SVM), three different Decision Tree Classifiers, and Naive Bayes algorithm are used to train the model using up to eight number of worker nodes. Our Spark cluster contains seven machines acting as worker nodes and one machine is configured as both a master and a worker. We use the CSE-CIC-IDS2018 dataset to evaluate the overall performance of these algorithms on Botnet attacks and distributed hyperparameter tuning is used to find the best single decision tree parameters. We have achieved up to 100% accuracy using selected features by the learning method in our experiments
