by Markus Wurzenberger and Florian Skopik (AIT Austrian Institute of Technology)
Recent years have seen a dramatic increase in damage caused by cyber-criminals. Although there are many IT security tools on the market, there is currently no way to test, compare and evaluate them without actually running them in real systems. The BÆSE testbed offers a solution to challenge and benchmark IT security solutions for dedicated network environments under attack conditions, without putting real systems in danger.
The demand for interconnected digital services has been growing rapidly in recent years. With the introduction of low-cost devices, which are even used in industrial environments now, the emergence of cyber physical systems (CPS) spanning widely spread network components has begun. While this broad connectivity improves society’s productivity and optimises industrial production through automation, it makes organisations and private life vulnerable to cyber-attacks. Modern attacks are sophisticated, tailored to specific purposes and often use customised tools. Consequently, prompt detection is difficult and victims are often oblivious to the attack. Attacks typically result in data breaches, high financial losses (actual reports estimate $500 billion per year [L1]) and damage to reputation.
Actual Situation and Problem Statement
There are various types of IT security tools in use, including: anomaly detection systems, intrusion detection systems (IDS), antivirus scanner and security information event management (SIEM) tools. These systems aim to promptly detect attacks, but they are often inadequate when it comes to recognising sophisticated and tailored intrusions. While the market for security is growing (projected to rise from $75 billion in 2015 to $170 billion in 2020 [L2]) and vendors are entering the market with new products in ever shorter cycles, the financial losses are still growing.
While many great IT security solutions have been developed, the problem of how to rate, compare and evaluate them to facilitate their optimal configuration and application in a specific organisational context remains unsolved [1]. This is because there is no effective way of testing solutions prior to their deployment. Challenging IT security solutions with attacks under realistic circumstances is mandatory to rate their detection capabilities. However, this needs to be performed in the target environment – or at least in an environment which simulates the target environment as closely as possible to rate the individual detection capabilities, since the structure of a network, and the way individuals use it, differs from one organisation to another. Unfortunately, this is a highly non-trivial task.
Goals and Innovations
In the BÆSE (Benchmarking and Analytic Evaluation of IDSs in Specified Environments) project, we invent the BÆSE testbed, which allows vendor-independent evaluation and comparison of IT security tools for user-specified environments. There are two steps to the BAESE testbed approach:
(i) Generate semi-synthetic test data based on the properties and characteristics of a specified network environment (see Figure 1).
(ii) Feed the generated test data into various detective IT security tools for comparing, rating and evaluating their capabilities with respect to different configurations using the BÆSE Testbed (see Figure 2).
Figure 1: NES data generation process.
Figure 2: BÆSE testbed.
Part (i) of the model takes a small part of captured logging, netflow, or packet data from a real network environment as input. This input is processed with machine learning methods to obtain the properties and characteristics of the considered network. For example in the case of log data, clustering can be used to group the log lines. Based on the results of the machine learning algorithms, stable and variable parts of the data within a cluster are determined. Furthermore, the transition probabilities between log lines representing various sorts of events are stored as well as the temporal distribution of these events. This information then allows the application of a Markov chain simulation to generate a semi-synthetic test data file of any size, which in Figure 1 is named Network Event Sequence (NES) since our approach is not limited to log data and can also be used, for example, with netflow and packet data. Iterative and interactive refinement of the analysis and the model allows NES data to be generated with varying degrees of detail. A detailed description of the approach can be found in [2].
Part (ii) describes the main building block of the BÆSE testbed. The testbed takes as input different sets of NES data, and a choice of IT security tools, which are evaluated with different configurations. The BÆSE testbed then compares and rates the tools and configurations, which are considered for evaluation. The outputs of the BÆSE testbed are reports, recommendation and statistics. Furthermore we plan to develop a concept to simulate realistic attacks in the NES data to stress the IT security tools in an appropriate way. This will allow an evaluation of security solutions under most realistic circumstances for a specified network environment, without exposing the real network infrastructure to risky situations. The objectives of BÆSE are to:
- find the optimal security solution
- optimise the usage of security tools
- find the most efficient configurations of security tools
- raise the detection capability of security tools.
The BÆSE Project
The BÆSE project is financially supported by the Austrian Research Promotion Agency FFG under grant number 852301, and carried out in the course of an industry-related PhD thesis. Project stakeholders are the Austrian Institute of Technology (AIT) as coordinator, the Vienna University of Technologies as academic partner, and T-Systems Austria. AIT has extensive experience in anomaly detection and is developing its own IDS named Automatic Event Correlation for Incident Detection (AECID).
Links:
[L1] http://www.forbes.com/sites/stevemorgan/2016/01/17/cyber-crime-costs-projected-to-reach-2-trillion-by-2019/#16549f753bb0
[L2] http://cybersecurityventures.com/cybersecurity-market-report/
References:
[1] ISO/IEC 27039, Information Technology – Security techniques – Selection, deployment and operations of intrusion detection systems.
[2] M. Wurzenberger, F. Skopik, G. Settanni, and W. Scherrer, “Complex log file synthesis for rapid sandbox-benchmarking of security- and computer network analysis tools,” Information Systems, vol. 60, pp. 13–33, Aug. 2016.
Please contact:
Markus Wurzenberger
AIT Austrian Institute of Technology, Austria
+43 664 8157937
Florian Skopik
AIT Austrian Institute of Technology, Austria
+43 664 8251495
