ARIMA Security Metrics: Facilitating Decision-making Processes and Situational Awareness in Threat Intelligence

by Jan Kohlrausch (DFN-CERT)

At DFN-CERT, we work on augmenting Security Metrics with a family of stochastic models. For a given Security Metric, an Autoregressive Integrated Moving Average (ARIMA) model is selected that encapsulates the sequence of metrics results and provides objective mathematical properties. This additional mathematical layer results in a better understanding of the metrics properties, facilitates decision-making processes, and supports situational awareness in Threat Intelligence.

Security Metrics and Threat Intelligence
Following the National Institute of Standards and Technology (NIST), Threat Intelligence and Security Metrics can be briefly summarised as follows:

• Threat Intelligence can be characterised as refined threat information supporting decision-making processes ([L1]). Moreover, NIST defines threat information as any information that can help an organisation to identify, assess, monitor, and respond to Cyber threats (e.g., Indicators of Compromise).
• Security Metrics are a tool to facilitate decision-making processes ([L2]). Technically, metrics usually quantify or measure security data. For example, an important efficiency metric in incident response is the "mean-time-to-fix" recovering from an attack.

Building upon the common objective of facilitating decision-making processes, Security Metrics are a promising tool implementing the refinement process for Threat Intelligence.

Augmenting Security Metrics with ARIMA Models
Considerable efforts have been spent on Security Metrics, addressing the design of metrics (e.g., "SMART" criteria) and their classification into different groups. In practice, a large number of metrics for Information Security has been proposed. In almost all of these metrics, the underlying technical result is a sequence of time-dependent measurements (aka time series). According to the definition, this sequence has to facilitate or drive the decision-making process of the corresponding metric. Our key finding is that this process benefits by mathematically modelling the metrics sequence of results, providing additional objective data properties. We selected ARIMA models because they are a solid and well-researched approach stochastically and have already proved their applicability for many similar use cases. From an operational point of view, the ARIMA data model provides an additional mathematical layer that improves a Security Metric by the following properties:

Estimating measurement and data uncertainties: ARIMA models allows estimation of the uncertainties and variances affecting the measurements of a Security Metric. It is important to note, that this especially works for random errors; however, systematic errors have to be addressed by other means.

• Predicting metrics values and detecting anomalies: Upcoming results and their margin of uncertainty are predicted by ARIMA models, allowing the deduction that a new value does not fit into the series of historic results. Such anomalies (mathematically outliers) are all measured values that are outside of the predicted uncertainty margin (confidence interval). Since anomalies may be caused by malicious activity pertaining to a severe incident, they require a further analysis to identify their root cause. Technically, this could be addressed by putting additional sophisticated sensors in place, providing data for a more detailed analysis of the attack activity.
• Facilitating decision-making processes based on objective criteria: The ARIMA approach provides a detailed model of the metrics results. In addition to anomaly detection, time series analysis provides information whether the results are statistically constant (stationary stochastic process) or if there is a constant change that can be caused by a linear trend. Detecting trends is important, because they indicate a changing threat landscape, which may have a considerable impact on security governance. Thus, these mathematical properties are of great importance and complement or may even substitute an interpretation based on gut feeling or visual inspection.

The process of augmenting Security Metrics with ARIMA models is further detailed in [1].

For the previously mentioned "mean-time-to-fix" metric, an ARIMA model substitutes the simple “mean” by a mathematical model that accounts for more complex data properties. As detailed above, the model allows the prediction of the time-to-fix for future incidents, which has a serious impact if incidents grow in complexity or severity over time.

Application for Situational Awareness in Threat Intelligence
Because of the capabilities of ARIMA Security Metrics detecting anomalies, situational awareness in Threat Intelligence becomes a promising field of application. In the following, we demonstrate how an ARIMA Security Metric is applied to detect a significant increase in attack activity against the Microsoft Azure cloud service pertaining to the "OMIGOD" exploit. It is based on the report and data of the Internet Storm Center (ISC) of the SANS institute that was published on 20 September 2021 [L3].

The ISC data that represent the results of the ARIMA Metric “Daily number of targets being attacked on port TCP/12702 are shown in Figure 1 (blue line). A rapid rise of attacks can be seen starting on 15 September 2021. For fitting the ARIMA model, we used the Python module "statsmodels". Based on the Box-Jenkins method, the ARIMA (2,0,2) model has been selected and applied to predict in-sample data points (orange dashed line) and the 95% confidence intervals of the prediction (gray area between red and green dotted lines). It can be seen that the ARIMA Security Metric reliably identified the rapid rise in attack activity on 15 September 2021 (the measured value exceeds the 95% confidence interval by a lot). Although the rise can be easily spotted, manual inspections or interpretations do not scale for a larger number of graphs. In contrast, ARIMA Security Metrics are an ideal tool for automatically monitoring a virtually unlimited number of time series to detect unusual activity requiring a deeper analysis. Thus, ARIMA Security Metrics can direct the attention to the critical events or situations that require further analysis.

Figure 1: Number of targets being attacked on port TCP/1270 (ISC data pertaining to OMIGOD report [L3]) and ARIMA analysis (in-sample prediction and confidence interval).

ARIMA Security Metrics are currently deployed in the CONCORDIA Horizon 2020 project [L4] delivering attack landscape awareness for the data in the Treat Intelligence platform.

Links:

[L1] https://csrc.nist.gov/publications/detail/sp/800-150/final
[L2] https://csrc.nist.gov/publications/detail/sp/800-55/rev-1/final
[L3] https://isc.sans.edu/forums/diary/OMIGOD+Exploits+Captured+in+the+Wild+Researchers+responsible+for+half+of+scans+for+related+ports/27852
[L4] https://www.concordia-h2020.eu/

Reference:
[1] J. Kohlrausch, E. A. Brin: “ARIMA Supplemented Security Metrics for Quality Assurance and Situational Awareness”, in Digital Threats: Research and Practice, Volume 1, Issue 1, March 2020, https://doi.org/10.1145/3376926

Please contact:
Jan Kohlrausch
DFN-CERT, Germany
This email address is being protected from spambots. You need JavaScript enabled to view it.