by Jan Stampfli and Kurt Stockinger (Zurich University of Applied Sciences)
A novel alarm verification service applying various machine learning algorithms can identify false alarms.
False alarms triggered by sensors of alarm systems are a frequent and costly inconvenience for the emergency services and owners of alarm systems. Around 90% of false alarms are caused by either technical failures such as network downtimes or human error.
To remedy this problem, we develop a novel alarm verification service by leveraging the power of an alarm data warehouse. In addition, we apply various machine learning algorithms to identify false alarms. The goal of our system is to help human responders in their decision about whether or not to trigger costly intervention forces.
Approach
We are working with a security company that is a major player in secure alarm transmission with the aim of significantly reducing the number of false alarms. Alarms can be triggered by devices installed at banks, jewellery stores, private homes, etc.
The problem of alarm prediction is conceptually similar to anomaly detection [1] or prediction of failures [2]. Hence, we can borrow some ideas from these fields and take advantage of the latest progress in deep learning [3]. We have chosen four machine learning approaches to predict false alarms:
- Random forests
- Support vector machines
- Logistic regression
- Deep neural networks.
We based our experiments on a set of more than 300,000 alarms. For each alarm we had multiple features such as device ID, device location, type of alarm, alarm trigger time, etc. We used these features as input for our machine learning approaches. An overview of the system architecture is given in Figure 1.
Figure 1: Overview of our alarm prediction system: alarm streams are stored in a data-warehouse and each alarm is evaluated as true or false.
For security reasons we do not have direct information about whether an alarm is a false alarm or not. However, we have indirect information that we can use as labels for our machine learning approach. In particular, for each alarm we know when it was triggered and when it was reset again. Our hypothesis is, that if the time difference delta t between triggering and resetting an alarm is small, there is a high chance that the alarm is false.
Consider the case of an alarm system deployed at a private home. Further assume that kids or a pet triggered an alarm. In this case, the parents or pet owners might call the alarm receiving centre and inform them about a false alarm. In any case, the alarm receiving centre will reset the alarm after verification of the alarm system responsible or the caller identity. In this case, the delta t, i.e., the time between triggering an alarm and resetting it is very small. Details about the four experiments are beyond the scope of this article but can be found in our technical report [L1].
In summary, the goal of our machine learning approach is to learn whether the delta t is below or above a certain threshold value.
Results
In order to evaluate the effectiveness of our machine learning approach, we experimented with various values for delta t ranging between 1 and 10 minutes. The goals of our evaluation were as follows:
- Evaluate the accuracy of four different machine learning algorithms
- Study the impact of various deltas t on the prediction accuracy.
For the training and evaluation of our machine learning approach, we split the data into two sets containing about 170,000 alarms each. Apart from support vector machines, we observe that the performance of the algorithms is not affected by the deltas t (see Figure 2). Random forest and deep neural networks show the best performance with a prediction accuracy of up to 92%. These results show that our system is reliable even if we replace our hypothetical labels when we get access to the real ground truth in the future.
Figure 2: Evaluation of the machine learning algorithms used to predict alarm validity: prediction accuracy vs. delta t.
Conclusions
The results demonstrate that our machine learning approaches are very effective for predicting false alarms with an accuracy of up to 92%. These results can be directly used by typical alarm receiving centres for prioritising alarms and thus have a large potential to significantly reduce the costs of dispatching intervention forces.
As part of future work we will integrate this machine learning approach into our alarm data warehouse to enable stream and batch processing. The idea is to apply the machine learning algorithms in real time on alarm streams and correlate the results with the alarm response to potentially further increase the accuracy of false alarm prediction.
Link:
[L1] http://pd.zhaw.ch/publikation/upload/210931.pdf
References:
[1] V. Chandola, A. Banerjee, V. Kumar: “Anomaly detection: A survey”, ACM Comput. Surv. 41, 3, Article 15, 2009.
[2] F. Salfner, M. Lenk, M. Malek: “A survey of online failure prediction methods”, ACM Comput. Surv. 42, 3, Article 10, 2010.
[3] Y. LeCun, Y. Bengio, G. Hinton: “Deep learning”, Nature, 521(7553), 436-444, 2015.
Please contact:
Kurt Stockinger
Zurich University of Applied Sciences, Switzerland
