by Michael Sonntag (Johannes Kepler Universität Linz) and Michael Hofstätter (AIT)
Terrorism represents a serious threat to human life and infrastructure, and it is vital that police officers are equipped to investigate suspicious luggage, rubbish bins that might have been tampered with, etc., for improvised or unconventional explosive devices. Common investigation techniques currently include video cameras and transmission x-rays on mobile robots. Modern and advanced sensors are being incorporated into a prototype to allow more detailed investigations, while simultaneously ensuring the data is usable as evidence.
When a multitude of sensors are contained within a robot that investigates suspicious objects, it is important to be able to combine their output: E.g. if chemicals or radioactive material are detected, it is necessary to know their location in relation to the visual video. Similarly, infrared pictures or x-rays must “overlay” the normal video. As the robot will often be needed to operate in areas where no detailed 3D-plans of the building or outdoor area exist, acquiring an environment model and the exact localisation of the robot are also necessary for efficient handling. All this requires the fusion of a multitude of sensors outputs to rapidly produce an integrated display.
Figure 1: Overview of the project.
The Durchblick projects aims to create a prototype of such a system, based on robots already used by the police and the military, by extending them with new sensors and a significantly enhanced base station (see Figure 1). The first part of the project is therefore to investigate physically mounting multiple sensors (with the option for quick exchange of the sensor loadout) and integrating their communication. Based on this a prototype with selected sensors will be built and tested. Because of size and weight, some sensors will not be carried by the robot for the tests (mass spectroscopy), but with future miniaturisation or more powerful platforms, these might enter practical use too. Another part of the project is the communication with the base station, where the fusion of the sensor data will take place. A large amount of data needs to be transferred quickly (e.g. a laser scan of the surrounding environment (e.g. room or the local area) might require hundreds of MB), but at other times a continuous stream is necessary (e.g. visible/IR video). Because of the high bandwidth that is sometimes required the prototype will use a glass fibre cable for communication. To be independent of the transmission medium and fulfil security properties, a separate encryption layer is necessary. To simplify its use, secure USB modules (physically tamper-proof) are under investigation to distribute/install the keys needed at the various elements.
The individual sensor data is combined at the base station and not the robot, as this requires significant computing power (weight and power on the robot are limited). This includes not only the visualisation of individual sensors, but also the combination of several sensor’s outputs. An additional goal is the improvement of the human-machine interface. Investigating/defusing is a stressful situation and important data must be displayed prominently, while not producing additional stress or sensory overload. Any elements detected as dangerous must be clearly highlighted in all kinds of display. An additional complication is that operators might wear heavy protective gear, limiting manual dexterity as well as perception.
Figure 2: System with detailed view on the forensic support.
A final task of the project is to investigate gathering and preserving evidence, i.e. forensics support. This is particularly important because little may be left of a bomb following an explosion, and equally, it is important to be able to provide valid justification in situations in which harmless luggage is investigated or even damaged/destroyed. For this a separate forensics server is planned (see Figure 2). All data transmission to and from the robot will be stored as a network dump. This is not directly usable but enables detailed investigations. As the sensor protocols and commands employed are unencrypted (security is provided by an additional layer; see above), this is feasible. All sensors are accessed via this network, so their presence is also automatically identified (logging of DHCP, regular scans for devices etc). Because this only provides access to raw data, but no sensor data fusion output, for example, these results will have to be added and stored as files. For security and privacy reasons encrypted and signed data storage to prevent modifications and ensure confidentiality are investigated. Again, for simplified use of the security functions, small hardware security modules might be employed.
In conclusion, existing technology needs to be integrated to become usable in this field of work, whereby numerous outside restrictions (size, weight, power, security, evidence etc) need to be fulfilled. The Durchblick project also cooperates with a similar German project (of the same name), which focuses on a larger set of different sensors, but leaves out data fusion and forensics as a second focus.
This project has been funded by the Austrian security research program KIRAS of the Federal Ministry for Transport, Innovation and Technology (bmvit) (Grant Nr. 854756).
Please contact:
Michael Sonntag, Johannes Kepler University Linz, Austria +43 732 2468-4137,
Michael Hofstätter, AIT Austrian Institute of Technology, Austria, +43 50550-4202,
