by Zoltán Fazekas, Gábor Balázs, and Péter Gáspár (MTA SZTAKI)
In a pilot-study, an urban road environment detection function was considered for smart cars, as well as for self-driving cars. The implemented artificial neural network (ANN) based algorithms use the traffic sign (TS) and/or crossroad (CR) occurrences, along a route, as input. The TS-based and the CR-based classifiers were then merged into a compound one. The way this was accomplished serves as a simple, practical example of how to build upon modularity and how to retain some degree of it in functioning ANNs.
Modularity is a desired property of built systems; it serves a range of engineering demands throughout the system life-cycle (e.g., traceability, reusability). With respect to artificial neural networks (ANN), the latter property pertains to the network configuration. Modularity, however, comes at a price (e.g., additional layers, lower precision) [1]. In a pilot-study run by the Institute for Computer Science and Control (MTA SZTAKI), Budapest, Hungary, the urban socio-economic road environment detection (RoED) task was considered. The intention was to rely on highly processed, low-volume data as input that are available in smart cars. After the data collection car trips (e.g., in Csepel, see Figure 2), methods were sought that can infer the environment type along a route. The road environment (RE) information could be leveraged both in smart cars and in self-driving cars. A statistical change detection method ‒ applied to traffic sign (TS) occurrences ‒ was presented in [3], and an ANN-based one in [4]. Due to the input choice, the availability of an on-board TS recognition system is essential for any practical utilisation.
The usefulness of the RoED function was confirmed by a driving simulation study [2]. It examined the effect of driving experience on drivers’ adaptation to changing RE complexity in urban areas. Three complexity levels ‒ corresponding to the RE types considered herein ‒ were used. The drivers were grouped into three groups according to their experience. The most experienced drivers adapted better to increasingly complex REs than those in the other two groups. Thus, for less experienced drivers, the RoED function would be beneficial indeed; in self-driving cars, the RE information could be drawn upon to choose vehicle speed and acceleration.
In the pilot-study, ANNs exhibiting different levels of modularity were also looked at, and their detection performances were compared. The implemented ANN-based RoED classifier ‒ that relies on recognised TS and crossroad (CR) occurrences ‒ constitutes a simple practical example on how to build upon modularity in ANNs (e.g., by setting up a modular initial ANN), and how to retain some degree of it (e.g., in form of separable subnetworks, or subnetworks with limited interconnections, see Figure 1) even if some performance improvement seems necessary.
Figure 1: A full non-modular ANN with a random 10% selection of inter-modular synapses, i.e., the synapses of the modular TS and CR processing network (light grey lines) are augmented with those between the TS and CR processing subnetworks (black lines). The ANN simulations and the diagram was made with the Simbrain 3.0. Neural Networks framework.
Figure 2: Road environment types ‒ namely, downtown (orange line), industrial/commercial (dark grey line), and residential (green line) environments ‒ inferred by a modular full ANN along a route in Csepel, Hungary. (Map data: OpenStreetView, map-editor: QGIS 2.8.1.)
Figure 3: The road environment types manually recorded along a test-route in Csepel, Hungary (with the path-lengths shown below the stripes) serving as ground truth (middle), and those inferred by a modular full ANN (top) and by a non-modular one (bottom). The latter ANN comprises a random 40% selection of the possible inter-modular synapses. The colour scheme is the same as in Figure 2.
The urban RE surrounding an ego-car is classified into one of the three RE categories, namely into downtown, industrial/commercial, and residential areas. The classification is based on TS and CR data (namely, on the types and the along-the-route locations of TSs and CRs). The classification is carried out by an ANN devised for the purpose; this is referred to as a full ANN, as it is merged from a TS-based and a CR-based classifier with the help of a merging module.
An Android application was used during data collection trips for the manual recording of TS data and the RE types. It automatically records the car-trajectory. Eight TS types were identified as occurring frequently in urban areas and prevailing in one of the three REs. A shallow ANN was devised and trained ‒ using back-propagation ‒ to identify the actual RE from TS data [4]. The input features used by the ANN were the average distances between consecutive relevant TSs over the last 250, 500, 1000 and 2000 meters, and the number of occurrences of the typical TSs pertaining to each of the considered three REs in similar fashion. These features provide detailed information of the spatial frequencies of the TSs and CRs. The RoED step is repeated for each 50 m route-segment.
The CR data was added in a post-trip manner to the trajectory data from a public geographical information system. Five CR categories were considered. An ANN-based classifier was devised and trained also for the CR data and was re-applied as a functional module/subnetwork. The CR input features were similar to the TS ones. The TS and the CR modules were then merged with a merging module. The resulting modular full ANN appears in Figure 1 as the ANN with light grey synapses.
In some of the experiments, the TS and CR subnetworks were kept separate, while in others increasing percentages of the possible synapses between these subnetworks were allowed (Figure 1). It was hypothesised that the separable ANNs can achieve a reasonable RoED performance, and that the non-modular ones ‒ due to their more intricate data interactions ‒ can achieve even better.
Several training regimes were devised for the merged ANN using the analogy of mechanical grids. These regimes when applied to the full ANNs retain the weights and biases in certain (stiff) parts of the network, while the other (loose) parts can evolve more freely. The good starting values for the weights and biases ‒ inherited from the TS and CR classifiers ‒ shorten the necessary training effort.
After training, the ANN-based classifier processing TS data could achieve a 67.3% agreement with the ground truth, while the CR-processing classifier, on its own, achieved only a 59.7% agreement. As expected, merging these two resulted in higher values: a reasonably good agreement (71.9%) was achieved for a modular full ANN, and an even better (74.1%) for a full non-modular ANN with a random 40% selection of inter-modular synapses. In Figure 3, the RoED results can be visually compared for the modular and the non-modular full ANN.
In our view, the RoED function could be turned into a useful automotive function ‒ both in the context of smart cars and self-driving cars ‒ in the coming years, particularly, if the function is further developed and meticulously tested in various urban environments in different countries.
Link:
[L1] https://www.sztaki.hu/en/science/projects/lab-autonomous-vehicles
References:
[1] M. Woźniak, M. Graña, E. Corchado: „A survey of multiple classifier systems as hybrid systems”, Information Fusion, 16, 3-17, 2014.
[2] C. M. Rudin-Brown, J. Edquist, M. G. Lenné: „Effects of driving experience and sensation-seeking on drivers’ adaptation to road environment complexity”, Safety Science, 62, 121-129, 2014.
[3] Z. Fazekas, G. Balázs, L. Gerencsér, P. Gáspár: “Inferring the actual urban road environment from traffic sign data using a minimum description length approach”, Transportation Research Procedia, 27, 516-523, 2017.
[4] Z. Fazekas, G. Balázs, P. Gáspár: “Identifying the urban road environment type from traffic sign data using an artificial neural network”, in Proc. of the Int. Scientific Conference Modern Safety Technologies in Transportation, Kosice, 42-49, 2017.
Please contact:
Zoltán Fazekas, MTA SZTAKI, Hungary
+36 1 2796163
