by Wolfgang Herzner, Rupert Schlick, Manfred Gruber
The goal of MOGENTES (MOdel-based GENeration of Tests for Embedded Systems) is to significantly enhance testing and verification of dependable embedded systems. This is achieved by automatically generating efficient test cases by relying on the development of new approaches as well as innovative integration of state-of-the-art techniques. In particular, MOGENTES will apply this technology in large industrial systems in the automotive, railway control and off-road vehicle industries.
Embedded computer systems are increasingly being integrated with safety-relevant applications such as vehicles, medical equipment and control systems. Every possible measure must be taken to ensure the dependability of such systems. As a consequence, the cost of testing software, verifying its correctness, or validating it against functional and safety requirements accounts for an increasing fraction of the overall cost. A small survey that we recently carried out among some 35 people revealed that more than 40% of the participants put at least 20% and up to 50% of the overall development effort into test and verification.
One conceptually perfect means of proving the correctness of some (software) system is formal verification, eg model checking. However in general, formal verification approaches have several limitations and drawbacks:
- for larger systems, computing resources quickly go beyond what is feasibly available
- the notations used for the formal specification of systems and requirements are highly abstract (eg Z or VDM or NuSMV), making it hard for domain experts to efficiently apply these methods
- a 'sufficiently' complete formal specification of the system is hard to establish (it is noteworthy that the standards EN 50128 and IEC 61508 recommend the use of formal methods at higher safety integrity levels, but do not enforce it as the only highly recommended method)
- the notion of faults and fault effects are rarely included in formal models
- means for dependability (eg fault tolerance) are poorly addressed by existing tools.
Testing is therefore still the preferred method of verification. Manual testing, though, is expensive. In the survey mentioned before, in 60% of all addressed projects the number of test cases lies in the range of 100010 000, representing a significant effort. There is consequently a huge demand for test-case generation (TCG) and tools. However, in 60% of all addressed projects essentially none of the test cases were automatically generated, while in fewer than 5% of projects were more than 60% of test cases automatically generated.
One reason is that TCG requires abstract models of the target systems. Unfortunately, testers are often forced to manually reverse-engineer the implementation in order to achieve the coverage required for a successful certification (eg Modified Condition/Decision Coverage (MC/DC) as suggested by the RTCA DO-178B standard), because an abstract model of the system is not available or no longer conforms to the final product. As a consequence, model-based development must be complemented by corresponding improvements in model-based testing technology.
MOGENTES (MOdel-based GENeration of Tests for Embedded Systems) will demonstrate that with a combination of new and existing techniques, not only can the testing effort be significantly reduced by means of model-based test case generation, but that this can also be realized in a way accepted by domain experts with limited experience in formal methods.
A means to achieving this goal is the reduction of test cases by selecting the most effective ones. MOGENTES therefore has the following objectives:
to generate efficient test cases from system and fault models
to establish a framework for the integration of involved tools, including model transformations
to provide traceability of requirements and to match them to test analysis results
to foster the application of automated testing for satisfying functional safety standards requirements.
These objectives shall be achieved with the following concepts:
- define common modelling languages and semantics (meta-models), with UML as the primary candidate, to model domain-specific requirements
- develop a test theory that defines the conformance relation between the model and the implementation, and success and failure of a test case
- define fault models (for hardware and software) and extend the modelling languages to represent faults in (application) models
- define new coverage criteria under consideration of minimal cut sets, fault injection, mutation testing and safety aspects
- use model-based fault injection (MBFI) for automatically calculating minimal cut sets
- validate fault models and the generated test cases with physical fault injection
- use (bounded) model-checking techniques to generate stress test scenarios
- provide semantics-aware transformations from system models to inputs of specific tools.
The partners in MOGENTES are Austrian Research Centers - ARC (AT), Budapest University of Technology and Economics (HU), Swiss Federal Institute of Technology Zurich (CH), Graz University of Technology (AT), Prover Technology (SE), SP Technical Research Institute of Sweden (SE), and four industrial partners: Ford Forschungszentrum Aachen (DE), Prolan Irányítástechnikai (HU), Thales Rail Signalling Solutions (AT) and Re:Lab (IT), who not only provide the applications and requirements to be addressed, but will also evaluate the results and develop the final demonstrators.
MOGENTES is a Specific Targeted Research Project (STREP) in the 7th Framework Programme partially funded by the EC, and commenced in 2008.
Link:
https://www.mogentes.eu/
Please contact:
Wolfgang Herzner, Rupert Schlick, Manfred Gruber, Austrian Research Centers GmbH ARC
Tel: +43 50 550 {4231/4124/4183}
E-mail: {wolfgang.herzner, rupert.schlick, manfred.gruber}
arcs.ac.at
