by Sara Colantonio, Daniela Giorgi and Ovidio Salvetti
The European project SEMEOTICONS offers a fresh perspective on educational programs and lifestyle intervention for the prevention of cardiovascular diseases. SEMEOTICONS will develop a multisensory platform, in the form of a mirror, to be integrated into daily-life settings. The mirror will be able to detect and monitor over time facial signs correlated with cardio-metabolic risk, and give personalized advice to end-users on how to improve their habits.
The face is the pre-eminent channel of communication among humans: it is a mirror of status, emotions, mood. This is the base principle of Medical Semeiotics, which looks at the face as a revealer of the healthy status of an individual, through a combination of physical signs (e.g., skin colour, subcutaneous fat) and facial expressions. Medical Semeiotics dates back as far as Aristotle’s times. Nevertheless, it is still used by medical practitioners today, who rely on their ability to read the face of patients to steer the medical examination and decide on diagnostic investigations.
SEMEOTICONS, funded by the EC (FP7), began in November 2013 with the objective of moving Medical Semeiotics to the digital realm. SEMEOTICONS proposes the translation of the semeiotic face code into computational descriptors and measures, which can be automatically extracted from videos, images, and 3D scans of the human face. In particular, SEMEOTICONS addresses signs related to cardio-metabolic risk as cardiovascular diseases are the leading cause of mortality worldwide.
SEMEOTICONS also aims to bring semeiotic analysis closer to everyday life, ie, from the surgery to the home, the gym, the pharmacy, etc. This will enable people to self-assess and self-monitor the status of their well-being over time. A multisensory platform in the form of an interactive smart mirror, called the Wize Mirror, will be developed. The Wize Mirror is designed to fit into daily-life settings, by maximizing non-invasiveness and natural interaction modalities, according to the Ambient Intelligence paradigm. It will seamlessly integrate contactless sensors (3D optical sensors, multispectral cameras, gas detection sensors, microphones) with a user-friendly touch-screen interface (Figure 1).
Figure 1: The Wize Mirror is a multisensory platform which collects videos, images, 3D scans of the human face and gas concentration signals, looking for signs correlated with cardio-metabolic risk.
The heterogeneous data collected by the Wize Mirror ̶ videos, images, 3D scans, and gas concentration of individuals standing in front of the mirror ̶ will be processed by dedicated algorithms, which will extract a number of biometric, morphometric, colorimetric, and compositional descriptors [1]. The descriptors will be integrated to define a virtual individual model and an individual wellness index to be traced over time. The Wize Mirror will also offer suggestions and coaching messages, with a personalized user guidance, aimed at the achievement and the maintenance of a correct life-style.
The empowerment of individuals, in terms of their ability to self-monitor their status and improve their life-style, is expected to have a great impact on the reduction of health expenditure and disease burden. It is well-known that the cost of health systems grows exponentially with the aging of the population, together with the widespread use of complex, and sometimes inappropriate, diagnostic procedures. Prevention is the best strategy to limit the spread of cardio-metabolic diseases. SEMEOTICONS offers a fresh perspective on educational programs and lifestyle intervention.
SEMEOTICONS will address significant scientific and technological challenges. It promises touch-less data acquisition via a low-cost platform; temporal and spatial data synchronization; real-time processing of multimodal data; and intelligent solutions to map iconic facial signs to repeatable, reliable computational measures correlated with risk factors. The project consortium has been built to strike a good balance between innovation and technology-driven research. It comprises six research organizations (with competence in both ICT and medical areas), two industrial partners, and two SMEs, from seven European countries (France, Greece, Italy, Norway, Spain, Sweden, United Kingdom). The project coordinator is the Istituto di Scienza e Tecnologie dell’Informazione (ISTI) of the National Research Council of Italy (CNR), located in Pisa.
Link: http://www.semeoticons.eu/
Reference:
[1] E. Vezzetti and F. Marcolin: “3D human face description: landmark measures and geometrical features”, Image and Vision Computing 30 (2012), 698 - 712
Please contact:
Sara Colantonio - ISTI-CNR, Pisa, Italy
Tel.: +39 050 621 3141
E-mail:
