by Mario Drobics, Matthias Zima, Dalibor Hrg, Jan Bobeth and Steffen Budweg
Within the AAL-Joint-Programme project FoSIBLE, a Social TV community platform for elderly people, augmented by game technologies and smart furniture, is currently being developed. The platform aims to provide social support for interaction between peers, friends and families. We present first results from our approach of integrating different multi-modal interaction techniques for an integrated Smart TV with Social Community solution.
Within the FoSIBLE project, a Social TV community platform for elderly people is being developed. To ensure high usability and acceptance, a novel combination of input methods is used. The central element of the application is a Smart-TV system, which is used to display messages, images and videos. A dedicated application runs on the Smart TV and provides chat functionality, as well as games and virtual libraries. To encourage the elderly to deal with this application, different input methods are supported. In addition to traditional manufacturer-specific remote controls or applications, gestures can be used for navigating through the menus, while a tablet-PC provides easy text input as well as the ability to navigate the system. The system is completed by sensor information and smart furniture to detect the presence and state of the user.
Multi-modal input approaches for controlling Smart-TV systems seem promising from a usability perspective, as active user input can be reduced to essential tasks. Based on context analysis and evaluation of measurements provided by visual, pressure and movement sensors, appropriate assistance can be provided, for instance by pre-selecting relevant menu options, automating sign-in processes and suggesting relevant information. In conjunction with a natural user interface, based on touch interactions on a tablet-PC as well as gesture recognition, applications can be designed to support heterogenic user groups, even without previous technology experience. To reach this goal, guidelines for user interface design need to be taken into account, supported by extensive end-user involvement through the whole development process.
Smart TV is a recent approach to integration of web-based interactive content into modern television sets. Our application is designed as a widget that runs on a standard Smart TV platform and which is connected to our social community platform (Figure 1). It implements social features and functionalities identified during an end-user requirements process, including support for communication and awareness while watching TV, as well as a social media platform for exchanging information (eg on interesting TV programs or books) and playing games. A presence awareness implementation allows for further enrichments by making use of various sensors and smart furniture available in the future living room.
Figure 1: A prototype of the Smart TV widget for the FoSIBLE Social TV Community.
Gesture recognition promises to provide a simple to use, intuitive interface for interacting with Smart TV platforms. The appearance of Microsoft’s Kinect device lead to a boom in the development and demonstration of gesture control interfaces. However, devices like Microsoft’s Kinect regularly target only three-dimensional scene recognition and additional hardware and software are often necessary to perform the actual gesture recognition in real time. This results in increased system cost and complexity when deployed, because the device cannot be connected to a TV-set or set-top box directly. In FoSIBLE, we therefore targeted the development of a fully embedded system for hand gesture recognition based on a biology inspired “Silicon Retina” optical stereo sensor, which is ideally suited for the recognition of dynamic gestures. The sensor is only sensitive to light intensity changes and therefore tracks hand motion in its field of view robustly and with high accuracy. It can detect whether the user is in front of the TV set, and whether the user is in company or alone. The hand gesture recognition software (see Figure 2) for the embedded device is developed within the FoSIBLE project and will be evaluated against comparable gesture recognition software available for the Kinect device. The underlying gestures have been defined as a result of an end-user evaluation process, to ensure that the target group can cope with the defined gestures.
Figure 2: A sequence of still images provide an example of raw hand tracking data (left pane) and synchronous video (right pane) for a person performing a “circle” gesture.
Although gestures can be used for most navigation and selection tasks, not all inputs can be done using gestures alone. Furthermore, some users might not want to use gestures at all. Thus, additional input modes should be provided to users, which they can use to navigate the Smart TV system, but also to enter text (eg during chats or when writing a short article). As traditional input devices like keyboards and mice are quite cumbersome to use in a living-room environment, a tablet PC is used for this task. The tablet can also be used to display messages when the TV is switched off and is connected to the Smart TV system and the Social Community platform.
Additionally, smart furniture is used to detect the presence and state of the user and to adopt the system to the users need. For this purpose, touch sensitive tables are used, as well as sensor equipped chairs and beds. By detecting the position of the user, context sensitive functionality can be provided, such as to start an application when the person sits in front of the TV and interacts with the couch-table. Furthermore, it is possible to adjust the TV screen and the camera, so they are directed towards the user.
The integrated FoSIBLE Social TV community system will be deployed in multiple home environments in France and Germany throughout the next year, delivering more results from the real-life usage and more insights about the end-user acceptance and usability for novel multi-modal interaction techniques for Smart and Social TV systems.
Acknowledgments
The FoSIBLE project is funded by the European Ambient Assisted Living (AAL) Joint Program together with BMBF, ANR and FFG.
Link: http://FoSIBLE.eu
Please contact:
Mario Drobics
AIT Austrian Institute of Technology GmbH / AARIT, Austria
E-mail:
