by Doreid Ammar, Katrien De Moor and Poul Heegaard, NTNU
Web real-time communication has enabled hassle-free, no installation, in-browser applications such as Google hangout and appear.in. Multi-party video conferencing has now finally been made easy. But how can we provide acceptable quality of experience in such an interactive service? Our research aims to gain insight into what matters, and how to assess, design, and manage the services accordingly.
Web real-time communication (WebRTC) has become popular in recent years, with numerous free of charge applications, such as appear.in and Google hangouts, used in both private and professional contexts. Such applications enable real-time (audiovisual) communication in the browser with multiple parties, without need for plug-ins or other requirements. As a result, such applications are very easy to use and for some applications (e.g. appear.in [L1]), it is not even necessary to create a user account. Users can simply connect and access WebRTC-based applications from a wide range of devices (e.g. smartphone, tablet, laptop), using a web-browser that supports WebRTC (e.g. Chrome, Firefox) or a dedicated App.
However, WebRTC-based multi-party video communication is highly interactive and Web-RTC based applications can be used from multiple devices and in very different contexts. Moreover, the implications for users and their quality of experience (QoE), i.e., their degree of delight or annoyance when using the application [1] are not fully understood. Users may for example experience different types and gradations of quality impairments (e.g. video freezes, bad or no audio) during a conversation, which may make smooth communication nearly impossible. It is vitally important that application and service providers address such issues, in order to prevent users from getting so annoyed that they stop using the application and switch to a competitor. Preventing user frustration and providing the best possible experience requires thorough insights into the various technical and non-technical factors that may influence users’ QoE.
In the scope of the Telenor-NTNU research collaboration project “Quality of Experience and Robustness in Telecommunications Networks”, which started in 2015, we aim to:
- identify the most relevant influence factors, with a primary focus on technical (quality of service-related) factors, but also including non-technical (contextual, human level) influence factors;
- investigate in which ways and to what extent factors influence users’ QoE and corresponding user behaviour; and
- from an understanding of these relationships and thresholds, provide input to the development of QoE-aware adaptation strategies to reduce/avoid user annoyance, and foster user delight.
The WebRTC based application called appear.in, which enables video communication for up to eight parties and which can be accessed using browsers that support WebRTC, is used as a concrete use case.
A multi-method approach is used to gain more insight in the relevant influence factors and investigate the relationship between performance-related parameters and users’ QoE. We have conducted a literature study, analyzed historical data, and used exploratory focus groups with appear.in users. In our ongoing work, we perform real-time logging and analysis of performance and session-related statistics using a research version of appear.in. For this purpose, we use both Google Chrome’s WebRTC internal functionality and the WebRTC Analytics interface getstats.io. Different types of data are being collected and analyzed, including subjective, explicit user feedback (collected at the end of a session), implicit and behavioural user feedback (collected during a session), and objective performance-related statistics. So far we have run a series of tests with two parties according to different test scenarios and network conditions (see Figure 1). We collected real-time session statistics by means of Google Chrome’s WebRTC-internals tool. Although the Chrome statistics have a number of limitations, we have found that they are useful for QoE research. The results highlight the relevance of some of the investigated performance statistics for detecting potential QoE issues.
Figure 1: A two-party video communication using appear.in.
We are now planning to conduct both controlled laboratory and longitudinal “living lab” empirical studies involving real users. For this we are developing an experimental test platform, will log performance stats in real-time, collect explicit and implicit user feedback in test users’ natural environments, and perform in-depth data analytics and data visualization. Of particular interest is how users´ tolerance levels towards different types of QoE impairments and users’ expectations evolve over time and in a real-life context, as a complement to the ‘instantaneous’ QoE assessment in the lab setting.
We believe that our research activities will help to formulate concrete recommendations on how to meet and exceed users’ QoE requirements and to generate research-based knowledge on how to foster user delight and avoid annoyance in the context of WebRTC-based video communication.
The Telenor-NTNU research collaboration project “Quality of Experience and Robustness in Telecommunications Networks” involves a multidisciplinary team, including current ERCIM fellow Doreid Ammar and former ERCIM fellow Katrien De Moor, and is a collaboration between the following European research institutions: The Department of Telematics at Norwegian University of Science and Technology – NTNU (Trondheim, Norway), Telenor research (Trondheim, Norway), represented by Dr. Min Xie and the Department of Communication Systems at Blekinge Institute of Technology – BTH (Karlskrona, Sweden), represented by Prof. dr. Markus Fiedler.
Link:
[L1] Appear.in: https://appear.in
Reference:
[1] A. Raake, S. Egger: “Quality and Quality of Experience,” in Quality of Experience: Advanced Concepts, Applications and Methods, in: S. Möller and A. Raake (Eds.), Springer, 2014.
Please contact:
Doreid Ammar
ERCIM fellow at NTNU, Norway
Tel: +47 73 59 43 21
