Reliable and Inexpensive QoS Monitoring in Service Markets

by Radu Jurca, Walter Binder, Boi Faltings

We propose a novel infrastructure for Quality of Service (QoS) monitoring that promises a significant reduction in monitoring costs. Our approach uses an incentive-compatible reputation mechanism to accurately estimate QoS by aggregating quality ratings from the clients.

Managing change is becoming increasingly important in a dynamic and quickly evolving economy. Consequently, organizations need flexible IT systems that can easily support new business processes and strategic objectives. Service-oriented computing enables the construction of such applications by orchestrating a variety of services that offer basic functionality. Recent efforts have materialized into standards and tools that significantly facilitate the construction and interoperability of services.

Since services are often developed and run by different organizations, they are generally provided under a contract (or Service Level Agreement) that fixes both the type and quality of the service to be provided, and the penalties to be imposed if these are not met. For example, a payment service that is part of an online shopping site may incur certain penalties if it fails frequently. In another example, the provider of a communication service may be penalized if they fail to meet their guaranteed bandwidth. Yet another example is the provisioning of computation or data management services through a network: if a certain efficiency and capacity is guaranteed in an agreement, penalties may be incurred if these services are not provided.

An essential requirement for such service provisioning is the ability to monitor the QoS that is actually delivered. As the monetary value of individual services decreases, the cost of providing accurate monitoring takes up an increasing share of the cost of providing the service itself. For example, with current technology, to reliably monitor the quality of a communication service requires constant communication with a neutral third party and would be almost as costly as providing the service itself. This problem remains a major obstacle to the wider adoption of a service-oriented economy.

As a joint project between the Artificial Intelligence Lab of Ecole Polytechnique Fédérale de Lausanne (EPFL) and the University of Lugano, we are developing alternative mechanisms for monitoring QoS. The basic idea in our approach is to estimate the quality delivered by a service, based on the feedback provided by clients. Clients run the monitoring code, and periodically report feedback to a reputation mechanism (RM). The RM aggregates the reports and outputs QoS estimates for each service. The advantages of this approach are that (i) the RM acquires information on most transactions without actually being a bottleneck (there are no real-time constraints for reporting feedback, and the result of several interactions may be compressed into one feedback message), (ii) the monitoring process is as precise as possible (an immediate consequence of the first point), and (iii) service providers cannot directly tamper with the monitoring process.

A practical mechanism must, however, ensure that clients report feedback honestly. Consider for example a client who knows that by reporting negatively, the resulting QoS estimate might entitle him to collect penalties from the service provider. The client can tamper with the monitoring code to make the provider pay penalties and thereby obtain a share of these. The novelty of our approach is to use economic incentives rather than hard security to obtain reliable information. Clients get paid for reporting feedback, and the amount depends on the values of other reports submitted by other clients. These feedback payments can be designed such that it is provably best for every client to report the truth. In game-theoretic terms, honest reporting becomes a Nash equilibrium, where any individual lie decreases the expected payment of the reporter. Misreporting is thus uninteresting rather than impossible.

Initial experiments have provided very encouraging results. We have implemented a QoS monitoring framework based entirely on existing standards and technology. The extensions required for the collection and payment of feedback are lightweight, and do not introduce significant overhead. The payments for feedback can be made as low as one percent of the cost of the service, and the overall cost of monitoring can be decreased by an order of magnitude as compared to traditional techniques.

Traditional approaches to QoS monitoring involve: (i) centralized monitors that proxy every interaction, (ii) centralized monitors that sample the interactions or (iii) decentralized monitors. In our approach, most reports come from clients. A small percentage of interactions may also be sampled by a specialized monitor. Clients are paid in such a way that it is in their best interest to report honestly.

In our ongoing research, we are exploring an architecture that integrates reputation mechanisms with service directories and a publish/subscribe infrastructure. Clients subscribe for types of services in which they are interested and are notified upon significant QoS changes of certain services or upon the availability of matching services. Clients use this infrastructure to continuously evolve and improve their service-oriented applications. We will continue to use game-theoretic tools to ensure that the information distributed by the reputation mechanism is reliable, and to ensure that self-interested parties have a rational incentive to adopt the behaviour recommended by our mechanisms.

Links:
http://dx.doi.org/10.1007/11596141_30
http://www2007.org/paper420.php

Please contact:
Radu Jurca,
EPFL / SARIT, Switzerland
Tel: +41 21 693 6679
E-mail: radu.jurcaepfl.ch