by Benjamin Billet and Valérie Issarny
The Dioptase middleware provides developers with new methods for writing distributed applications for the Internet of Things (IoT). Dioptase leverages data streams as data model and continuous in-network processing as computation model, in order to deal with the challenging volume of data being continuously produced by the ‘Things’.
The Internet of Things (IoT) is currently characterized by an ever-growing number of networked ‘Things’, i.e., devices that have their own identity together with advanced computation and networking capabilities: smartphones, smart watches, smart home appliances, etc. These Things are being equipped with increasing numbers of sensors and actuators that enable them to sense and act on their environment, linking the physical and virtual worlds. Specifically, the IoT raises many challenges related to its very large scale and high dynamicity, as well as the great heterogeneity of the data and systems involved (e.g., powerful versus resource-constrained devices, mobile versus fixed devices, continuously-powered versus battery-powered devices, etc.). These challenges require new systems and techniques for developing applications that are able to: (i) collect and process data from the numerous data sources of the IoT and (ii) interact both with the environment using the actuators, and with the users using dedicated GUIs.
Solutions for the IoT currently rely heavily on third-party services and infrastructures, such as the cloud of Things where each device sends its measurements to centralized collection and computation points. Several problems emerge from these approaches, related to energy consumption (due to intense wireless communications) and device lifetime, network overload and privacy preservation. The challenge is then to allow Things to be much more autonomous and use third-party services only when required, such as in wireless sensor networks, but at a much larger scale.
To this end, the Dioptase [1] solution aims at providing a common middleware layer that runs directly on the Things, enabling them to manage the huge volume of data continuously being produced (measurements, events, logs, etc.) in a collaborative way. In this context, we consider data streams and continuous processing as the reference data and computation models for developing IoT applications. Continuous processing is indeed a very suitable paradigm for processing each piece of data one by one, without having to store the entire dataset.
Dioptase leverages a service-oriented architecture (SOA) revisited for supporting continuous processing. This SOA introduces the concept of stream services that produce and consume infinite data streams, in contrast to regular services that manage finite datasets. Stream services are classified into four families, depending on their roles:
- producers, which generate new data streams (e.g., from sensors);
- processors, which produce new data streams by continuously processing existing ones;
- storages, which store persistently or temporarily the data extracted from data streams and can serve them as new data streams when required;
- consumers, which acquire data streams for driving actuators or updating GUIs.
Once Dioptase is deployed onto a Thing, it enables developers to manage the Thing as an abstracted and homogeneous pool of resources that can execute stream services provided over time. These stream services can be developed by using the native languages of the device or a new lightweight stream processing language, called DiSPL, directly interpretable by Dioptase.
By composing these stream services, developers build their IoT applications as a set of tasks that are executed continuously by the hosting Things in a purely distributed manner. Developers are provided with dedicated tools for designing tasks graphically before injecting them into the network of Things at any time, using deployment Web services or through the Dioptase deployment server (DiDS). DiDS automatically computes where to deploy the stream services and then manages their execution over time. DiDS relies on a state-of-the-art task allocation algorithm [2], which guarantees the lifetime of each task, by minimizing the overall energy consumption or maximizing the fair allocation of tasks among the Things, according to the constraints expressed by the developers and the characteristics of the available Things.
Dioptase features a customizable middleware architecture that is versatile enough to be deployed on a large class of Things that vary significantly in terms of resource availability (e.g., embedded systems, smartphones or plug computers), provided that these Things are able to communicate directly through the Internet infrastructure. As illustrated in Figure 1, Dioptase embeds various modules for interacting with existing components being used as part of the IoT infrastructure: discovery systems and registries, computation and offloading infrastructure, legacy sensor and actuator networks, etc.
Figure 1: Dioptase, a data streaming middleware for the IoT.
This work was part of the larger CHOReOS project effort, which revisits the concept of choreography-centric service-oriented systems to introduce a dynamic development process and associated methods, tools, and middleware for the services in the Ultra Large Scale (ULS) Future Internet. Beyond CHOReOS, Dioptase represents a significant step towards an IoT where every Thing can be assigned tasks and complete them autonomously. As an ongoing area of research, Dioptase will be released soon as an open source project. For the near future, we plan to enable Things to automatically delegate, adapt and split their own tasks according to their environment, their load, their available resources and their capabilities in a decentralized collaborative manner, taking the role of DiDS. As a benefit, the network of Things would adapt itself to minor and major changes over time.
Link:
https://mimove.inria.fr/dioptase
References:
[1] B. Billet, V. Issarny: “Dioptase: A distributed data streaming middleware for the future Web of Things”, Journal of Internet Services and Applications, vol. 5, no. 13, 2014.
[2] B. Billet, V. Issarny: “From task graphs to concrete actions: A new task mapping algorithm for the future Internet of Things”, in proc. of the 11th IEEE International Conference on Mobile Ad hoc and Sensor Systems (MASS), 2014.
Please contact:
Benjamin Billet, Valérie Issarny
Inria, France
E-mail:
