by Parmenion Mokios and Michail Maniadakis (ICS-FORTH)
Synergetic performance within human-robot teams might be significantly enhanced by consideration of the temporal aspects of multi-agent interaction. For a number of years, FORTH has been equipping robots with human-like artificial time perception thus contributing a unique robotic cognitive skill that drastically improves fluency in human-robot interaction (HRI). We present an overview of the relevant technologies, which are constantly being improved and tested in naturalistic multi-agent scenarios.
Most biological organisms possess an innate sense of time. For several decades, a sense of time was missing from contemporary robotic systems, and this had clear negative impacts on their integration into human environments.
Through the EU-funded TimeStorm and EnTiment projects, FORTH has systematically investigated the strong coupling of time and mind, emphasising both the implementation of human-like artificial time perception and the implementation of time-aware robotic cognition. Along this line, we have developed technology that enables robots to effectively perceive the three main temporal views that living organisms – and particularly humans – develop of the world, namely, the past [1], the present [2] and the future [3]. Relevant experiments demonstrate that the developed technology significantly supports symbiotic HRI that assumes the long-term, timely and fluent cooperation of humans and robots.
The prioritisation of jobs and their attribution to the most appropriate agents (humans and robots) may significantly improve multi-agent HRI in naturalistic setups. To this end, HRI systems should ideally consider and exploit:
- the emotional state of humans that is known to drastically affect their perception of time and thus the temporal aspects of their personal satisfaction criteria,
- the skills that each agent brings into the team, further analysed into the time requested by a human or robot to implement a job and the quality of the underlying job implementation,
- the ordering of tasks in association to the actual progress accomplished by each agent.
Figure 1: Human-robot interaction in prioritised meal serving.
The newly introduced Daisy Planner can effectively address the above issues, effectively guiding and coordinating multi-agent activities. For example, in setups assuming several humans being served by a single robot, the planner exploits information about the emotional state of the individual humans in order to estimate the pace of their subjective time perception (i.e., fast or slow flow of time) and thus be able to effectively balance between the time pressure of humans and the expected completion time of the requested tasks. Following this approach, it is possible to prioritise human requests in a way that improves the composite level of satisfaction for the whole group of interacting humans. More specifically, the system guides the robot to prioritise service to individuals at high arousal state (i.e., those who are having trouble waiting) given that the completion time of their request is not very long, providing lower priority to people in a low arousal state (i.e., people who would tolerate a short delay).
Interestingly, in unstructured real-world environments the occurrence of unexpected events may temporally disturb multi-agent collaboration (e.g., while cooking, the baby may cry, or the phone may ring). In such cases, it is very practical to adopt a progressive, short-term planning of the composite behaviour, in line with the pragmatic unfolding of task execution. Following this approach, the Daisy Planner attributes tasks to the available agents, effectively guiding them towards the incremental accomplishment of the composite goal. The attribution of tasks to agents assumes a multi-criteria optimised local matching of agents and tasks, in order to effectively exploit the heterogeneous skills of the individual agents for the benefit of the team. A key issue in fluent human-robot interaction is the synchronicity of agents and their delegation to tasks which help other team members reduce their idle times.
Figure 2: Daisy Planner-driven multi-agent collaboration based on different optimality criteria. The employed methods (from left to right) prioritise fully unconstrained tasks, promote the attribution of tasks with the fewest constraints, consider agent-task matching i.e., time + implementation quality, consider team benefit i.e., time + implementation quality, contrasts expected and maximum task value with emphasis on time, contrasts expected and maximum task value equally balancing between time and implementation quality, contrasts expected and maximum task value with emphasis on implementation quality.
We have examined various formulations of the same Daisy Planner using alternative time-informed multi-criteria measures to coordinate the activities of heterogeneous agents. As shown in Figure 2, the relevant measures can effectively balance between minimising the implementation time of the composite behaviour, improving the quality of task implementation and minimising the idle time of participating agents.
In order to accomplish the long-term goal of human-machine confluence, it is imperative that temporal aspects of human-robot interaction are investigated. FORTH will continue to systematically explore the temporal, short- and long-term aspects of symbiotic HRI, targeting the seamless integration of robots into the heavily time structured human society.
References:
[1] M. Sigalas, M. Maniadakis, and P. Trahanias, 2017. Episodic memory formulation and its application in long-term HRI, 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017).
[2] E. Hourdakis and P. Trahanias, 2018. A robust method to predict temporal aspects of actions by observation. International Conference on Robotics Automation, (ICRA 2018).
[3] M. Maniadakis, E. Hourdakis, and P. Trahanias, 2016. Time-informed task planning in multi-agent collaboration. Cognitive Systems Research (2016).
Please contact:
Michail Maniadakis
ICS-FORTH, Greece
