by Mohamed Behery, Philipp Brauner, Martina Ziefle and Gerhard Lakemeyer (RWTH Aachen University)
Shorter product lifecycles, more product variants, individualised production, and the desire for sustainable production call for agile control frameworks that enable smarter robotic control and collaborating human-robot teams. We propose generalising and standardising “Behaviour Trees” that use human action nodes as a process model and task-execution-monitoring approach for human-robot collaborative assembly processes to increase the agility of human-robot teams while ensuring a safe and trusted human-robot interaction. Within the DFG (Deutsche Forschungsgemeinschaft) funded Cluster of Excellence “Internet of Production”, we take a cross-disciplinary approach to conceptualisation and validation to ensure algorithmic soundness, technical viability, and social acceptance by the workers of increasingly agile human-robot teams.
Production processes that involve Human-Robot Collaboration (HRC) are required to meet the demands for more product variant requests, initialised production, and short product lifecycles, which calls for agile control frameworks that ensure a safe, trusted, and socially accepted integration of humans in the robots’ workspace.
Moreover, the exchange and replication of production setups and pipelines is desirable in the World-Wide Lab (WWL) [1] and requires an abstract modular transferable process representation. The Cluster of Excellence Internet of Production (IoP) [L1] tackles these problems, among others, using process Digital Shadows (DSs) [2]. DSs are context- and task-specific process models that can be stored in online servers to be used in many ways including process control, verification, and decision support systems. Additionally, they can be used to efficiently couple the different entities of the WWL.
HRC processes have so far not been integrated into the WWL. The integration is challenging due to the lack of DSs for these processes. This can be attributed to the ad-hoc robot-specific programming involved in these use cases, which can be laborious and time consuming. They are inherently challenging because they cover a wide range of setups, all of which have a proximity to human workers.
HRC processes need DS to enable real-time introspection, code reuse, and integration into the WWL. A DS of an HRC process must conserve the safety requirements (and guarantees) of the process. For example, an HRC assembly must represent the requirement of a fully assembled product while maintaining the safety of the human co-worker.
We propose generalising Behavior Trees (BTs) [3] as a modelling framework for industrial HRC applications and using BTs as a DS in the WWL. Recent work showed several extensions and applications of BTs to support HRC processes, heterogenous multi-agent systems, verification, as well as learning and synthesis of trees [3].
Figure 1: A sample behaviour tree for an assembly task. The leftmost branch of the tree (blue) ensures that the robot is not in a collision state and that the battery is not empty. The two remaining branches (green) pick a screw and place it in a clear hole. The latter branches contain guards that ensure that the preconditions of pick and place actions hold.
BTs are tick-based action representation- and execution-monitoring models. They represent tasks in a tree structure whose execution starts by ticking the root; internal control flow nodes propagate the tick to the leaves, which are responsible for the robot-specific execution. This structure splits processes into standalone modules with different responsibilities (e.g., a branch of the tree can focus on safety while others focus on navigation and manipulation). This can be seen in Figure 1, where the leftmost branch prevents the robot from moving in cases of collision and low battery and the core pick and place branches contain guards that ensure the respective action preconditions hold before execution. They represent the general execution flow and contain placeholders for robot-specific behaviour. They also represent priority of tasks (or subtrees) over others thanks to the tree’s ordering; they are executed from left to right and top to bottom. Their reactivity, modularity, and flexibility give them an edge as a task-level controller in many domains [3].
Given a BT representation, a process can be modified by replacing one subtree with another. This replacement can take place between nodes of different types (or a leaf and a branch) as long as the tree remains valid. Additionally, we can have a tree that ensures safety and use it as a subtree (with high priority) to guarantee the safety of the rest of the tasks in the new tree. This ensures that BT representations are flexible, modular, and reusable.
Our previous work proposed using a human-action node with a rule-based system allowing robots to switch tasks and react to the outcomes of human sub-tasks instead of waiting for them. Using BTs improves the “explainability” of the robots' actions and thus increases trust between the human workers and the robots. One of the demonstrators in the IoP shows how BTs can represent and execute processes that involve keeping safe distances between humans and robots during the execution. Additionally, ongoing work in our cluster explores the safety requirements involved in collaborative processes between humans and robots and how we can use different properties of BTs to satisfy them.
BT models of HRC processes are easy to share, find, and synthesise in the WWL. After the integration of BTs as the standard DS representation in the WWL, we plan to further extend BTs for increased robustness and reactivity through mixed initiative planning and supporting further use cases such as robot teleoperation.
This work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy – EXC-2023 Internet of Production – 390621612. It is partially supported by the EU ICT-48 2020 project TAILOR (No. 952215).
Links:
[L1] https://www.iop.rwth-aachen.de
References:
[1] P. Brauner, et al., “A computer science perspective on digital transformation in production”, in ACM Trans. Internet Things 3(2) (2 2022). https://doi.org/10.1145/3502265
[2] M. Liebenberg, M. Jarke, “Information systems engineering with digital shadows: concept and case studies” in Int. Conf. on Advanced Information Systems Engineering, pp. 70–84, Springer (2020).
[3] M. Iovino, et al., “A survey of behavior trees in robotics and ai”, Robotics and Autonomous Systems 154 (2022): 104096.
Please contact:
Mohamed Behery, Knowledge-Based Systems Group
RWTH Aachen University, Germany
