New technology lets humans & autonomous robots understand each other
01 October 2020
RESEARCHERS FROM the ORCA (Offshore Robotics for the Certification of Assets) Hub, a consortium of five universities led by Heriot-Watt and Edinburgh universities, have unveiled a new method of communication that allows machines and humans to speak the same language and understand each other’s actions in real time.
Named MIRIAM (Multimodal Intelligent inteRactIon for Autonomous systeMs), the new method of communication is a significant breakthrough that enables people to work with autonomous robots. It allows users to ask robots questions, understand their actions and receive mission status through a digital twin1. MIRIAM uses natural language, allowing users to speak or text queries and receive clear explanations about the robot’s behaviour in an intuitive way.
By improving how robots communicate, MIRIAM will help to build the confidence of those who will be using them and overcome any adoption barriers in the workplace.
MIRIAM will be used by a new collaborative team that sees Heriot-Watt University, through the ORCA Hub, working and integrating its research with the expertise of Phusion, an engineering software firm and Merkle Aquila, a data science firm. They are the latest additions to global energy partner Total’s multimillion-pound research consortium that is working to develop autonomous capabilities with multiple robotic systems for work on offshore energy installations. MIRIAM will initially work with a tracked robot that will be deployed at Total’s Shetland Gas Plant.
Helen Hastie, professor of computer science at Heriot-Watt University and an ORCA work package lead, explains: “When robots have an autonomous element, it means that they sense the environment and can make certain decisions by themselves. However, there is currently a communication barrier between robots and their human supervisors in relation to the reasoning behind system actions. This is particularly problematic in remote, highly challenging, and hazardous environments such as offshore, which can involve multiple vehicles and platforms.
“This lack of transparency can lead to a reduction in situation awareness and therefore user-confidence in the robots, hindering the human-robotic team and resulting in unnecessary aborts or laborious manual manipulation of the robot. It is therefore essential that a supervisor understands what the robot is doing and why.
“MIRIAM’s technology is designed to provide invaluable transparency in this process. Our team of researchers have taken a user-centred design approach, which supports Total’s forward-thinking efforts to making operations safer offshore. We have also modified MIRIAM to the specifics of Total’s robotics programme.”
Phusion IM is supporting the project with the implementation of a Master Data Model that will enable the framework for a digital twin of Total’s facilities. This will allow the creation of autonomous routines, enabling Total’s robot to undertake maintenance checks and feedback results in a consistent manner to the digital twin – creating a single source of information for the operating asset.
MIRIAM will allow humans to communicate with this digital twin, and the data held in it, including the robot, via text or speech. This means supervisors can ask where the robot is going, what it plans to do next and why, and query data it has gathered during its missions.