Digital twin linefollower

The University of Antwerp Ansymo group develops digital twin architectures and studies product families of digital twins. To be able to do virtual experiments with a simple line-following robot (Lego Mindstorms), they wanted to have a detailed digital twin in addition to their physical set-up.

This twin is developed by Confirmat. This concerns a digital model of the physical line-following robot plus a model of the measurement setup with a depth camera used to locate the robot. For digital twins, Confirmat uses Unity3D.

A depth camera model, as well as a red-light sensor (measuring the location of the line to follow), have been developed in Unity3D (“virtual sensors development”). The camera images and sensor values ​​are published in Unity3D via DDS, the middleware also used in ROS2. This means the sensor values are available in a remote ROS2 application. The line-following controller used in the physical setup is also implemented in ROS2, and this controller is used to control the robot in Unity3D. In ROS2, an image is generated from the depth camera stream using OpenCV. In this way a link is made between ROS2 and the virtual environment, with the dedicated virtual sensors, in Unity3D.