Teaching a robot by demonstrating what it should do. Currently, this is quite complicated. In the future, it will bring new opportunities for automation. Engineers from RoboHouse, TNO, TU Delft and Do IoT Fieldlab are analysing different applications, to evolve the remote robot teaching standards.
Duration
September 2025 – February 2026
Partners (TU Delft)
Do IoT Fieldlab, TU Delft, RoboHouse
Partners (other)
Labor shortage, economic factors and technological developments urge scientists and businesses to rethink how employers will be working in the future. Robots will perform more and more tasks to assist humans. Think of repetitive tasks, heavy tasks, extremely precise tasks or working in a hazardous environment. Currently, robots are programmed manually to perform a task. The task description must be highly detailed and precise, to get the right result – and with every variation, new programming is needed. This approach has limitations: the more varied the environment and task, the less suitable it is for this kind of automation, because it’s difficult to cover all possible cases. New ways are explored to teach a robot to perform a task, with the ability to generalize its knowledge and self-adapt to new situations.
Simultaneous
With this new approach, a person provides demonstrations, and the robot imitates their way of working. The robot literally learns from its teacher. Through the technology of IMitation Learning and Visual Language Action models, the robot learns to understand the context of what it ‘sees’ through its cameras and sensors.
Human operators and robots interact in a natural way and the robot can learn new tasks, such as picking, packaging, cleaning, assembly, stacking objects, folding and filling a box for shipping, etc. The possibilities will be broad, but there are still many steps to take before this type of flexible automation is reached.
VR, AR and 5G technology
The experiments involve a combination of various technologies. The robot receives its motion directions from an instructor. Physical presence is not always possible for the operator, thus there is a need for remote teleoperation. For a smooth-running teleoperation, 5G is essential to process huge amounts of data with extremely low latency. Furthermore, in the future, mobile robots will perform flexible tasks; they need to be 5G and 6G ready. Currently, the experiments are carried out with VR; tech experts expect that this will be upgraded to AR in the future.
In this project, a proof of concept of robot controlled by an imitation learning policy will be developed, together with intuitive interfaces, such as hand guiding, controllers, voice) to operate the robot. The results will be available in an open, reusable testbed for SMEs, education institutes and research organisations, enabling future advancement of concrete applications.
