Retrospect: research on virtual training

February 2023 – Quickly, safely and efficiently learning something new: that is the goal of developing virtual training for the manufacturing industry. Do IoT Fieldlab facilitated a partnership to bring this to fruition. The results are promising and follow-up research is in preparation.

Research design

Several parties worked together to develop and test a new training program using Do IoT Fieldlab’s 5G facilities. The end goal is to allow trainees to practice technical skills, in a virtual environment using a high-tech glove.

Senseglove was able to test its latest VR glove, which has a very realistic feel when you touch, pick up or put something away. The Barn is a serious games developer and wanted to test a new VR training course for manufacturing industry trainees. And Craft Education, as a vocational training provider, contributed practical experience for its direct application in training. The Dutch research institute TNO focused on the technical aspects of 5G on the one hand, to align it with the various hardware and software components of the training tool. In addition, an ergonomist advised on proper posture.

Puzzle

“It was quite a challenge to bring together the different pieces of the puzzle from different modules,” says Sylvie Dijkstra-Soudarissanane, TNO project manager. “There is a lot that is not ‘5G-ready’ yet. The 3D aspect and the XR (Extended Reality) interaction were very complex. We had to do so much pioneering in different areas, simply because 5G technology is still so novel, that we had to choose what we could explore and what not to explore yet.”

The training itself is very realistic, fully matching the procedures in a factory or workshop. Sylvie: “A trainee is instructed to pick up tools, for example, and use them. The trainer supervises and gives instructions. The student sees a 3D image of the trainer in the virtual environment and experiences this as very realistic. We have found that this way of learning is effective for remembering all the actions in the training. It feels like you are actually in a room together, while in fact you are in different locations. This offers huge opportunities: you can literally exchange knowledge and learn from each other worldwide. Imagine how much travel this saves!”

Frank Krause, also from TNO, was involved in the preparations from the human factors aspect. “We had to keep the training compact so that it remained achievable for all parties involved. Together we chose to train with preparations before you start working with a milling machine. In the virtual environment you get step-by-step instructions, for example that you have to blow away grindings. Using the glove, you virtually grab the tool from the table. You walk over to a machine and start blowing it clean. Then you actually see that grind fly away. Wonderful! When you complete a task, it is clearly indicated and you move on to the next task. Virtual training offers the opportunity to safely learn about operating a machine. Furthermore, a virtual trainer can supervise multiple trainees simultaneously and costly material waste is avoided. The Barn, which developed the virtual environment, really worked very hard to get the graphics right in every single detail. It’s an enormous amount of work to make sure the 3D environment is consistently accurate.”

High quality

Bas van Nuland of game developer The Barn accepts the compliment with a smile. “We make various games and see high potential in applying VR in the manufacturing industry. We had worked with Craft Education previously, although that involved an online environment, in a browser. We did see opportunities to interconnect things now. The VR glasses already exist, the glove already exists and the trainings for the manufacturing industry are there as well. Now we are bringing it together. At the same time, this complicated it, since how do you get this to work well? In the future, once the technology gets to the point where you can stream it from a server, we will be able to offer this at high-quality online. At a school, in people’s homes, at a company… The opportunities offered by 5G are huge. But we are not there yet. We have now laid a groundwork, in the sense of some simple actions to practice virtually. A next step is to exploit the full potential of 5G. We haven’t got around to that. We also had bad luck sometimes, when, for instance, someone was ill or there was a technical bottleneck. Then everybody worked a bit fragmented on their own part. Good project management is essential in such a project. You cannot oversee everything. With all parties, we tried hard to complete the study at a high level and within the deadline.”

Valuable

Educator Roy van Bussel of Craft Education joined the experiment because he deems the added value of virtual practice. “We conduct many machine training courses and already have experience with e-learning. Practising with a glove and 5G adds a new dimension. If a company orders a milling machine, the staff can virtually practice all operations in advance. This way, employees are already prepared. Once the machine is in place, production can be run. We teach someone to think according to a certain structure. So: which steps to take to use the machine in the appropriate way. I don’t expect VR training to completely replace on-the-job training in the future, but the training time will be significantly reduced. We see VR technology becoming increasingly cheaper. While VR glasses cost 10,000 euros a few years back, now you can buy a good pair for 300 euros. Developing software to create beautiful images is also becoming more affordable. At present, the technology is not so mature that an entire setup operates with 5G, but I absolutely believe in future applicability. For us, getting in on this development early is very valuable.”

Evaluation

The testing of the virtual training with test subjects was designed and evaluated according to scientific method. Fourteen of the sixteen students involved in the test consider the VR training a pleasant way of learning, in preparation to the real factory work. Now that the research has been completed, a VR demo setup has been installed at the TNO guided manufacturing lab at RoboHouse so that companies and governments can see – and better still, experience for themselves – that this functions well.

What will be the next step? Everyone agrees: this is yearning for more. A new grant application is in the pipeline: the parties cannot take such an intensive, innovative development ‘on the side’. The relations are good and the goal is clear: in a few years’ time, employees in the manufacturing industry will train online, from different locations, with professional guidance from a virtual trainer. By then, the chat at the coffee machine will probably no longer be “I have to take another training course”, but “Yes, I get to train online again! What about you?”.

More information

Want to know more?

Read our project page.
The research partners involved were: TU Delft, RoboHouse, TNO, SenseGlove, The Barn, Craft Education, MCS
This research was co-funded by the Dutch Province of Zuid-Holland and SMITZH, and supported by InnovationQuarter.