A research consortium around the Cologne Game Lab at TH Köln is developing a learning environment with artificial intelligence to support the independent learning of psychomotor skills.
Anyone who wants to learn a sport trains new psychomotor skills. Until now, this has required role models such as teachers on site who explain, demonstrate and assess certain procedures. As part of the joint research project MILKI-PSY, a consortium led by the Cologne Game Lab at TH Köln is developing a learning environment with artificial intelligence (AI) to support the training process.
Artistic, athletic and manual activities require psychomotor skills in which physical movements are linked to mental processes. “Learning these requires hands-on practice, direct feedback, and reflection. Everyone is familiar with video tutorials, but they lack any feedback,” explains project leader Prof. Dr. Roland Klemke from the Cologne Game Lab.
The research project “Multimodal Immersive Learning with Artificial Intelligence for Psychomotor Skills” (MILKI-PSY) designs an innovative environment for independent learning of psychomotor skills. For this purpose, the correct movement sequences are recorded by trainers using cameras and sensors. A virtual avatar generated from this recording will then serve as a model for the learners. This can be displayed on a large screen, in an augmented or virtual reality environment, for example. With the help of artificial intelligence and automated error detection, the learning progress is analyzed and individual feedback is generated.
“The learning environment will be created for basic sequences and fundamental movements of sports. There, it can be a useful complement to the trainer by allowing skills to be practiced and immediate feedback to be provided with the highest precision in real time. In this way, it will be easier for dancers, for example, to improve their psychomotor skills and adapt their own movement to the optimized movement of the expert,” says Klemke. In collaboration with the Institute for Product Development and Design Technology (IPK) at TH Köln, the approach is also being investigated for psychomotor training in the field of human-robot interaction.
In the project, the Cologne Game Lab is responsible for constructing the immersive environment with which users will interact. Immersion refers to diving into another world. “Using technologies from video games, modern instruments from sensor technology and artificial intelligence, we want to test the extent to which independent learning of psychomotor skills can be better supported,” says Klemke.