Metavers in motion

One of the aims of the European ShareSpace project, in which the EuroMov laboratory is participating, is to improve communication between individuals who cross paths in virtual universes. It's a challenge that involves reinjecting all aspects of movement into these metavers, to facilitate social interaction. Explanations from Benoît Bardy, a researcher specializing in the study of human movement.¹.

Imagine, a colleague at work offers you a cup of coffee. His hand goes to the cup on the table in front of him and grabs it. Is he going to put it to his mouth to drink, or is he going to hand it to you? In reality, you probably don't need to ask yourself this question, because you've already identified his intention. " The way he looks from the cup to you, the way he grabs it, the angle of his elbow, the direction taken by his wrist - these are all signs that give very early indications of your interlocutor's intention," explains Benoît Bardy, researcher at the EuroMov laboratory.

Sensorimotor communication

All these cues, which researchers call "movement primitives ", form the basis of sensorimotor communication, which plays a major role in social interaction. " Glances, gestures, emotions - these expressions are central to the way we communicate," explains the movement specialist. So many cues that we're able to perceive face-to-face, but what about when your interlocutor isn't in the same room? "Then we lose much of our ability to detect these intentions in the gestures of our interlocutors.

This is a real research challenge at a time when virtual reality is on the rise, and when the metaverse- defined as a hybrid world that can go beyond the real world, thanks in particular to augmented reality - is making more and more headlines. How can we find these driving signatures of intention in digital universes? This is one of the challenges of the ShareSpace European research program, in which the EuroMov laboratory is participating.

A project with an innovative vision: the creation of future social hybrid spaces shared by humans and avatars. Here, the famous social sensorimotor primitives are captured using innovative mobile connected sensors, then reconstructed using a new extended reality technology. " We can thus promote virtual communication by recreating these gestures, which can be amplified or even attenuated depending on the application," explains Benoît Bardy.

Avatar

And to enter the metaverse, ShareSpace integrates four levels of interaction into its scenarios, mixing the real and the virtual. At the first level, called L0, humans interact in the real world to better assess, identify and calibrate sensorimotor situations. " At this level, we propose synchronization games, recognized as vectors of empathy and social affiliation," explains the movement specialist.

At the next level, or L1, all humans find themselves in a virtual world, where everyone has an avatar that will copy our gestures. "This is a non-autonomous avatar that reproduces our movements and interacts with the avatars of other human beings. In practice, each participant remains "at home" equipped with a virtual reality headset, and joins the others in the metaverse.

The higher L2 level is characterized by a certain autonomy given to avatars. "For example, when we play a synchronization game together, if one of the participants starts to get out of sync with the rest of the group, his avatar can adapt its sensorimotor response to get back into unison with the group." Finally, on the last level (L3), we meet Sarah, an artificial intelligence built using a mathematical model. "If one of the participants is behind in the synchronization game, Sarah is capable of noticing, picking them up and taking them away in a personalized way to resynchronize them with the group. We create a hybrid world combining reality, virtual reality and augmented reality."

Virtual rehabilitation

What are the concrete applications for this shared metaverse? " SharesSpace is positioned in the three dimensions of health, sport and art," answers Benoît Bardy. On the health side, the project aims to treat low back pain by enabling patients suffering from chronic back pain to take part in group rehabilitation sessions with a therapist. Sessions like no other, where real and virtual patients follow the advice of an autonomous avatar. " A therapist represented by his or her avatar can closely monitor whether the patient is performing the necessary exercises correctly, and correct him or her if necessary", explains the EuroMov laboratory specialist.

For the sporting dimension, ShareSpace offers cyclists the chance to learn how to ride together in a peloton in a safe environment, "an application very well suited to children who will learn, for example, how to get around an obstacle with the help of Sarah, who is there to amplify the parents' gestures to make it easier to understand their intentions", explains Benoît Bardy. An application will also be offered to cyclists with a competitive vision, "where you sometimes need to know how to mask your intentions to better prepare for a breakaway from a peloton", illustrates the researcher. See you at the ShareSpace stand in Paris during the 2024 Olympic Games.

Finally, as far as the artistic dimension is concerned, the project's authors have given carte blanche to a group of artists from the European foundation Ars Electronica, who will use this technology to create an immersive choreography for the Art and Technology Festival in Vienna in September 2024. A show that can be experienced in reality, as well as in virtuality.

1. Euromov (UM, IMT Mines Ales)
   ︎