[LUM#20] The Metaverse in Motion
Improving communication between people who encounter one another in virtual worlds is one of the goals of the European ShareSpace project, in which the EuroMov laboratory is participating. This challenge requires reintroducing movement in all its dimensions into these metaverses in order to facilitate social interactions. Benoît Bardy, a researcher specializing in the study of human movement, explains1.
Imagine this: a coworker offers you a cup of coffee. His hand reaches for the cup sitting on the table in front of him and picks it up. Is he going to bring it to his mouth to drink, or is he going to hand it to you? In reality, you probably don’t even need to ask yourself this question, because you’ve already figured out his intention. “His gaze shifting from the cup to you, the way he reaches for it, the angle of his elbow, and the direction his wrist takes—these are all signs that provide very early clues about the other person’s intention, ” explains Benoît Bardy, a researcher at the EuroMov laboratory.
Sensory-Motor Communication
All these cues—which researchers refer to as “movement primitives”— form the basis of sensorimotor communication, which plays a major role in social interactions. “Eye contact, gestures, emotions—these expressions are central to the way we communicate, ” explains the movement specialist. These are all cues that we’re able to perceive in face-to-face interactions, but what happens when the person you’re talking to isn’t in the same room? “Then we lose much of our ability to detect these intentions in the gestures of the people we’re talking to.”
This is a major research challenge at a time when virtual reality is on the rise and the metaverse—defined as a hybrid world that can transcend the real world, particularly through augmented reality—is garnering increasing attention. How can we identify these motor signatures of intention in digital environments? This is one of the key challenges addressed by the European research program ShareSpace, in which the EuroMov laboratory is participating.
A project with an innovative vision: the creation of future hybrid social spaces shared by humans and avatars. Here, the well-known social sensorimotor primitives are captured using innovative, connected mobile sensors and then reconstructed using a new extended reality technology. “This allows us to enhance virtual communication by recreating these gestures, which we can amplify or even attenuate depending on the application, ” explains Benoît Bardy.
Avatar
And to explore the metaverse, ShareSpace incorporates four levels of interaction into its scenarios, blending the real and the virtual. At the first level, called L0, humans interact in the real world to better assess, identify, and calibrate sensorimotor situations. “At this level, we offer synchronization games in particular, which are recognized as vehicles for empathy and social bonding, ” explains the movement specialist.
At the next level, or L1, all humans find themselves in a virtual world, where each person has an avatar that replicates their movements. “This is a non-autonomous avatar that will replicate our movements and interact with the avatars of other human beings.” In practice, each participant stays “at home,” equipped with a virtual reality headset, and meets up with others in the metaverse.
The next level, L2, is characterized by a certain degree of autonomy granted to the avatars. “For example, when playing a synchronization game together, if one of the participants begins to fall out of sync with the rest of the group, their avatar can adjust its sensorimotor response to get back in step with the group.” Finally, at the last level (L3), we meet Sarah, an artificial intelligence built using a mathematical model. “If one of the participants falls behind in the synchronization game, Sarah is able to notice this, go find them, and guide them in a personalized way to resynchronize them with the group. We’re creating a hybrid world that blends reality, virtual reality, and augmented reality.”
Virtual Rehabilitation
What are some practical applications for this shared metaverse? “SharesSpace focuses on three areas: health, sports, and art,” replies Benoît Bardy. In the health sector, the project aims to treat lower back pain by enabling patients with chronic back pain to participate in group rehabilitation sessions with a therapist. These sessions are unlike any others, as both real-world and virtual patients follow the guidance of an autonomous avatar. “A therapist represented by their avatar can closely monitor whether the patient is performing the necessary exercises correctly and correct them if needed, ” explains the specialist from the EuroMov laboratory.
On the sports side, ShareSpace offers cyclists the chance to learn how to ride together in a peloton in a safe environment . “It’s an app that’s particularly well-suited for children, who will learn, for example, how to navigate around an obstacle with the help of Sarah, who is there to amplify the parents’ movements so their intentions are easier to understand, ” explains Benoît Bardy. An app will also be available for cyclists with a competitive focus, “where, on the contrary, you sometimes need to know how to hide your intentions to better set up a breakaway from the peloton,” the researcher explains. Be sure to visit the ShareSpace booth in Paris during the 2024 Olympic Games.
Finally, regarding the artistic aspect, the project’s creators have given carte blanche to a collective of artists from the European Ars Electronica Foundation, who will use this technology to create an immersive choreography for the Art and Technology Festival in Vienna in September 2024. This performance can be experienced both in the real world and in the virtual world.
Check out theUM podcasts,UM available on your favorite platform (Spotify, Deezer, Apple Podcasts, Amazon Music, etc.).
- Euromov (UM, IMT Mines Ales)
↩︎