[LUM#20] The Metaverse in Motion
Improving communication between people who encounter one another in virtual worlds is one of the goals of the European ShareSpace project, in which the EuroMov laboratory is participating. This challenge requires reintroducing movement in all its dimensions into these metaverses in order to facilitate social interactions. Benoît Bardy, a researcher specializing in the study of human movement, explains1.
Imagine this: a coworker offers you a cup of coffee. His hand reaches for the cup sitting on the table in front of him and picks it up. Is he going to bring it to his mouth to drink, or is he going to hand it to you? In reality, you probably don’t even need to ask yourself this question, because you’ve already figured out his intention. “The way their gaze moves from the cup to you, the way they reach for it, the angle of their elbow, and the direction their wrist takes—these are all signs that provide very early clues about the other person’s intention, ” explains Benoît Bardy, a researcher at the EuroMov laboratory.
Sensory-Motor Communication
All these cues—which researchers call “movement primitives”— form the basis of sensorimotor communication, which plays a major role in social interactions. “Eye contact, gestures, emotions—these expressions are central to the ways we communicate, ” explains the movement specialist. These are all cues we’re able to pick up on in face-to-face interactions, but what about when the person you’re talking to isn’t in the same room? “In that case, we lose much of our ability to detect these intentions in the gestures of the people we’re talking to.”
This is a major research challenge at a time when virtual reality is on the rise and the metaverse—defined as a hybrid world that can transcend the real world, particularly through augmented reality—is garnering increasing attention. How can we identify these motor signatures of intention in digital environments? This is one of the challenges addressed by the European research program ShareSpace, in which the EuroMov laboratory is participating.
A project with an innovative vision: the creation of future hybrid social spaces shared by humans and avatars. Here, the well-known social sensorimotor primitives are captured using innovative, connected mobile sensors and then reconstructed using a new extended reality technology. “This allows us to enhance virtual communication by recreating these gestures, which we can amplify or even attenuate depending on the application, ” explains Benoît Bardy.
Avatar
And to explore the metaverse, ShareSpace incorporates four levels of interaction into its scenarios, blending the real and the virtual. At the first level, called L0, humans interact in the real world to better assess, identify, and calibrate sensorimotor situations. “At this level, we offer synchronization games in particular, which are recognized as vehicles for empathy and social bonding, ” explains the movement specialist.
At the next level, or L1, all humans find themselves in a virtual world, where each person has an avatar that mimics their movements. “This is a non-autonomous avatar that replicates our movements and interacts with the avatars of other human beings.” In practice, each participant stays “at home,” equipped with a virtual reality headset, and meets up with others in the metaverse.
The next level, L2, is characterized by a certain degree of autonomy granted to the avatars. “For example, when playing a synchronization game together, if one of the participants begins to fall out of sync with the rest of the group, their avatar can adjust its sensorimotor response to get back in step with the group.” Finally, at the last level (L3), we meet Sarah, an artificial intelligence built using a mathematical model. “If one of the participants falls behind in the synchronization game, Sarah is able to notice this, go find them, and guide them in a personalized way to resynchronize them with the group. We’re creating a hybrid world that blends reality, virtual reality, and augmented reality.”
Virtual Rehabilitation
What are some practical applications for this shared metaverse? “SharesSpace focuses on three areas: health, sports, and art,” replies Benoît Bardy. In the health sector, the project aims to treat lower back pain by enabling patients with chronic back pain to participate in group rehabilitation sessions with a therapist. These sessions are unlike any others, as real-world and virtual patients follow the guidance of an autonomous avatar. “A therapist represented by their avatar can closely monitor whether the patient is performing the necessary exercises correctly and correct them if needed, ” explains the specialist from the EuroMov laboratory.
On the athletic side, ShareSpace offers cyclists the chance to learn how to ride together in a peloton in a safe environment . “It’s an app that’s particularly well-suited for children, who will learn, for example, how to navigate around an obstacle with the help of Sarah, who is there to amplify the parents’ movements to make their intentions easier to understand, ” explains Benoît Bardy. An app will also be available for cyclists with a competitive focus, “where, on the contrary, you sometimes need to know how to conceal your intentions in order to better set up a breakaway from the peloton,” the researcher explains. Be sure to visit the ShareSpace booth in Paris during the 2024 Olympic Games.
Finally, regarding the artistic aspect, the project’s creators have given free rein to a collective of artists from the European Ars Electronica Foundation, who will use this technology to create an immersive choreography for the Art and Technology Festival in Vienna in September 2024. This performance can be experienced both in the real world and in virtual reality.
Check out theUM podcasts, now available on your favorite platform (Spotify, Deezer, Apple Podcasts, Amazon Music, etc.).
- Euromov (UM, IMT Mines Ales)
↩︎