Dance transcends cultural boundaries, but can it overcome the obstacle of physical separation?
With the power of 5G and the Advanced Media Production Network, artistic duo Gibson/Martelli is defying the confines of location and physicality and exploring expression, movement and connection between dancers located in studios over 250 miles apart.
WOW+FLUTTER is the latest project by Gibson/Martelli, the immersive focused art duo. It represents a significant advancement in dual-site performance and interactive virtual production. Utilising cutting-edge 5G technology, it establishes a real-time connection between performers across North and South East England, facilitating the exploration of new digital performance spaces.Â
WOW+FLUTTER builds on previous multi-site immersive technology explorations, like the DAZZLE project which merged dance and optical illusions in an interactive VR exhibition inspired by Vorticism, Futurism and Dadaism. Performers manipulate virtual objects that correspond to physical objects, creating a unique blend of mixed reality that is immersive and responsive. The project enables creative interactions between physical and digital environments through high-speed video transmission in virtual production studios, motion capture, virtual environments, live music, costumes, and advanced digital rendering.
The Production
The project’s core is built on a digital twin concept, where physical and virtual spaces mirror each other in real time. This is enabled by a 10GB interconnect, allowing for data-intensive, low-latency connections, essential for immersive, multi-site performances between Advanced Media Production PROTO and Advanced Media Production Dock Street. The 5G technology amplifies these capabilities, providing ultra-low latency, high bandwidth, and the ability to handle complex data streams. This ensures that performers can interact in real time with digital elements while maintaining natural movement, despite the distance.
The 5G interconnect also enables real-time transmission of motion capture data, fundamental to the digital twin experience. Performers in motion capture suits control their digital avatars, with movements instantly mirrored on LED screens at both sites. Close-up shots of rehearsals show how performers engage with physical and virtual props, blending them seamlessly as if they exist in both worlds.
The Challenge
One challenge during the rehearsal process was technical lag between sites. The 5G network’s ultra-low latency reduces these issues, however through the project the team explored how to adapt to the small delays. Likening it to an improvisational jazz session, the performers learnt how to use them creatively rather than as disruptions. This experimental approach underscores the project’s innovative nature, allowing for fluid, reactive performance.
Integrating multiple systems—motion capture, virtual environments, and video streaming—into a cohesive performance was also a challenge. The technology stack was described as a "house of cards," requiring flawless operation at each layer to sustain the illusion. Multi-GPU workflows and edge-compute capability managed the computational load, essential for handling large motion capture and video datasets while maintaining low-latency interaction.
The Result
The WOW+FLUTTER project bridges physical and virtual spaces, setting a new standard for immersive, dual-site performances, merging technology and artistry to create innovative experiences that push the boundaries of traditional stagecraft. This multi-site collaboration feels live and intimate, transcending traditional stage setups and offering new audience engagement and remote collaboration possibilities.
The use of 5G technology has commercial applications beyond performance art, offering potential for training, education, and remote collaboration across industries. The technology developed for WOW+FLUTTER could be adapted for sectors like healthcare, manufacturing, and corporate training, where real-time interaction between physical and virtual worlds can be transformative.
The Feedback
The performance raised ideas for augmented reality (AR) integration, such as using AR glasses to let performers interact with virtual avatars or remote partners as if they were physically present. While some dancers saw potential in this addition, others felt there was more to explore with current technology before introducing AR. Still, integrating AR with 5G points to exciting future directions, enabling richer engagements in mixed reality environments.
Performers: Zara Philips, Rob Anderson, Ally Clarke, Ruby Portus
WOW+FLUTTER is funded by Innovate UK Launchpad (2024), with generous support from BFI London Film Festival Expanded (PAN+TILT, 2022), Research England, Centre for Dance Research, Coventry University (Performers of the Future, 2023), and Pro Helvetia - Swiss Arts Council, CIE Gilles Jobin-Studio 44 Mocaplab Residency (2023).
Interested in exploring virtual production? It's not just for filmmakers. Chat to us about how we can get you started: info@studiot3d.com.
Comments