How Gaming Technology Is Reshaping Live Experience Across Cultural Sectors

In discussion with Allan Rankin, Managing Director of Target3D

What elements of gaming are being adopted in sectors such as live events, theatre and cultural institutions - and why are they so engaging?

Sectors like live events, theatre and cultural institutions are increasingly adopting elements from gaming - particularly the use of game engines and real-time systems. These engines allow creatives to create both linear narratives and trigger-based events, where the next action or scene is determined by a performer’s movement or the audience’s location.

Queer Ecology, Brigitte Baptiste and Institute of Digital Fashion, Our Time on Earth, Barbican Centre, Fran Taylor, 2022.

For instance, a performer stepping into a certain area on stage might trigger video or audio content. In a museum, a visitor walking in front of a painting might activate a sensor that plays a soundscape or launches a visual sequence. These interactions can even have cumulative effects, depending on how many times they’re “triggered” - making the content feel more responsive and alive.

Game engines allow for full 3D previsualisation of environments, performances and narratives. You can simulate everything in advance - lighting desks, audio systems, audience flow - and even rehearse without being in the venue.

This is a powerful way to collaborate creatively across teams and adapt content on the fly. If a dancer can't complete a move due to stage limitations, for example, the choreography and corresponding visuals can be updated in real time.

All of this creates experiences that feel more interactive, immersive and personal -  that’s what makes them so engaging.

How does mocap or 3D tracking support these experiences?

While people often refer to motion capture, in these kinds of live or immersive experiences it’s often more accurate to talk about 3D tracking. Tracking refers to tools that identify where a performer or participant is in time and space, and this can trigger content based on those actions and or their proximity. 

A strong example is Studio Drift’s installation Murmuring Minds, where robots move dynamically based on a tracked participant’s proximity. The participant wears a tracking device, and the robots - which could just as easily be projections, lighting effects, or audio elements - behave differently depending on how close the person is.

That kind of tracking creates responsive environments. It’s different from performance capture, which is usually for re-animating digital characters. Tracking is about real-time spatial awareness, allowing a performer’s movements to trigger or influence physical or digital content around them.

Are we seeing a shift from tension to collaboration between creativity and technology?

Yes - increasingly, artists are using technology as a means to expand their creative language, not compete with it. What used to feel like a tension is now becoming a true partnership.

WOW+FLUTTER, Gibson/Martelli, 2024.

For example, a dancer’s movement might be captured and rendered not as a literal human avatar, but as abstract visual forms - bubbles, cubes, or even a flag fluttering in space. This gives a new, expressive shape to a human performance. It becomes a new storytelling medium, where movement translates into something symbolic or emotionally resonant.

Artists like Gibson/Martelli do this beautifully - their digital installations reimagine physical performance through stylised avatars. And then you have Unwired Dance Studio’s Where We Meet, where a performer dances in real space while the audience listens to their internal monologue through headphones. At a certain point, the performer invites you to move with them. Whether you do or not affects the flow of the experience - it becomes co-authored, a conversation between performer and audience.

These aren’t just technical experiments - they’re new forms of emotional and creative communication.

Performer Livia Massarelli. Where We Meet, Unwired Dance Studio, 2025.

How is digital media reframing how we consume culture in traditional institutions like museums?

It’s adding new layers to how stories are told. Traditional museum experiences are often passive - looking at static objects or reading placards. But digital media adds sound, movement, spatial interaction - that activates a different emotional response.

When multiple senses are engaged - visual, audio, spatial - people form deeper connections to the content. Artists and curators now have more tools to guide emotion, shape narrative and create more lasting impressions.

It also creates multiple access points.

Some people engage best by reading, others by listening, others by doing. Interactive digital layers mean a museum piece isn’t just something you look at - it’s something you feel, hear and participate in.

So yes - digital media is reframing the experience. It’s not replacing traditional forms, it’s enhancing them, and allowing people to engage on a more personal level.

What are the key technologies enabling this crossover between gaming, art, and physical space?

At the centre of it is 3D tracking, projection tools and game engines like Unreal Engine or Unity. These allow for real-time rendering, spatial awareness and interactive content to adapt based on who’s in the space and what they’re doing. This is also achieved in real-time. 

Look at places like Meow Wolf or TeamLab - they’re creating projected spaces where the walls and floors are dynamic and reactive. A visitor’s position causes content to change. You feel like a player inside the artwork - not just a spectator.

This is made possible by technology that allows creators to map projection surfaces precisely, integrate tracking data and feed all that into an engine that updates visuals in real time. The result is immersive, reactive environments that feel alive - and crucially, personalised.

You’re not watching something - you’re inside it. 

That’s the crossover: gaming’s sense of agency, merged with storytelling, art and live performance.

---

Speak to our sales team at sales@target3d.co.uk or call us on (+44) 0203 488 2575 and take the first step towards bringing your motion capture project to life.