Immersive tech reshapes music and film landscape with Bono, Metallica and 'Matrix' taking the leap

INGLEWOOD, Calif. — With a mic in hand, Bono belts out ''Vertigo,'' close enough to reach out and touch, as if you're sharing the stage with him at the Beacon Theatre in New York.

As he performs, the front row seat experience becomes deeply personal, with the U2 frontman seemingly locking eyes with you before the camera swings to his side to reveal the crowd, pumping their fists in excitement. The song's lyrics leap from beneath Bono's feet through 3D animation, gathering around him while hovering over the crowd.

This isn't your typical concert. It's an all-access pass into Bono's performance and documentary film ''Bono: Stories of Surrender," which premiered last week on Apple TV+ and gained new life through the Apple Vision Pro. The device places viewers right onstage with Bono, delivering it all in 8K with spatial audio for a fully immersive 180-degree video.

''There's a kind of intimacy afforded to us,'' said Bono, who added that immersive technologies tap into viewers senses and emotions more deeply than traditional media. ''It's not just being in the room. You can be in your head.''

These days, experiences like Bono's concert film have become the next frontier of music, film and live performances. Headset devices like the Vision Pro and Meta's Quest 3, alongside next-gen venues such as the Sphere and Cosm, are making these experiences borderless, turning every seat into the best one in the house.

Some of music's biggest names — including U2, Metallica, Megan Thee Stallion, Alicia Keys, T-Pain, Phish and The Weeknd — have had projects featured on these platforms. Even the iconic film ''The Matrix'' has been reintroduced in this immersive space.

''This is the future. These experiences enhance the connection,'' said Robert Trujillo, guitarist for Metallica, whose powerhouse Mexico City performance was delivered in an immersive concert experience through the Vision Pro in March. After watching their show on the device, he was in awe.

''It almost felt like you could touch the crowd and us,'' he said. ''It was really incredible. The sound is amazing. The dynamic ranges from the ambient noises and the crowd. It's a special experience.''

How music greats view future of immersion tech for industry

Bono has long embraced innovation and technology to forge a connection with U2's audience. The band's 2014 album ''Songs of Innocence" was added to every Mac, iPhone, iPad, and iPod. In 2023, the foursome became the first band to perform at the Sphere, launching a successful residency at the $2.3 billion, 366-foot-tall Las Vegas venue that illuminates the city's skyline.

U2 guitarist The Edge felt the Sphere was a ''quantum leap forward'' in terms of the sensory impacts of a live concert.

Now, with ''Stories of Surrender,'' Bono goes solo, reflecting on his life from childhood and family to his career as a rock star and activist. It's the first feature film shot in Apple Immersive Video, bringing the artwork from his book ''Surrender: 40 Songs, One Story'' to life with 3D animation.

Bono even used the Apple Pencil Pro to create images featured in the film. He believes the immersive technology can connect with audiences on a profound level and hopes the $3,500 Vision Pro will eventually become more affordable.

''All of this is impressive,'' he said. ''I think about the film, when I'm singing about my mother, and (my mother's name) Iris start to appear. I was completely taken aback, even though I'd drawn the Iris. There are other moments when I see my father in the chair beside me as a drawing partially by me.''

Bono added: ''Then I thought, ‘We might be getting beyond technology here into some kind of magic. Not just a magic trick like 'Hey, presto.' But more alchemy, more of ‘What did I just feel?'"

Trujillo, who typically avoids watching replays of his performances, found himself drawn in by the immersive experience of their show, which included interviews and close-ups of fans. He said the technology amplifies the energy of a live performance at a time when that excitement can feel diminished.

''For a lot of bands, especially in the hard rock genre, it felt like it was disappearing,'' he said. "To me, this really brings it back and relevant again. It shows you the strength in the performance and the emotional attachment to the fans.''

How immersive venues are making their mark

While virtual reality headsets offer a personal window into performances, venues like the Sphere and Cosm are expanding these experiences to entire crowds.

In their own way, both are redefining the possibilities of live performances and shared experiences.

''It's virtual reality without the glasses,'' said Baz Halpin, the CEO and founder of Silent House Group, who produced and designed the Eagles' residency at the 17,500-seat Sphere.

Halpin said Silent House will also be involved in the Backstreet Boys upcoming Sphere residency, kicking off in July.

''You can create optical illusions that I didn't think were frankly possible,'' Halpin said. ''It's such a big swing. It's such a bold idea. It's still that concert experience, but somehow the visuals have turned into a rollercoaster and movie, and you're apart of it."

Shared reality through "The Matrix"

While the Sphere looms large over Las Vegas, Cosm offers a more intimate experience, featuring a new venture into Keanu Reeves' iconic world as Neo.

Cosm is stepping into ''The Matrix'' with a shared reality cinematic experience, celebrating the film's 25th anniversary with screenings at their locations in Dallas and Inglewood, California, starting June 6.

Inside Cosm's 87-foot LED dome, the original film plays while the environment dynamically mirrors the on-screen action, pulling audiences deeper into the world.

''We are continuing to push the immersive tech industry forward,'' said Jeb Terry, the CEO and president at Cosm, which also hosts live sports, ''O'' by Cirque du Soleil, ''Orbital'' and ''Big Wave: No Room for Error." The company plans to open its third location in Atlanta in 2026.

With ''The Matrix,'' Cosm believes the shared reality experience will pave the way for showcasing more classic films, complementing the traditional movie theater experience.

''We're really commemorating some of these legendary amazing films,'' said Alexis Scalice, Cosm's vice president of business development and entertainment. ''We're keeping that interactive and exciting approach to it.''

What's next for Sphere?

The vision for a second Sphere location will come to fruition in the United Arab Emirates. In collaboration with the Abu Dhabi Department of Culture and Tourism, the new venue will replicate the cutting-edge technology and scale of the Las Vegas version.

The Abu Dhabi Sphere is part of a broader plan to establish additional Spheres globally and domestically, alongside smaller U.S. venues with seating capacities of 3,000 to 6,000.

''We are advancing our vision of this global network,'' said Jennifer Koester, the president and COO at Sphere. She said the smaller Spheres will make the concept more deployable across different markets.

As construction of the Abu Dhabi location progresses, the Las Vegas Sphere is preparing to bring a timeless classic to life. In late August, the original ''The Wizard of Oz'' is set to premiere at the venue.

Koester said AI models were trained on original scene drawings, set designs and materials sourced from the Academy Museum to fully fill the Sphere's massive screen. She said that evolving the technology is key to staying ahead.

''In the industry, we're seeing a lot of people talk about bringing forward experiences that are more immersive,'' she said. ''People are looking for it, and people are expecting it. The bar is going to continue to get higher and higher.''