The virtual orchestra

In January last year, the Philharmonia Orchestra previewed its first virtual reality (VR) experience to journalists at Southbank Centre’s Classical Season press event. Produced in partnership with Southbank Centre and specialists Inition, the film is a four-and-a-half-minute VR presentation that transports viewers to the South Bank in London, then backstage into the Royal Festival Hall with our Principal Conductor and Artistic Advisor Esa-Pekka Salonen as he prepares for and then conducts a performance of Sibelius’ Fifth Symphony.

We are unusual for an orchestra in that we have a dedicated in-house digital department, which reports directly to the Managing Director, with genuine engagement from Salonen. Over the past ten years, the digital team has created a wide range of experiences and products, from interactive web documentaries and iPad apps, through to a music truck and large-scale immersive exhibitions that have toured globally.

One of the most rewarding aspects of the project was to see the positive response from classical music lovers

Creatively, VR felt like a natural progression for us, and I believe it’s a fantastic medium for orchestral music. 2D video and stereo audio can struggle to convey the complex dynamics and visceral energy of a live symphony orchestra. This could be why livestreaming orchestral performances to cinemas has not had the widespread success enjoyed by theatre and opera companies. An orchestra just comes across as too flat, with the players at risk of looking detached and remote from the viewer.

First project

With limited budget and resources, the only way we could embark on our first VR project was in partnership, and I approached London-based VR company Inition in early 2015. They had one of the few cameras at that time that could film live action in full 360-degree, 3D (stereoscopic) video, and we were both particularly interested in developing 3D audio.

After the press preview, The Virtual Orchestra was then premiered in London at the digital festival at Southbank Centre in September last year. The festival showcased the VR on headsets in the foyer, with the much larger installation ‘Universe of Sound’ occupying the Clore Ballroom. Both works aimed to challenge preconceptions about orchestras, whether the audiences were newcomers or aficionados.

One of the most rewarding aspects of the project was seeing the positive response from classical music lovers, as they often have the most emotional reactions, experiencing a piece they love, like Sibelius 5, from a completely new perspective.

As with the installations, the VR plays a key role in our audience development strategy, taking the orchestra outside the concert hall – whether into the foyer, a tent or a truck – first digitally, but then live in concert. With all of our digital projects, the concert hall is also the end-destination. At the end of the festival, Salonen conducted the orchestra with a specially programmed concert at the Royal Festival Hall, catering directly for the new visitors who had come to try the digital experiences.

Since releasing that first VR film, we’ve gone on to work on two more projects. The second film, Beethoven’s Fifth, was commissioned by Google in partnership with NASA, and will be released later this month. A further VR project is in development. Our goal is to make VR filming part of our in-house video team’s skillset, so that we can make it a core part of our output.

The main challenges

Although I believe VR has a bright future, in the short term there are three major obstacles for arts organisations when considering whether to embark on a VR project: cost, quality and distribution. At the moment, VR is expensive to produce. The main challenges are that the camera rigs are still at the very early stages, and most require a high level of manual ‘stitching’ work in post-production.

Similarly, although the experiences are genuinely compelling, consumer VR headsets are really at the very first iteration, so they can be bulky and the quality is often low. For example, you can see stitching errors and pixels when watching most VR films. However, the technology is progressing extremely rapidly, with costs expected to drop alongside a noticeable improvement in quality over the next two to three years.

Finally, the biggest barrier is distribution, both technically and commercially. From a technical perspective, we underestimated how much time it would take to complete the post-production work on our first VR film, and then distribute it as a film or app to the various VR platforms that are currently available. Inition did a fantastic job throughout the project, but the stitching of our first VR film took 300 edits in post-production.

Similarly, we are still encountering problems preparing the film for the various VR platforms that are available. We made the strategic decision to prioritise Sony PlayStation VR for a few reasons. It is one of the highest-quality platforms with a pre-existing market of 80 million global users with PlayStation 4s (who can buy a VR headset accessory for around £400, and they’ve already sold well over 1 million in less than a year). And it can be combined with ‘traditional’ 2D content on your TV, which allows VR to dovetail with ‘traditional’ video content on television.

Commercial and artistic outcomes

From a commercial perspective, we embarked on our first project knowing that the market for VR is still very small. We don’t yet have accurate figures to share on sales of our first app release on the PlayStation, but we planned for sales to be in the thousands initially.

Our core strategic aims for investing in VR were to widen the portfolio of digital installations that we can offer to audiences to complement the touring orchestra, and to develop the skills and partnerships to create VR in-house. To give an idea of figures, the large-scale installations have been seen by over 370,000 visitors since 2009, and the VR film has now reached over 10,000 physical visitors since beginning to tour in September last year.

From an artistic perspective, as yet we have only worked in ‘live action’ 360 video in VR, but we would be interested to explore the wider possibilities for interactivity and animation in VR. As an artform, VR seems to sit somewhere between film and the games world, although I believe it’s in its infancy, and has hugely exciting possibilities for music.

Luke Ritchie is Head of Digital Innovation and Partnerships at The Philharmonia.
www.philharmonia.co.uk

From Thursday 28 September to Sunday 1 October, The Virtual Orchestra: Sibelius 360, can be experienced on Level 2 at the Royal Festival Hall in London.