I have long been fascinated by headphone virtualization, that is, audio processing for headphone playback, so it seems to come from different directions outside of your head. One of my first experiences with it was from an Australian company called Lake, whose technology was purchased by Dolby and became known as the Dolby Headphone.
Since then, several companies have introduced headset virtualization systems. Among the most recent, the 360 Reality Audio from Sony, which was widely presented for the first time at CES 2019. For a demo that year, Sony installed an immersive speaker system (facades, surrounds and ceiling lights) ) in a room with several seats, each with its own helmet. We listened to a few clips on the speakers, then on the headphones again. The headphones sounded remarkably similar to the speaker system, including the individual sound-emitting objects that seemed to be well out of my head, coming from everywhere around and above me.
Last October, 360 Reality Audio was launched to the consumer market, so I was looking forward to trying it out. It was definitely worth the wait.
features
Sony’s 360 Reality Audio is an immersive object-based audio system, much like Dolby Atmos or DTS: X in principle, although its intended application is music rather than soundtracks – and, of course, played with headphones. (It can also be used with the Amazon Echo Studio speaker, which I’ll explain briefly.) Content creators mix audio tracks using immersive audio tools, placing instruments anywhere in a space Virtual 3D. The final mix is encoded in MPEG-H, an audio codec that supports immersive audio.
Music mixed and encoded in 360 Reality Audio is delivered via streaming providers. Currently, three suppliers offer 360RA headphone titles: Tidal, Deezer and Nugs. (Amazon Music HD offers 360RA tracks for the Amazon Echo Studio speaker.) Of course, each of these providers charges subscription fees to access their catalog, but they also offer a free trial to check them out. As of this writing, these providers are offering a total of more than 1,000 new and classic tracks that have been mixed or remixed in 360RA format and generally delivered at a bit rate of around 1.5 Mbps or less, depending on the service.
All headset virtualization systems simulate the effect of sounds reaching our ears from different directions in our environment. For example, a sound from your right reaches your right ear before reaching your left ear, and your brain uses this tiny difference in time to distinguish exactly where the sound comes from. In addition, the specific shape of your head and outer ears changes the spectrum of sound by diffracting around them before entering your ear canals, providing your brain with additional clues as to where the sound comes from. This can be mathematically modeled with something called a head-related transfer function (HRTF).
Headset virtualization systems, including 360RA, can use a generic HRTF derived from an average of many individual measurements or a dummy head that is “average”. This allows 360RA titles to be played on any headset, regardless of make or model, and it works surprisingly well.
But 360RA goes even further. If you have a pair of Sony headphones selected (click this link and scroll down for a list of compatible models, you can use the Sony Headphone Connect app for iOS or Android to customize the HRTF for the unique shape of your individual ears. Using the app, you take a picture of each ear, after which the app calculates your specific HRTF and optimizes the streaming provider’s app accordingly.
In addition to the headphones, 360RA is also designed to work with certain speakers. The only speaker of this type available today is Amazon Echo Studio, which can play 360RA tracks from the Amazon Music HD streaming service. This speaker includes speakers that shoot in four lateral directions as well as upward, creating an omnidirectional soundstage. In this case, the 360RA rendering engine is integrated into the speaker; with a headset, the rendering engine is implemented in the streaming application.
To evaluate 360 Reality Audio for this review, I used a pair of Sony WH-1000XM3 Bluetooth headphones and the Tidal streaming provider on an iPhone XS. TechHive has already reviewed the WH-1000XM3, so I won’t duplicate this effort. I will say that I fully agree with critic Adam Murray that this is an exceptional noise canceling Bluetooth headset; in fact, it is the most comfortable over-ear headphones I have ever used.
User interface
To optimize the 360 Reality Audio experience for your individual ears, you need a pair of selected Sony headphones and the Sony Headphone Connect app. After pairing the headset with your mobile device, you go to the 360 Reality Audio Setup section, where you can play a short clip in “normal” audio and 360RA. I found the difference striking.
Then the app takes pictures of each ear. It guides you to orient the phone directly in front of you so that your face fills an outline on the screen. He then asks you to turn your head to the right so that you can take a picture of your left ear. If the alignment is not correct, a voice asks you to tilt the phone up, down, right or left; when the alignment is good, it automatically takes the photo. Once done, you repeat the process for your right ear.
After taking photos of your ears, the app analyzes the shape of your ear and asks you to select the streaming provider you want to optimize. Once you have made a selection, it launches the supplier application and applies the optimization. The whole process is quite simple.
Performance
I started by listening to several Tidal tracks in their conventional stereo HiFi versions and their 360 Reality Audio versions before going through the individual optimization process – in other words, I used the generic HRTF which is applied to any helmet for this part of my assessment. Tidal has over 1000 360RA tracks, so I had a lot of choices. (Interestingly, when I searched for “360 Reality Audio”, I only saw 17 tracks, but by pressing “360 Reality Audio” on the Explore page, the entire catalog of full albums was displayed.)
The first was Herbie Hancock’s classic “Watermelon Man” Headhunters album. In conventional stereo mixing, everything seemed very close and “in my head” with a strong left-right separation. The 360RA mix was much more expansive and out of my mind. In fact, it was a much more comfortable listening experience. I particularly liked the intro in 360RA, with various instruments and vocal sounds added in layers all around in the 3D space.
Then, “Take Five” from Dave Brubeck’s album Free time. As before, the stereo mix was all in my head, as I have heard countless times. And again, the 360RA mix was much more extensive; the drums and bass were still on the left, the piano was still on the right, and the alto sax was in the center-right, but they seemed more distant with more space between them. The sax sounded a bit artificial in this version, but overall, the sound was much more relaxed and more pleasing to the ears.
I wrote almost exactly the same notes for Marvin Gaye’s “Sexual Healing” from his album Midnight love. The stereo mix sounded good, but it seemed quite crowded in my head compared to the 360RA version, which spanned the space around me. In all cases so far, the overall level of the 360RA mix seemed slightly lower than that of the stereo mix.
One of my favorite bands in college was the Mahavishnu Orchestra, conducted by amazing guitarist John McLaughlin, so I listened to “Birds of Fire” from the album of the same name. As I expected, the stereo mix was all in my head and the 360RA mix was more extensive, but not as much as the previous tracks. In addition, the drums and bass were much weaker in the mix than in the stereo version, and the opening gongs and bass had a very different tone. Clearly, whoever did the 360RA remix decided to balance and equalize the instruments differently. In this case, I preferred the original stereo mix.
For a more modern song, I listened to “Don’t Let Me Down”, a Chainsmokers single with singer Daya. Of course, the stereo mix was all in my head, and it sounded slightly muffled in places. The 360RA mix didn’t seem as far from my head as most of the other songs I listened to, but it was still much larger and more open with more space between instruments and vocals. It’s an extremely consistent mix, and it didn’t have those slightly muffled moments. It was a joy to listen, even if it’s not my cup of tea musically.
Unfortunately, there is no way to quickly switch between generic and individual HRTF. Once you’ve optimized the streaming app, it renders all 360RA tracks using your unique HRTF. However, I applied my individual optimization and listened to the same tracks as before. As far as I can remember, I did not hear a significant difference between generic and individual HRTFs. I guess it’s because the optimization process doesn’t take into account the shape of your head, only your ears.
In addition to the WH-1000XM3, I also listened to a few tracks on a pair of 1More Stylish Bluetooth in-ear headphones, which can only use generic HRTF. I noticed very similar results, with the stereo mix ringing entirely in my head and the much more expansive 360RA mix.
Conclusion
I am extremely impressed with Sony’s 360 Reality Audio system for headphones. It allows artists to effectively expand the soundstage in a 3D space around the listener, resulting in a much more comfortable and fascinating musical experience.
Of course, this experience ultimately depends on the artist who mixes their music in 360RA. With most of the songs I listened to, I strongly preferred the 360RA mix over the stereo mix; the only exception was “Birds of Fire” by the Mahavishnu Orchestra, which in my opinion could have been remixed much better.
With over 1,000 titles already available, you are sure to find lots of music to suit your particular tastes. Even better, thousands more are in the works; for example, David Bowie’s “Space Oddity” will soon be published in 360RA.
If you use certain Sony headphones, you can customize the HRTF used to create the illusion of space based on the shape of your individual ear, although I didn’t find it very different from the generic HRTF. Either way, Sony’s 360 Reality Audio adds a new dimension to audio streaming over headphones; in fact, it’s worth getting a subscription to Tidal, Deezer or Nugs just to hear it. I highly recommend listening to it for a long time.