Exploring Binaural Audio: An Artist’s Journey

You’ve been working with binaural spatial music for many years now. What began your interest in this type of music?

Well… it actually began in the 80s when I worked with the Syncalvier – the first Digital Audio Workstation (DAW). I didn’t own one; they were extremely expensive, so I learned how to operate it and had access to one. You could program the sound moving left to right and right to left using different kinds of sound waves - sine wave, square wave, sawtooth, triangle … I was writing film scores and doing sound design for commercials. That’s when I started thinking more in terms of sound than music per se, if that makes sense. Basically, I found that writing and orchestrating music, and sound, or sonic textures was pretty much the same thing.

Then fast-forward to the early 2000s when I seriously became determined to play my horn through a bunch of pedals in pursuit of adding another dimension to my sonic palette. It started out mono and then expanded to stereo. Somewhere between 2010 and 2014 Marshall Grupp, a close friend and award-winning sound designer, turned me on to the YouTube video “The Virtual Barber Shop.” An actual haircut and story that was recorded binaurally. Well, that did it and I was hooked and started on a mission to create spatial immersive music.

Finally, in the spring and summer of 2015, I found how to create surround sound for a project I was collaborating on with Ikuo Nakamura, an award-winning holographer and 3D filmmaker. We created Minimus 3D Arkestra, an hour-long 3D visual and surround sound performance. Together, we created 8 pieces based on his videos from around the world.

Each one had a beginning, middle, and end, and during our performances, he would edit and mess with his footage in real time, and I would play my solo, layered accompanying pieces in real time.

Unlike playing to silent films that are static and set visually in length –something I had done a lot of, Ikuo and I could actually interact visually and sonically with each other, like jazz musicians do with sound. We changed the lengths of the pieces, tempos, and opened them up, etc. We did it at the off-Broadway theater, The 13th Street Repertory Theater for 5 weeks. That was my first formal performance in Quad, where I used 4 speakers, and placed one in each corner surrounding the audience.

After our 5-week run, using MAX MSP and SPAT, Ikuo designed a better, more flexible environment for me to work in that included both Quad and Spatial Binaural software.

Then, during COVID, I realized it was time to really explore SPAT because without live Quad performances (which I was doing a lot of) there is no way to share the music I create where it moves around ones head. All I could do was pan and move sound left and right which is basically 2D and very flat.

So that’s when I donned headphones and learned how to program SPAT to be able to hear the sounds move around your head binaurally like in a live quad performance. The only caveat is that to hear it move around as it's intended, it’s imperative to listen in with headphones or earbuds, or it doesn’t work.

What is the preparation process like when performing binaural spatial music live? Is it different than standard live music?

That’s a fairly complex question. In many ways, preparation for both types of gigs are very similar. For me, like I’m sure with most musicians, the more my chops are up, both playing and listening-wise, the more I have to draw from.

For my solo work, whether in Quad or Binaural, it's not about playing a lot of fast notes; actually, it’s quite the contrary and more about the simplicity and quality of the line. What I find, though, is the more limber and technical prowess I have available, the more comfortable and easier I feel to play less. Plus, when playing with effects the more flexible my embouchure is, the more I can adjust my playing to more precisely achieve the desired sound I’m going for with more control and clarity.

Probably the biggest difference between standard jazz gigs with a group and my solo binaural work is that it's all on me. It’s about all the parts and layered lines, melodies, and sounds I am playing, listening to, and reacting to, fitting together in a cohesive way that makes musical sense to me. It also requires being extremely present, in the moment, and really listening to the bigger picture and hearing all of the parts, because as I’m playing, I'm mixing it all live in real time.

Another aspect is that everything I play is in real time, nothing is pre-recorded and all of it begins with an acoustic analog sound. I don’t play tunes with my solo stuff, which really defines the music. I work in zones where I am improvising all of the time. Over the years, I have created a language for myself that I draw from, but am always changing it up and exploring different sonic ideas. What’s most difficult is getting out of my own way and trusting that if I am really listening and not thinking the musical path will present itself.

Finally, the biggest challenge is intonation, because with loopers, a single sour note can ruin an entire piece. So, for that, I practice simple scales and intervals with tuning drones on all of my instruments. And then depending on what instruments I'm inspired to play - alto or soprano sax, C or alto flute, alto clarinet or bass clarinet I may be improvising at any time in up to four different keys- concert, G, Eb, and Bb, which for me can get a little confusing. It’s easy for me to forget which key I started in and on which instrument, especially when using all the textures and effects that add a whole other dimension and complexity to the music and sound.

Do you have any current projects you’re working on?

Yes, I have a Solo binaural recording coming out titled Painting in Sound on Sunnyside Records this spring on May 15th, and I am in the process of choosing the music. Plus, I've just started livestreaming on my YouTube channel, https://www.youtube.com/@paintinginsound and hope to soon have a regular day and time for people to tune in live or watch at their convenience. And finally, once I get a better handle on all of this streaming technology, I hope to be inviting other players and artists to join me in this wonderful dimension.

And as this music gets out into the world, I've learned that most colleges and Universities now have Max/MSP and SPAT programs. I'm excited to share what I’m working on with both undergraduate and graduate students. This is a whole new frontier for creating music and sound, and, from my experience, one that may offer even a bit of medicinal properties that can really help people of all ages reduce stress, heal, sleep, and soothe physical and emotional issues. After all, music is magic, and binaural spatial audio and surround sounds expands our hearing perceptions.

The other project I’ve been working on for many years and wrote a book that Routledge published in 2024 on their Eye on Education Book series is Creative Sound Play for Young Learners, a method I created for early childhood educators PreK thru 3rd grade to use sound and silence –NOT music with their students that becomes a profound play-based educational tool that solves transition times, enhances active listening and executive function skills, develops social emotional learning, mindfulness, independence, agency, builds community and so much more. It’s straightforward, requires no additional skills, inspires teachers, helps them manage their classrooms better, reduces burnout, and is fun for all. Unfortunately, what I find is a world that is sound asleep because sound is missing from early literacy and educational conversations. I have never met a child who didn’t absolutely love and adore making any kind of sound. Have you?

About Hayes Greenfield

Hayes has been living, working, and performing as both sideman, session player and leader with his own bands in New York City. He has earned awards in all of his endeavors, has released 10 critically acclaimed CD's under his own name, and traveled throughout the US, Canada, and Europe performing at international festivals, concerts, and clubs with his own bands. Notable jazz artists Hayes has worked with as leader and / or sideman include: Jaki Byard, Rashied Ali, Paul Bley, Richie Havens, Danny “Big Black” Ray, Adam Nussbaum, Jay Anderson, Hiram Bullock, Barry Altschul, Bob Cranshaw, Dave Liebman, Bob Stewart, Mike Clark, Vincent Chancy, Jorge Sylvester, Billy Byers, Tony Scott, Joe Lee Wilson, Dean Johnson, David Berkman, Victor Jones, Tony Reedus, Tom Harrell, Ray Drummond, Norman Simmons, Paul Socolow, Leroy Williams, Frank Lacey, Leonard Gaskin, Jerome Harris, Denis Charles, Bill Ware, Frank Kimbrough, Uri Caine, Victor Jones, Roger Rosenberg, Myles Griffith, Tony Moreno, Rob Garcia, Dave Hofstra, Sunny Jain, Todd Turkisher, and many others.