Well-designed and executed blind listening tests have shown that people generally prefer loudspeakers whose anechoic measurements reveal a flat on-axis frequency response and similarly smooth off-axis response. However, using such speakers does not ensure accurate sound reproduction in many rooms, even with careful speaker placement and acoustical room treatment.

Since the advent of electromechanical sound systems, researchers have sought sound-system performance measurement techniques that correlate with aural perception. Progress abounds, but we are not there yet. Aural evaluation remains the final arbiter of reproduction accuracy. This is a discussion about sound-reproduction accuracy, not preference. For this paper, accuracy refers to sounding as close as possible to the same sound in real life, whereas preference refers to what the listener might like, want, or expect to hear, which might differ from accurate reproduction. Although this disquisition focuses on movie playback, the concepts also can be applied to evaluation and calibration of music playback systems, whether in the home or recording studio.

I believe that effective calibration of sound systems for movies and for non-movie events, including music, is critical to creating a sonic palette that can accurately reproduce, or for live events produce, the source signal for an audience of one or many, whether in one venue or in many (such as the hundreds of locations worldwide that simultaneously play the Metropolitan Opera live HD theatercasts).

Currently that seems to require an objective, measurement-based methodology plus far too much adjustment based on aural (subjective) evaluation of the sound system’s performance.

I suggest that the best possible solution would be an objective, measurement-based methodology that closely correlates with aural perception, thus reducing the amount of aural-evaluation-based adjustment required. There are groups in more than one professional organization working toward that goal for large-venue sound systems. To realize the goal, evaluation of measurement-based methodologies requires aural evaluation of how well those objective approaches match what we hear from good and from flawed sound systems:

• To perform meaningful aural evaluation requires source content that can be used by listeners to fairly evaluate the accuracy of its reproduction, as opposed to listener preference. I maintain that in general, movie soundtracks provide a more reliable source of such content than other types of material.

• In addition, the evaluating listeners need to be able to judge the accuracy of that reproduction, not their preferences, separate from the idiosyncrasies of the sound system on which the content might have been created.

Movie Soundtracks As Preferred Evaluation Content
I consider the reference for accurate sound reproduction to be life’s experience. A major objective of movie/TV soundtrack designers/mixers is to record and create dialog and many (not all) sounds that when properly played back mimic real-life experiences — sounds that the audience instantly recognizes and accepts as real, transporting the listener to that environment. Thus, what we hear, not measure, would be the ultimate judgment of their success.

In certain commercial movie theaters and in my home theater, almost all movies from different studios and different soundtrack designers/mixers sound quite natural — dialog, music, and many of the sound effects are inherently recognized as accurate representations of what is heard, and is expected to be heard, when the action that the sound is associated with occurs in life. That includes voices, acoustic musical instruments, walking on various surfaces, chains jangling, rain, vehicle noises, and so many others (however, thankfully, not nuclear explosions or plane crashes).

In other commercial movie theaters, some dubbing stages, and over many other home playback systems, movie soundtracks from various studios don’t sound natural, forcing the listener to work to correlate the sound with what is expected from the same action in real — not reel — life. Sometimes those sounds aren’t even ‘heard’ because they don't sound like anything seen in the scene (yet when the same scene is played over one of the sound systems mentioned in the previous paragraph, that sound is instantly heard and recognized). I first became aware of this situation in the early 1990s.

It is known that the sound quality from a number (not all) of dubbing stage systems used by soundtrack designers/mixers varies. Therefore the described experiences demonstrate that those professionals can inherently, whether consciously or not, make outstandingly accurate, natural-sounding and consistently excellent soundtracks, even when working with less than perfect dubbing-stage sound systems. If this was not the case, no given commercial movie-theater or home sound system could play back soundtracks that sound natural from any but a very limited subset of studios and soundtrack designers/mixers.

Evaluating a sound system's reproduction accuracy requires known source material and a reference. Because movie soundtracks exhibit such consistent sound quality across many studios and soundtrack designers/mixers, and because there are commercial movie theater and residential sound systems that deliver accurate natural-sounding playback, I believe that the most reliable valid reference for reproduction accuracy (again, not preference) is real-life sound, and that the primary content that should be used for aural evaluation of sound system accuracy is the movie soundtrack.

For most of the past decade, at CEDIA Expos Frederick J. Ampel (Technology Visions Analytics) has been instructing his course “Learn To Listen”, during which he demonstrates how to use movie soundtracks to analyze and adjust the accuracy of home theater sound-system playback. At CEDIA Expo 2012 I co-instructed an updated version of that course. At CEDIA Expo 2013 (25-28 September 2013 in Denver, CO) Ampel and I will be jointly instructing an expanded and further updated version.

Potential Aural Evaluator Issues
Many people who try to evaluate and adjust sound systems have developed subconscious biases that interfere with making impartial aural evaluations of playback accuracy.

One example: I have learned from multiple sources that there have been instances where a lower-distortion midrange driver has replaced a similar-sounding driver in dubbing-stage sound systems, and some sound designers/mixers have expressed initial difficulty accepting the new lower-distortion speaker as providing a more accurate sound, sometimes even to the point of requiring electronically generated distortion that simulates the distortion in the older driver.

This and other anecdotes lead me to conclude that real life is not their sonic reference, that in their minds they inherently have 'adjusted' the real-life sounds they are trying to create to accommodate the colorations in the dubbing-stage sound systems they use to make those soundtracks. In other words, to make a natural sound, they have had to make it sound unnatural in the way that it is colored by the dubbing-stage sound system they are using. This is not a flaw or a failing, but is an exceptional talent and skill for which they should be lauded. Thus they judge the sound of any other sound system against the sound they hear from their dubbing-stage system, using their reference system as being ‘right’. However, that is why I consider it questionable whether some soundtrack designers/mixers can rate the accuracy (not preferability) of a sound system.

I am sure that there are individuals who can judge accuracy, but they need to be identified to ensure that their judgment is of a sound’s reality, not its adjusted quality. I believe the same applies in the evaluation of home music systems.

Another example: For a similar reason, it is not possible to ask most musicians to evaluate the accuracy of music reproduction by a sound system. Musicians are always positioned to hear acoustic instruments from angles and distances never recorded or familiar to the audience.

Most orchestra and chamber group concerts do not use amplified reinforcement — the audience hears the natural sound of the acoustic instruments.

Conductors of those groups are as close as you can get to performing musicians who might be able to evaluate musical reproduction accuracy, as they are in front of the performers, although rather close to them. However, many of them, including some famous conductors and soloists, have clearly demonstrated that they listen to the performance, not the sound quality — negating their ability to judge a sound system’s accuracy. Assistant conductors often serve as the conductor’s ears during rehearsals by being in the audience area and feeding suggestions to the conductor (as with many soloists, pianist Leon Fleisher's wife has served in that role for him). These individuals and music concert critics (not recording critics) could conceivably serve as accuracy judges, as they hear unamplified vocal and acoustic-instrumental performances from the audience's perspective, just as moviegoers who are the audience for movie soundtracks. [Typically they are listening to performance and balance, not tonal accuracy too much. They are used to a wide range of idiosyncratic and individual frequency responses and radiation patters with a range of instruments of the same type. DRM]

Perhaps The Best Aural Evaluators
Individuals who in their personal, not professional, lives have a heightened dependency on their hearing might have the highest sensitivity to the sounds of life, and likely will more readily recognize reproduction inaccuracy.

Restating The Objective
Scientifically based subjective analysis of sound-system quality (aural perception; perhaps similar to Floyd Toole and Sean Olive’s listening-test methodology — see his Audio Engineering Society papers) needs to be integrated with objective technology-based measurements in as scientific a manner as possible in order to develop a meaningful measurement-based calibration technique that can differentiate between good and bad sound-system performance, that can help identify specific problems, and that can assist in efforts to make the sound system deliver high sonic quality consistently through multiple calibrations in a single dubbing stage, in commercial movie theaters, and among different and various kinds of facilities for movies and for non-movie events, including home sound systems.

Discussion is expected, and will be welcome.

Enjoy the Music.com highly encourages our readers to join the Boston Audio Society by clicking here). 

This article is copyrighted © by the author or the Boston Audio Society. It is posted on Enjoy the Music.com with their permission, and with all rights reserved.