A Survey Of Early Stereophonic System Subjective Evaluation

DarqueKnight

Introduction

Subjective non-blind evaluation methods, based on listener training, careful listening, and documentation, were preferred at the inception of stereophonic sound by its inventor, Dr. Harvey Fletcher, and by other scientists at Bell Telephone Laboratories, the General Electric Corporation (GE), the Radio Corporation of America (RCA), the British Broadcasting Corporation (BBC) and by scientists at many other reputable research organizations who worked on improvements to stereophonic sound equipment.

This article provides an overview of the subjective stereophonic sound evaluation methods from the 1930's to the 1960's. The term "stereophonic" did not enter the literature until it was introduced in 1927 by Bell Laboratories (the research and development division of the Western Electric Corporation). Up to then, multichannel audio systems were known as "auditory perspective" systems. After 1927, the terms "auditory perspective" and "stereophonic" were used interchangeably in the scientific literature. "Stereophonic", or "stereo" for short, can correctly be applied to any multichannel audio system using two or more speakers where the intent is to generate a three dimensional sound stage.

The basic definition of stereophonic is: "of or having to do with a system of sound reproduction in which the sound reaching each of two or more microphones placed apart is reproduced by one of a corresponding number of loudspeakers, also placed apart, giving an effect of depth and direction much like that of the original sound." [1]

The word stereophonic is derived from the Greek words "stereos" meaning "hard" or "solid" and "phone" meaning "sound" or "voice". Literally, a stereophonic system is one which creates an illusion of "solid sound".

Historical Perspective

The first documented experiment in multichannel audio systems was conducted in 1881 by Clement Adler. Adam Blumlein invented and patented a two channel (binaural) audio system using headphones in 1933. Dr. Harvey Fletcher, a contemporary of Blumlein, led the research team at Bell Laboratories that developed the modern concept of the "home stereo" system. The original prototypes were three speaker systems. This was revised down to a two channel, two loudspeaker system that was less cumbersome, and therefore more acceptable, to consumers. This work lead to Dr. Fletcher being honored as "the father of stereophonic sound".

In a technical paper presented at the Winter Convention of the American Institute of Electrical Engineers* in New York City in 1934 (January 23-26), Dr. Fletcher stated:

"This symposium describes principles and apparatus involved in the reproduction of music in large halls, the reproduction being of such a character that may give even greater emotional thrills to music lovers than those experienced from the original music. This statement is based upon the testimony of those who have heard some of the few concerts reproduced by the apparatus which will be described in the papers of this symposium. [Emphasis mine.]

The most intense peaks in music come in the range between 200 and 1000 cycles per second. Taking an average for this range it may be seen that there is approximately a 100-dB range in intensity for the music, provided about 10 dB is allowed for the masking of sound in the concert hall even when the audience is quietest.

The music from the largest orchestra utilizes only 70 dB of this range when it plays in a concert hall of usual size.

For halls like the Academy of Music in Philadelphia and Carnegie Hall in New York City,...,the power of the sound source is approximately 400 watts.

A person would experience the sense of feeling when closer than about 10 meters to such a source of 400 watts power, even in free open space. [An emphasis on the word "feeling" appears in the original text.]

These then, are the general fundamental requirements for an ideal transmission [playback] system. How near they can be realized with apparatus that we now know how to build will be discussed in the papers included in this symposium."[2]

J. Moir and J. A. Leslie, in a paper presented to the British Institution of Radio Engineers in London in 1951 stated:

"In a stereo system the acoustic characteristics of the studio can be faithfully reproduced and the long tunnel effect [of monaural systems] disappear. At first sight this and the concomitant advantage of being able to "place" the artiste or instrumentalist do not appear to be great advantages, but in practice they do produce a most remarkable improvement in the acoustic illusion.

A somewhat unsuspected result is the increase in clarity due to the spatial separation of the artistes, a result that is probably connected with the ear's ability to disregard sounds that approach the listener from directions other than than in which he desires to listen."[3]

In 1941, Dr. Fletcher published a paper in the Proceedings of the Institute of Radio Engineers* which stated:

"This paper gives the requirements for ideal systems for the transmission of speech and music."

and

"During the past two years a survey of the hearing capability of persons in a typical population has been made by the Bell Telephone Laboratories. This was done in connection with the exhibits at the World's Fairs at San Francisco and New York City, sponsored by the Bell Telephone Companies. At these exhibits records of the hearing of more than one half million persons were analyzed.[4]

Some interesting results from experiments in the reproduction of stereophonic depth and width were reported by Steinberg and Snow of Bell Telephone Laboratories. Using a subjective listening test methodology with 12 subjects, they found that a three speaker system provided superior depth and a two speaker system provided superior width.[5]

The term "sound stage" was introduced into the technical literature in 1959 in a paper presented to the I.E.E.** Convention on Stereophonic Sound Recording, Reproduction and Broadcasting by British Broadcasting Corporation engineer T. Somerville:

"Sound Stage-It is proposed to use this term to describe the region between the loudspeakers in which the stereophonic images appear. The term "sound field" is deprecated because "field" has other connotations."[6]

Two Bell Telephone Laboratories scientists, F. K. Harvey and M. R. Schroeder, presented a paper at the 12th Annual Convention the the Audio Engineering Society on October 11, 1960 with abstract as follows:

"In transmitting and reproducing two-channel stereophonic signals, the original program material may be modified deliberately or unintentionally. Separation upper and lower cutoff frequencies as well as full-band channel separation (in dB) have been evaluated subjectively in terms of detection of spatial difference, preservation of spatial resemblance, and listener preference. In addition, other pertinent observations on subjective aspects are reported."[7]

In their discussion of their test procedure, Harvey and Schroeder stated:

"Critical listeners were sought in these tests because of a desire to set permanent standards. At the moment, only a small percentage of people fully appreciate high fidelity. Even less appreciate or understand stereo. However, there is a growing sophistication evidenced among users of stereo equipment. Anticipating the future, it seemed wise to avoid naive or unconcerned personnel in these tests to prevent establishing loose standards which eventually might have to be abandoned.

The listeners chosen were sophisticated in the art of sound localization either by working in this field or by education before testing. They were felt to be the equal of any serious listener who is accustomed to playing the same records many times and thus becomes familiar with the more subtle artistic and technical effects." [8]

Harwood B. Moore, a research scientist with General Electric, published a paper in 1960 which concluded:

"Subjective listening tests have been completed which indicate that stereo, in any of the forms compared, is preferred over monaural, but normal stereo from two full-range speakers well physically separated is, in all probability, the most preferred.

Listeners were selected from draftsmen, engineers, technicians, and secretaries. Only one listener was permitted in the testing area at a time. Each test required 15 decisions because it included six systems compared two at a time.

The listeners were educated to the extent that the only significant changes which should be judged should concern the reproduced sound panorama." [8]

____________________________________________
*The A.I.E.E. and the I.R.E. merged in 1963 to become the Institute of Electrical and Electronics Engineers (I.E.E.E.), the world's largest technical society.
** British Institute of Electrical Engineers (not affiliated with the American I.E.E.E.)

DarqueKnight

K. Schjonneberg and F. Olson published a mathematically rigorous evaluation of various subjective listening test methods used to rank consumer audio products:

"Comparative listening tests to make subjective evaluations of the acoustic performance of radios, phonographs, and high-fidelity equipment are very important to successful design and development efforts. This paper describes appraisal techniques and statistical treatment of listening test results.

Test panels of judges often are used to rank consumer products where personal opinion is a factor.

A problem of bias exists in all ranking tests. The subject material used in the test, position of the set, and the order in which the sets are tested are only three examples.

Every effort must be made to remove any form of bias from the test. Visual prejudice may be removed by a curtain if required." [9]

Out of the 21 papers I read, the Schjonneberg and Olson paper was the only one that mentioned the use of some type of blinding technique (a curtain), although they did not use any curtains in their analysis nor did they provide any insight regarding the specifics of curtain use (e.g. between subject and entire system or between subject and a specific piece of equipment. The focus of Schjonneberg and Olsen was somewhat different from the other subjective analysis papers I read in that they were concerned with bias influencing the ranking of consumer audio products rather than bias solely influencing the perception of sonic performance of those products.

Beaubien and Moore published a paper in April of 1960 in the Journal of the Audio Engineering Society which discussed results on perception of the stereophonic effect as a function of frequency:

"A literature study and listening tests have been conducted to contribute to an understanding of the stereophonic effect as a function of frequency.

Technical and nontechnical listeners were used. Half of the nontechnical listeners were women. The need for concentration by the observers was considered important enough for us to bar kibitzers [non participating people] from the area.

Medical hearing tests (125 cycles to 12,000 cycles) were conducted on most of the listeners but no correlation existed between hearing ability and listening test results.

All listeners were educated as to what they were comparing and told that low (or high) frequency instruments would seem to be further to the outside under full stereo conditions." [12]


Dr. Fletcher, the "father of stereophonic sound", said that the target market for stereophonic sound systems were "music lovers". The evaluation methodologies used by the eminent scientists during the invention and early development of home stereophonic sound systems were specifically non-blind and were based on scientific and mathematically rigorous subjective techniques. Scores of these early papers on stereophonic sound system evaluation, which were published in the peer-reviewed scientific literature over three decades, emphasized the following performance metrics and evaluative concepts:

1. Optimization of sound stage width.
2. Optimization of sound stage depth.
3. Stable stereo image placement.
4. Clarity.
5. Detail.
6. Dynamics (dynamic range).
7. Tactile impact (feeling!)
8. Emotional thrills.
9. Hearing acuity of listeners.
10. Listener training.
11. SONIC REALISM.

Conclusion

Subjective evaluation methods of stereophonic sound systems are thought by some misinformed individuals to be "unscientific". A paper presented by David Clark**** in 1981 to the 69th Convention of the Audio Engineering Society in Los Angeles stated:

"Listening tests used to evaluate audio equipment can seldom be considered scientific tests". [14]

One can only wonder why Mr. Clark, and others of his mindset, ignored decades of scientific results which indicated that subjective listening tests were optimal methods for evaluating the sonic performance of stereophonic sound systems. When we go to the peer-reviewed scientific literature, we find that the work of respected scientists at Bell Laboratories [2], [4], [5] [7], the British Broadcasting Corporation [6], the General Electric Corporation [8], [9], [11], [12] and the Radio Corporation of America [10] demonstrated that properly applied subjective evaluation methods were preferred to forced choice A/B or A/B/X methods. The papers listed in the reference section of this article are a representative sample of the evaluative analysis papers published from the 1930's to the early 1960's. The reference sections of those papers point to many other papers on the topic.

It is also commonly mistaken that the basic subjective evaluative methods and audiophile terminology for stereophonic systems originated in the audiophile press. The term "audio-phile" did originate in the audiophile press and it first appeared in the first issue of High Fidelity magazine in September of 1951. The phrase "Devoted to the Interests of Audio-philes" appeared at the bottom of the magazine cover.[13] Terms such as "stereophonic", "sound stage" and "stereo image" first appeared in the scientific literature published by scientists working in the field. Furthermore, descriptive terms such as "depth", "width", "clarity", "three-dimensional" and "detail" as applied to the sound images and sound stage proportions generated by stereophonic systems were first defined and used in the scientific literature and then adopted by the audiophile press.

The terminology and listening evaluation methodologies used by audiophiles and audiophile press reviewers are very similar to, and in many cases identical to, those used by the scientists who invented and participated in decades of early development of stereophonic sound systems.

References

[1] World Book Dictionary, 1973 ed. Vol. 2., p. 2033.
[2] Fletcher, Harvey, "Symposium on Wire Transmission of Symphonic Music and Its Reproduction in Auditory Perspective-Basic Requirements", Bell System Technical Journal, Vol. 13, 1934, pp. 239-244.
[3] Moor, J. and Leslie, J. A., "The Stereophonic Reproduction of Speech and Music", Journal of the British Institution of Radio Engineers, London, September 1951, pp. 360-366.
[4] Fletcher, Harvey, "Hearing, The Determining Factor for High-Fidelity Transmission", Proceedings of the I.R.E., Columbus, OH, June 1942, pp. 266-277.
[5] Steinberg, J. C., and Snow, W. B., "Symposium on Wire Transmission of Symphonic Music and Its Reproduction in Auditory Perspective-Physical Factors", Bell System Technical Journal, Vol. 13, 1934, pp. 245-258.
[6] Somerville, T., "Survey of Stereophony", Proceedings of the Institution of Electrical Engineers, Convention on Stereophonic Sound Recording, Reproduction and Broadcasting, London, March 1959, pp. 201-208.
[7] Harvey, F. K. and Schroeder, M. R., "Subjective Evaluation of Factors Affecting Two-Channel Stereophony", Journal of The Audio Engineering Society, Vol. 9, No. 1, January 1961, pp. 19-28.
[8] Moore, H. B., "Listener Ratings of Stereophonic Systems", IRE Transactions on Audio, September-October 1960.
[9] Schjonneberg, K. and Olson, F., "Listening Test Methods and Evaluation", Journal of the Audio Engineering Society, Vol. 9, No. 1, January 1961, pp. 29-36.
[10] McCoy, D., "Distortion of Auditory Perspective Produced by Interchannel Mixing at High and Low Audio Frequencies", Journal of the Audio Engineering Society, Vol. 9, No. 1, January 1961, pp. 13-18.
[11] Moore, H., "Effect of System Parameters on the Stereophonic Effect", Journal of the Audio Engineering Society, Vol. 9, No. 1, January 1961, pp. 7-12.
[12] Beaubein, W. H. and Moore, H. B., "Perception of Stereophonic Effect as a Function of Frequency", Journal of the Audio Engineering Society, Vol. 8, No. 2, April 1960, pp. 76-86.
[13] Fowler, Charles, "As The Editor Sees It", High Fidelity Magazine, Vol. 1, No. 1, September 1951, p. 8 and cover.
[14] Clark, D., "High-Resolution Subjective Testing Using a Double-Blind Comparator", Journal of the Audio Engineering Society, Vol. 30, No. 5, May 1982, pp. 330-338.

_______________________________________________
****Mr. Clark later went on to facilitate the famous Stereo Review article published in January of 1987 which found no audible difference in amplifiers. The test used ABX methodology and indicated that there was no audible difference between a $200 consumer grade transistor receiver and a $12,000 premium quality tube amplifier. Mr. Clark concluded: "But for now, the evidence would seem to suggest that distinctive amplifier sounds, if they exist at all, are so minute that they form a poor basis for choosing one amplifier over another. Certainly there are still differences between amps, but we are unlikely to hear them."

Mr. Clark and his associates formed the ABX Company to market ABX comparator devices. The company was in business from 1982 to 1987 and ceased operation shortly after the appearance of the Stereo Review article. I do not know if this was coincidence or if the absurd results in the article was a contributing factor. Mr. Clark, in a paper entitled "Ten Years Of A/B/X Testing", (presented at the 91st Audio Engineering Society Convention, New York, NY, October 1991), only mentioned that:

"Ten years ago [in 1982] the present author presented a paper to the AES on double-blind testing using the A/B/X technique. For the next five years, a device to conveniently implement this test was commercially available. It was thought by the author and his associates that general use of this system would resolve "The Great Debate" of whether or not small differences in audio components were audible."

audiobilly

Good Reading DK. Thanks for posting this.

markmarc

Absolutely fascinating stuff DK.