Notice Of Sensory Science Journal Publication

jinjuku

cstmar01 wrote: »

and DK I look forward to the paper. I'm not really sure why we have to argue the merits of what someone has done and try to disprove it when no one has read the paper yet, nor show some respect that DK was able to publish the paper in a journal with peer reviews. Not sure what other people's creditably is related to the topic or related degrees, nor do I think most people really give a crap about it. I am just happy to see the man got the paper published and that he is a member on the forum offering up his view point.

Thank you for a sane post in this thread.

From what I have read unc2701 is asking about the testing, measurement, and N. Nothing wrong with that, others are going to ask the same.

I look forward to the release of his work, to see what he has to say, and to see the flip side of the coin.

Folks, he is publishing a paper about a protocol. There could be many protocols out there in the audio world for critical listening. He is taking a position with ABX testing as an antithesis (If I read his abstract correctly) I am sure that view point will be looked at by other engineering giants in the industry. I wait with curiosity until then.

P.S. just because someone publishes a paper doesn't make it gospel. Just like "I read it on the internet".

schwarcw

Congrats Raife on your publication! I've always admired your interest in the science of the audio hobby.

Me, I don't like to think too hard when I'm listening to music. My son took critical listening courses in college when he was studying to become a recording engineer. He can hear things in music that I can never hear. He can almost instantly recognize when the recording used some "auto tuning" software to put the singer's voice in tune. Am I the only one who can't hear that?

madmax

schwarcw wrote: »

He can almost instantly recognize when the recording used some "auto tuning" software to put the singer's voice in tune. Am I the only one who can't hear that?

Yep.

There is so much crap that happens today with digital post processing that it ruins the recording for those of us who really love a great recording. For some the term "great recording" means fixing every timing glitch and tune of everything but for me "great recording" means it holds all the tone of the original performance. Perfect recordings are ****.
madmax

analog97

He can almost instantly recognize when the recording used some "auto tuning" software to put the singer's voice in tune. Am I the only one who can't hear that?

I didn't even know this was technique "auto tuning" was possible. Maybe this is an example of an audible "sound" that requires some experience to detect and you simply have not heard it enough to be able to discern as easily as a a "young-eared" one. I think it likely that the ability to detect certain audible changes may benefit from what is commonly referred to as the "practice effect", i.e. the more you do something, the better you become.

My hope for DK's work is that a simple method is available for reliable discerning more subtle changes in a 2-channel system. Subtle is the key word here for me. I am not talking about "Darqueknight and day" differences, e.g. the ones we have shared together, like your tube pre-amp addition and the proper gain for the LO MC cartridge offered by the Piccolo. Those are MAJOR and to me, do not require anything more than a 2-second listen. Once one has a good audio playback system in place (yes, like YOURS), determining the additional benefit from , say a cable change or a different cartridge, tonearm, TT or simple tube-swapping is what I am most intrigued by. I am hopeful that DK's methodology can reliably do that for me. As I've said before on this Forum, to my ears and brain, the time-lag between component changes is my biggest hurdle to my being able to truly convince myself that an incremental improvement was acieved. Again, I am hopeful that DK's methodology will work for me. PS: Just got Morph the Cat....awesome on vinyl!!

hearingimpared

analog97 wrote: »

I didn't even know this was technique "auto tuning" was possible. Maybe this is an example of an audible "sound" that requires some experience to detect and you simply have not heard it enough to be able to discern as easily as a a "young-eared" one. I think it likely that the ability to detect certain audible changes may benefit from what is commonly referred to as the "practice effect", i.e. the more you do something, the better you become.

My hope for DK's work is that a simple method is available for reliable discerning more subtle changes in a 2-channel system. Subtle is the key word here for me. I am not talking about "Darqueknight and day" differences, e.g. the ones we have shared together, like your tube pre-amp addition and the proper gain for the LO MC cartridge offered by the Piccolo. Those are MAJOR and to me, do not require anything more than a 2-second listen. Once one has a good audio playback system in place (yes, like YOURS), determining the additional benefit from , say a cable change or a different cartridge, tonearm, TT or simple tube-swapping is what I am most intrigued by. I am hopeful that DK's methodology can reliably do that for me. As I've said before on this Forum, to my ears and brain, the time-lag between component changes is my biggest hurdle to my being able to truly convince myself that an incremental improvement was acieved. Again, I am hopeful that DK's methodology will work for me. PS: Just got Morph the Cat....awesome on vinyl!!

Larry, although I have a pretty good aural memory, I take notes in a certain copy book(s) while A/B'ing anything and everything and compare them to each other. By keeping these notes in that book, I can go back over notes from years before to compare what I've heard on differnent gear especially when I want to compare music performances. It's great to be able to see how a piece of music played on a piece of gear which has been replaced by a better or different piece of gear compare. It also enables me to see how, if at all, how my hearing is changing for better or worse.

nduitch

Wouldn't a better method be listening to you gear and then going and listening to a live performance? Rather than just comparing things that aren't the "real thing" anyways?

analog97

I, too think that notes can help. That said, written words, despite the highest degree of objectivity possible, to me remain subjective and insufficient to convince me that a subtle positive/negative change has occurred. To me, the only A/B comparison that has occurred rapidly in my system, and hence most easily evaluable, was the "on-the-fly" switching of cartridge loading. I fact, that's the only rapid A/B evaluation I know of. That's why A/B suffers so much, in that it is nearly impossible to rapidly change the vast majority of components. When a switch-box is put in the loop, all bets are off as that introduces multiple variables. The methodology to be proposed by DK hopefully will shed some light on these problems and offer a reasoned, "middle of the road" approach between dusting off 4-yr old notes and the "best" (to me) example I referred to on cartridge loading. Thanks for your input and experience, Joe.

heiney9

I'm going to post this here as well as the other place I posted it. Pay special attention to page 5 the lower middle column and upper right column. A pretty succinct and accurate statement in my eyes. This interview continues to talk about circuit differences in amps and why they sound different, etc.

www.linearaudio.nl/JDpubs/Interviews/DiddenInterview_MMM110.pdf

DarqueKnight

analog97 wrote: »

My hope for DK's work is that a simple method is available for reliable discerning more subtle changes in a 2-channel system. Subtle is the key word here for me.

Subtle is the key word for many of the performance differences among audio gear. Discerning such subtle differences requires training, careful listening and careful documentation of the sound stage attributes. Hence:

DarqueKnight wrote: »

"... the protocol proposed in this paper is administered under realistic, non-stressful conditions and it maximizes and optimizes the amount of aural and tactile information received by the subject."

nduitch wrote: »

Wouldn't a better method be listening to you gear and then going and listening to a live performance? Rather than just comparing things that aren't the "real thing" anyways?

In my opinion, a better method would be first listening to many live musical performances, building mental profiles of the aural and tactile stimuli from that, and then comparing stimuli derived from electro-mechanically reproduced music to the stimuli derived from real music.

analog97 wrote: »

I, too think that notes can help. That said, written words, despite the highest degree of objectivity possible, to me remain subjective and insufficient to convince me that a subtle positive/negative change has occurred. To me, the only A/B comparison that has occurred rapidly in my system, and hence most easily evaluable, was the "on-the-fly" switching of cartridge loading.

Short term memory has been shown to be very unreliable. Written notes, accompanied by sketches of sound stage image locations, allow "getting to know" the component under evaluation and are very reliable if they are taken over an adequate length of time. The accuracy of written notes is enhanced when a standard methodology and terminology for describing what is heard is used.

When I am doing critical evaluations, I do not concentrate on trying to hear differences. I concentrate on trying to accurately document the location, sizes and attributes of the sounds within the sound stage. Often, I do not become aware of (often subtle) differences until I compare the lateral and aerial sound stage maps between two pieces of gear. For example, the only difference between two amps may be subtle shifts in the locations of sound stage images. Trying to pick up on that difference, while your ears are being bombarded by other, more forceful stimuli, might be difficult. If you carefully and patiently go through the sound stage (side to side, front to back, and top to bottom) and map (document) everything that is there, you are less likely to miss something.

Jetmaker737

DarqueKnight wrote: »

The accuracy of written notes is enhanced when a standard methodology and terminology for describing what is heard is used

I'd go a step further and say that standard terminology for describing what is heard is absolutely essential. I think a good analog are food, wine, scotch, and cigar connoisseurs. They have a mutually understood lexicon of unique terms to describe subtle differences in flavor, texture, and aroma that the lay person wouldn't even detect.

So DK, in your method do you have descriptive terms pre-defined and then train the users to apply them?

DarqueKnight

Jetmaker737 wrote: »

So DK, in your method do you have descriptive terms pre-defined and then train the users to apply them?

Yes! This is one of the core concepts of descriptive analysis. The paper provides a lexicon of descriptive terms applicable to audio evaluation.

DarqueKnight

Introduction

The field of sensory science has unfortunately been "typecast" as only being relevant to the field of food science. Food scientists do use sensory science method, but sensory science is not a sub-discipline within the field of food science. The following definitions may prove instructive:

Definitions:

Food Science: Food science draws from many disciplines such as biology, chemical engineering, and biochemistry in an attempt to better understand food processes and ultimately improve food products for the general public. As the stewards of the field, food scientists study the physical, microbiological, and chemical makeup of food. [Source: Institute of Food Technology Website.]

Sensory Science: Sensory science is the study of the reactions of the five senses - sight, hearing, smell, taste and touch - to the characteristics of physical matter. The discipline does not just deal with "likes and dislikes," but scientifically evokes, measures, analyzes and interprets psychological responses to physical stimuli, and thus belongs to the specialized field of psychophysics. [Source: Central Food Technology Research Institute Website.]

Sensory Food Science: "Sensory food science," adopts a unique approach to identification of the attributes that matter most to the end-consumer. It employs trained "panelists," and where possible sensitive instruments, to provide the right guidance to the food manufacturers on what perceived attributes need to be incorporated into their products for their marketing success. [Source: Central Food Technology Research Institute Website.]

Case Study

This case study illustrates how misunderstood the field of sensory science is in the minds of some individuals and how such misunderstanding leads to an unfortunate resistance to the acceptance of sensory science methods in fields outside food science.

The following two quotes refer to the article content of the Journal of Sensory Studies.

unc2701 wrote: »

That’s 29 food articles, and 3 about something else. But maybe it’s not the food articles that are really outstanding for them. Maybe their high profile articles have a different profile? Let’s check their top five… Food, Statistics, Food, Food, Food.

The Journal of Sensory Studies (JOSS) does not publish articles on food, it publishes articles on the sensory analysis of food. They also publish articles on the sensory analysis of everything else, from paint to audio equipment to medicine to perfume. It just so happens that the majority of people submitting articles to the journal are conducting research in food related areas. Common sense should inform that, were the Journal of Sensory Studies a food science journal, they would have never accepted a paper entitled "A Descriptive Evaluation Methodology For Consumer Audio Equipment". Moreover, editor-in-chief of a food science journal would never make the following comment:

"We look forward to receiving further articles from you, especially if they contain sound sensory data that will push the sound research field to greater use of appropriate sensory methods."

unc2701 wrote: »

Regarding your hypothetical list, I wasn’t aware that ISI used the titles of the journals for grouping them… seems like they’d use something more reliable, such as the actual content, no? Speaking of content, let’s check out the last few issues of JOSS shall we? I’m just gonna throw the articles into some broad categories. Food, beverage, taste, and methodology to evaluate ‘em, I’m just going to call “Food”; anything else I’ll note.

Haddock BALLS??? Wow. I’ll put this under food, I guess.

The person making the above statement mentions looking at actual content, but he failed to do so. Let's look at actual content. Regarding the "Haddock Balls" article, which is in the current June 2010 issue of JOSS. The full title of the article is:

"SENSORY ATTRIBUTES OF HADDOCK BALLS AFFECTED BY ADDED FISH PROTEIN ISOLATE AND FROZEN STORAGE "

The abstract of this article states:

"Fish protein isolate (FPI) made from haddock (Melanogrammus aeglefinus) cut-offs by the pH-shift process was added to haddock mince to make two groups of fried fish balls. The proportions (%) of mince to isolate were 100:0 (control group), 75:25 and 50:50. All groups were air packed and kept frozen at −18C. The sample groups were evaluated by sensory evaluation 1 day after processing and after 2, 4 and 8 weeks of storage at −18C. The results indicated that added FPI to mince and frozen storage affected the odor, flavor, texture and appearance of fish balls significantly, possibly because of chemical and biochemical changes of all groups. This study also revealed that most negative features are attributed to the groups containing 50% mince and 50% isolate. The results can be considered for product development of FPI."

When we look at the article content, we see that the authors are concerned with the sensory attributes (odor, flavor, texture, and appearance) of haddock balls. This is an article on the sensory analysis of a food item.

Since JOSS is a journal which concerns human sense reactions, it would be more appropriate to group articles in the broad categories of sight, sound, touch, smell and taste rather than the "broad" categories of food, beverage, taste, and methodology. With that in mind, the current June 2010 issue of JOSS features nine articles grouped as follows:

1. Taste, touch, sight, smell (fish).
2. Taste and smell (tainted air and water).
3. Taste (dairy products).
4. Taste (soy products).
5. Taste (apples and cheese).
6. Sound (auditory cues in food and drink).
7. Taste and touch (taste and texture of yogurt).
8. Taste and touch (taste and texture of whey protein).
9. Taste (bottled water).

The unifying thread through all nine articles is the application of sensory science evaluative methods.

unc2701 wrote: »

One of the criticisms of impact factor is that obscure fields can have inflated impact simply because there are a limited number of sources to cite, so the same ones are used heavily. So JOSS being grouped in a saturated category like Food Science would more likely be to its advantage since the food citations are spread over a larger number of journals.

The editorial board of JOSS informs that it is a journal which is concerned with the sensory aspects of anything. From JOSS's overview page on its website:

"Read the international journal that publishes human reactions to basic tastes on foods, beverages, wine, liquor/beer, the environment, medications, and other human exposures in every day life. This is the journal you’ll read for the latest information on experimental design methods and statistical analysis of data on human reactions."

JOSS publishes food related articles IF such articles concern sensory analysis of food. The vast majority of food science journal articles do not concern the sensory analysis of food. Therefore, it IS NOT to JOSS's advantage to be grouped with food science journals, because most of the authors writing in the field of food science would not be citing articles whose subject is the sensory analysis of food.

madmax

DarqueKnight wrote: »

Yes! This is one of the core concepts of descriptive analysis. The paper provides a lexicon of descriptive terms applicable to audio evaluation.

Are you aware of a glossary of terms other than the ones in your paper? I sadly make up my own terms when I am unaware of the proper one.
madmax

DarqueKnight

Robert Harley's "Complete Guide to High-End Audio", 3rd Ed., has a 25 page glossary.

madmax

Thanks, I see it on amazon.
madmax

DarqueKnight

The Harley book also has 8 pages of discussion on the setup and outcomes of a few blind listening tests. I am sure you will be shocked to find that those listening tests ended up with results similar to the two that I mention in this thread.;)

Jetmaker737

DarqueKnight wrote: »

The Harley book also has 8 pages of discussion on the setup and outcomes of a few blind listening tests. I am sure you will be shocked to find that those listening tests ended up with results similar to the two that I mention in this thread.;)

Thanks for the book recommendation! I just received it and I think I'll enjoy it very much. :cool:

doctorcilantro

Interesting point indeed.

Has anyone taken into account what type of music is used for evaluation? The Chesky discs hi-rez demo discs are usually pretty relaxed, well you could have some drum demos, and one track is an orchestral choir showcasing soundstage depth and imaging. We aren't sitting down in putting on pure tones or sine waves for example. By relaxed I mean there is usually some space in the music, it's well-recorded, and you can discern the performers and instruments with ease (hand claps, wood blocks, piano); but hey I guess you could listen to Helmet or Mach3 Rachmanioff too if that's what your familiar with.

When I sit down to listen, even focused listening sessions, many songs have quite a bit "going on", but I'm not trying to focus on everything at once, nor am I staring at just the bassline. You may be able to take in the sound all at once while not partitioning your attention to the various aspects of the reproduction, but I would guess that everyone's experience is highly subjective no matter how the track is mixed. My attention may get pulled towards a conglomerate of 3 or 4 things, as opposed to you being amazed at how the bass and drum lock in and how deep the bass is digging.

But maybe focusing our energy on aspects of the reproduction makes more sense, I'm just not sure this is what we do with casual or even focused listening sessions.

When I demoed a new DAC the the other day I was testing my wired connection to music server so I was using a large bitrate file @ 192kHz; I don't often listen to classical but this string solo amidst muted playing background of the orchestra was incredible on one DAC, and blurry on another, interesting and attention grabbing vs. boring. I was shocked at the timing of the same piece when comparing from DAC to DAC; one one I knew where the piece was and on another I just lost when that solo was ending. I'm a big proponent of the types of methodology put forward here, but I was just switching back and forth (same Lynx transport w/ 2 output cables so it was as easy moving the headphone jack) - and it was obvious. I wonder if I could ABX it though?

Given today's plethora of new breed DACs it would seem further study is in order.....

Just some thoughts....

madmax wrote: »

Early on I was mentally lazy as a listener, did not know how to concentrate on a particular area of sound and had no idea what to listen for. This made me search for a way to KNOW what was best without having to decide for myself. I was incapable of deciding for myself. Since I could not KNOW I started comparing sounds, essentially A-B tests. OK, I thought, now I KNOW which of two samples I think sound best. I would pick one or the other based on one or two variables. As I've progressed I now realize you can't focus on two or three variables out of thousands, which is what comparison listening forces you to do. You also cannot map thousands of variables at a single time in your mind. The only way to address the sounds of different equipment is to seperate the different qualities and map them. The mentally lazy will NOT do this. Neither will the person who needs an absolute answer on which is best. His mind will not allow it to remain a question so he will make an unqualified decision through any easy unqualified method such as A-B comparison or ABX etc. Who knows, maybe the decision will be made on how close to the speakers the dog wants to be. Whatever the process, the easier the better, a decision must be made by these types. In reality we must break out all of the qualities, understand them, listen to them and pick out what seems best. This journal entry does exactly that. It is not for the mentally lazy or someone in the mindset of having already made a decision. Strangely enough, what sounds best in a comparison is usually not what will make us happy in the long term. We tend to pick things based on whatever is most different between the two items being compared. This leaves us with nothing.
madmax

DarqueKnight

doctorcilantro wrote: »

I wonder if I could ABX it though?

You should try it and post a thread of your results.

DarqueKnight

doctorcilantro wrote: »

Has anyone taken into account what type of music is used for evaluation?

When I sit down to listen, even focused listening sessions, many songs have quite a bit "going on", but I'm not trying to focus on everything at once, nor am I staring at just the bassline.

As with any type of evaluative training, a person should start out with the simple and move toward the complex as their experience and proficiency grows.

There was a study conducted by some Audio Engineering Society scientists (Ford et. al., 2001*) who were investigating techniques for graphing the spatial attributes (image locations) of stereophonic music. They used three subjects who were second year music students at a university. Each student drew wildly different sketches of the type and location of sound stage image placements.

The test used:

1. Complex musical selections.
2. An "acoustically transparent" curtain between the subject and the speakers.
3. A short listening period per musical selection: 90 seconds.
4. Subjects who had some training in music, but no training in critical evaluation of stereophonic sound stage representation.

In the conclusion, the AES scientists wondered if the complexity of the music, lack of subject training and the curtain had some undue influence on the results.

---

*"Graphical Elicitation Techniques for Subjective Assessment of the Spatial Attributes of Loudspeaker Reproduction – A Pilot Investigation", Natanya Ford, Francis Rumsey and Bart de Bruyn, Institute of Sound Recording and Department of Psychology, University of Surrey, Guildford, UK, Proceedings of the 110th AES Convention, Amsterdam, The Netherlands, May 2001.

madmax

My system sounds different for a short amount of time when the vision of it changes. It only takes a few minutes before it is back to normal. Things like this do change your perception of the sound, at least initially. If you ever doubt different senses interact with each other try eating potato chips while listening to very loud music. They taste like crap.

DarqueKnight

madmax wrote: »

My system sounds different for a short amount of time when the vision of it changes.

I'm not clear on what you mean by "vision". Does vision refer to:

1. The way the system visually appears due to changes in room lighting?

2. Your mental concentration on specific areas or aspects of the sound stage?

3. Neither of the above?

hearingimpared

I think "Lady Be Good" on Jazz at the Pawnshop is by far the best piece of music to critically listen to. The instruments are well placed (even though they are on a crowded 6' to 8' stage and you can focus on any of the five instruments at any given time and still get the feel and sound of the other instruments.

Things that a glaring are; where, how far apart, depth of distance between instruments. You can even tell that Lars Erstrand"s vibes are set up and start right of center and and angles back to left of center and is about 5' long. You get the same feel for the drum kit across the stage. The ping of the piano hammers hitting the strings is evident too.

Magnificant piece for evaluating gear for imaging, tone and soundstage and that 3D holography is breathtaking!

analog97

Joe,

Do you think you could take written notes, carefully and as prescribed by a specific written methodology to feel confident you could detect a subtle improvement or decrement between, say a cable change? That's a complex piece. So much going on!!

All the best

hearingimpared

analog97 wrote: »

Joe,

Do you think you could take written notes, carefully and as prescribed by a specific written methodology to feel confident you could detect a subtle improvement or decrement between, say a cable change? That's a complex piece. So much going on!!

All the best

Larry, I have and still do. I have two copy books, one for gear and one for music that I keep just for the analytical purposes. I have to admit it, now that things have become what they have, I don't take up this practice as much. I just listen. However when the time comes to be able to buy gear and music, out will come the copy books.

The thing is, it is not a one shot listen take notes and listen again. It takes a lot more time and effort. Focusing on each instrument and vocal in each test and writing the observations down takes a lot of repetiton if one cares to get an accurate assessment of the changes, or lack thereof.

DarqueKnight

analog97 wrote: »

Joe, Do you think you could take written notes, carefully and as prescribed by a specific written methodology to feel confident you could detect a subtle improvement or decrement between, say a cable change? That's a complex piece. So much going on!!

hearingimpared wrote: »

The thing is, it is not a one shot listen take notes and listen again. It takes a lot more time and effort. Focusing on each instrument and vocal in each test and writing the observations down takes a lot of repetiton if one cares to get an accurate assessment of the changes, or lack thereof.

I think, this is the part a lot of people don't like...the time and effort involved in becoming proficient at critical listening. As with any stimulus picked up by human senses, the more experience and training you have, the more you are able to perceive...and perceive quickly...even in a complex evaluative environment.

Frustration with the time and effort required for true critical evaluation has lead to the adoption of evaluation methods which sacrifice accuracy for quick, easy results. Instead of one person at a time sitting in the stereo sweet spot, some people adopt a "group listening" methodology. Instead of taking the time to listen to entire musical selections repeatedly over an adequate period of time, some people adopt a "quick switch" methodology where musical selections are listened to in 90 second snippets and then the equipment under evaluation is switched.

Time and effort...there is just no getting around it.:)

madmax

DarqueKnight wrote: »

I'm not clear on what you mean by "vision". Does vision refer to:

1. The way the system visually appears due to changes in room lighting?

2. Your mental concentration on specific areas or aspects of the sound stage?

3. Neither of the above?

I meant number 1. Change of lighting, covering the speakers with a different color cloth, really any visual change in the enviroment. It could be that I'm distracted by the change for some amount of time and it affects the way I listen.

headrott

DarqueKnight wrote: »

Time and effort...there is just no getting around it.:)

I heard that Raife! Well said.:)

Greg

hearingimpared

headrott wrote: »

I heard that Raife! Well said.:)

Greg

See, I think that is the point of people coming up with using ABX testing for testing audio gear. It fast, convenient and because of that unrealiable. That is why I said you have to repeat your listening through the whole piece of music while taking notes focusing on each instrument and vocal(s). There's no getting around the time and effort factor if you really want to get an accurate assessement of what it is you are testing being gear or music.

Can this be done at a brick & mortar store? It depends on the store but there is nothing like taking a particular piece of gear or music and placing in your own rig to compare it. Can you say "30 day money back gurantee?" That IMHO is solid testing especially when the differences can be very subtle, if at all, good or bad.

inspiredsports

Great work, DK! You've hit the nail on the head in positing a structured protocol where random chaos exists. I'm looking forward to October, December or February to give it a read.