Listening test

[timc]

Hi.

Im currently doing a project wich involves a blind listening test. We will have a group listening to recordings done with different speakers. The recordings will be done with an artificial head, and played back over purpose built headphones.

The question is: Can someone point me to some litterature (preferably on the internet) wich explains the pros and cons of different approaches. We have to decide using a short test with soundclips of 5secs, and compare just 2 and 2. And then extract the results. The other option is to do an intermidiate lengt test. The clips beeing about 15-20secs and the listeners will hear all clips following eachother.

If someone have other suggestions i would appreciate it.

The method should be documented.


-Tim

[ratitifb]

make some google searchs about "sound quality" "audioconformity" "paired comparison" "psychoacoutics" "HRTF" ...

I have some very interesting links at office and will sent some of them tomorrow ... ;)


http://www.acoustics.salford.ac.uk/res/cox/sound_quality/index.php?content=intro

[timc]

make some google searchs about "sound quality" "audioconformity" "paired comparison" "psychoacoutics" ...

I have some very interesting links at office and will sent some of them tomorrow ... ;)


This explains my problem. My searchwords were not good. "Paired Comparison" gave a heap of results. :)

Looking forward to reading your links.



-Tim

[ratitifb]

i have so much documentation on this subject ... let me take time please :p

but several doc are actually in french ...could you read them ?

http://acoustics.aau.dk/research/SQRU/projects.php#automotive_sq

www.sea-acustica.es/WEB_ICA_07/fchrs/papers/ppa-02-028.pdf (http://www.sea-acustica.es/WEB_ICA_07/fchrs/papers/ppa-02-028.pdf)

www.acoustics.hut.fi/icad2001/proceedings/papers/bonebri2.pdf (http://www.acoustics.hut.fi/icad2001/proceedings/papers/bonebri2.pdf)

http://www.hydrogenaudio.org/forums/index.php?showforum=40

make a google search with "MDS Analysis" "perceptive space"

[timc]

Thanks. Very informative links :)


-Tim

[ratitifb]

Don't forget that with paired comparison method the number of pair is increasing very quickly with the number of sound samples and ... the test duration should not exceed 20 minutes or 30 minutes MAX in order to avoid any fatigue from the listeners (jury) ...

Never orient the listener before jury testing : the question before test has to be "Which sound do you prefer A or B ?".
Remind to all the listeners that there is never a "bad answer" and each listener is totally "free" ...

Associated with paired comparison, the verbalisation method is very important (do verbalisation at the end of each listener's jury test : why, how, ...). This step is very important to better understand the post analysis of collected data and to define the best metrics.

timc : are you able to understand tech paper written in french ?

[timc]

Don't forget that with paired comparison method the number of pair is increasing very quickly with the number of sound samples and ... the test duration should not exceed 20 minutes or 30 minutes MAX in order to avoid any fatigue from the listeners (jury) ...

Never orient the listener before jury testing : the question before test has to be "Which sound do you prefer A or B ?".
Remind to all the listeners that there is never a "bad answer" and each listener is totally "free" ...

Associated with paired comparison, the verbalisation method is very important (do verbalisation at the end of each listener's jury test : why, how, ...). This step is very important to better understand the post analysis of collected data and to define the best metrics.

timc : are you able to understand tech paper written in french ?



I'm not able to read french even the slightest. Im fluent in English. and can stutter throug some German.

Thanks again for the help :)

-Tim

[Rolf]

Over the years I have attended some blind tests. The biggest problem was to move large speakers to the test site, so this was mostly done with "movable" speakers.

To test electronics, this is of course much easier, but then, one is not able to listen the a large speaker system. To use earphones, this have never been "my thing", as I find the sound unrealistic.

[ratitifb]

when bass reproduction is needed to sound realistic (speaker test, car noise, railway noise, ...) you have to add subwoofer(s) to the headphone in the listening room during jury testing sessions (hearing by the body ...)

[timc]

Over the years I have attended some blind tests. The biggest problem was to move large speakers to the test site, so this was mostly done with "movable" speakers.

To test electronics, this is of course much easier, but then, one is not able to listen the a large speaker system. To use earphones, this have never been "my thing", as I find the sound unrealistic.


I agree to your last statement, but the unreal sound will be applied to all recordings, so the difference should remain the same.


ratitifb: In my case the speaker rolls off below 70ish Hz, so it no problem. The most important thing for us to determine is speech inteligiblity. (not sure about the spelling)



-Tim

[ratitifb]

The most important thing for us to determine is speech inteligiblity.what about STI method applied on speakers ?

[timc]

what about STI method applied on speakers ?


The method has been decided. It will be a 5sec pairing test. This is because it is well documented, and easy and quick to do. Thanks for all input. Been a big help.


-Tim

[ratitifb]

The method has been decided. It will be a 5sec pairing test. may i suggest in that case to don't exceed arround 10 sound samples for pairing comparison ...

In case of more samples than required to not exceed the max recommended test duration, you can perform a pre-listening tests to group sound by familly (categorisation test) in order to reduce the number of samples to listen during paired comparaison.


This is because it is well documented, and easy and quick to do.depending on what you want to learn from such test, actually the most difficult work reside in the post analysis of the answered lists (understanding of perceptive spaces, what metric has to be linked with each axis, quantifying each metrics with the main relevent values ...)

[timc]

Yep. I does require som post work.

However. This is a Bachelor assignment, wich meanst that the only man we have to please is the guy giving us the grade :)

So we follow his recommandations so he can't complain afterwards ;)

He says this is the way to do this particular test, so......

Just for the record: I don't agree. I find it very hard to determine quality after only 5 secs.


-Tim