11/16/2010

David and I have met and tried out the patch with newly recorded testing material. The test music consisted of excerpts from Michael Jackson's Off the Wall, Dave Grusin's Discovered Again, Prokofiev's Romeo and Juliet, and The Weather Report's Heavy Weather.

The results were that we found difficulty in discerning the difference between both the Prokofiev and Grusin excerpts, while it was considerably easier to identify which of the test trials were comparing similar systems, i.e., Michael Jackson recording from the more expensive system to an second recording of Michael Jackson from the same system. It is our opinion that this discrepancy in results is founded in the repeatable phrases in the excerpts chosen: The strong reoccuring rhythms in the Michael Jackson and Weather Report create a grid within which we were able to repeatably hear and analyze the difference between the two recordings. Moreover, it was easy to use the switch button within a single playback and hear a similar if not the same phrase repeat immediately. This is not to say that the Grusin or the Prokofiev were absent of rhythm! Rather, the rhythmic repetition was either less apparent or on longer time scales.

Now what to do? Its clear that the two systems need to be made closer for this comparison. While they are certainly close, it is our opinion that we must change the components, then retest using these same audio examples. The options available to us for adjusting the systems are as follows:

1. cartridges: we have already adjusted the cartridge, and in doing so have added a step-up amplifier (because we were using an MM cartridge originally, and now are using a MC cartridge). This change significantly improved the difficulty of the comparison. Perhaps an additional adjustment to another MC cartridge more akin to the Ortofon in use on the expensive system might be an appropriate next step, especially considering that we are using the DL-103, which is on the less expensive side of our cartridge collection (see cartridges here).
   
2. pre-amplifer: before changing cartridges from MM to MC, we tested the comparison of the expensive turntable and cartridge through the GSP combo (Jazz Club and Elevator EXP) and Manley Steelhead using the variable output (set to a similar perceptual volume loudness to the GSP combo). This comparison produced a very difficult comparison.
   

It is also our opinion that we purchase a duplicate set of true test records: Now that we have determined that the audio used will play a *large* factor in the differentiability of the 2 systems, we think it only appropriate to assess the difficulty of the test on the actual test material, rather than relying on close material.

We already have a duplicate of Dark Side of the Moon. Our plan for this week is to adjust the system setup and retest using the albums of difficulty, then once we are satisfied, also include Dark Side of the Moon.

11/8/2010

Our testing software is now just about ready. As a first step to assure that the patch will in fact behave correctly, we will be conducting the experiment ourselves, using audio recorded from the 2 systems (but not the audio from the actual experiment). If there is a noticeable difference between the 2 systems, we will begin to modify the setup of the less expensive system such that it is more similar to the more expensive one.

Here is a screenshot of the present patch:


The patch opens first for the experimenter to enter the participants ID. Upon entering the ID and hitting OK, the patch in the background (purple) opens in full screen. Once the participant selects OK, the audio begins for the first trial. During the initial playback of both A and B, all buttons and dialogs are hidden from view, leaving only the purple box with the A and B boxes, trial number and help button visible. Once these 2 files have played, the additional functionality appears and the user may use the patch as required (e.g., replay A or B, switch preserving the timing between the 2, and pause). Button presses and selections are recorded and stored with the time from the trial inception. Each block consists of a single musical passage, recorded twice on each system. This creates 4 versions of the passage, which will be compared in 8 trials per block, presented in a double-blind order. The trials are as follows (order is *not* preserved):

1. A1, A2
2. A2, A1
3. B1, B2
4. B2, B1
5. A1, B1
6. B1, A1
7. A2, B2
8. B2, A2

The next blogpost will display the results of the initial experiment as run on Guillaume, David and myself (the developers of the patch), as well as recommendations as needed for a closer comparison between the 2 systems.