I had an interesting interview with Beverly Freeman of eBay on Friday, following on from my comments in “Eye Tracking Methodology” the white paper by Kara Prenice and Jacob Nielsen of Nielsen Norman Group.

Beverly was interested to find out why I advocate NOT SHOWING users their eye gaze data in the retrospective interview, when the latest release of Tobii Studio (2.0) has significant functionality to support moderators showing users exactly this.

In 2002, we were the first commercial organisation to purchase a Tobii eye tracker (a Tobii ET17). During our early experimental days I was inspired by watching a gaze replay video of Tom Dodd (son of Dr Jon Dodd, my co-founder) when he was two years old.

During seconds 26 to 31 of the video, Tom makes a decision about which Teletubbie he would like to play with. You can hear him subvocalising his decision with an “errrrr” sound as he looks from the yellow one, to the group, to the red one and back to the yellow one. As Tom was only two year old at the time he could not articulate his decision making process, but, that didn’t matter as I had access to his eye movements and could infer what the decision making process was likely to have been.

I was inspired with the potential insights I could gain and the technology that made it possible. As a result I went on to develop a retrospective methodology based on showing users their own gaze replay videos and/or gaze paths and asking them about what they had done and what their motivation was. In 2005 we ran an experiment with Lancaster University to test the methodology to see if it worked better than concurrent think aloud, and it did.

Since 2005 the retrospective methodology has been further developed to overcome a specific problem; users are often enamoured by the eye tracking technology and the results it produces. This takes the focus away from the user’s experience of the website and focuses it on the eye tracking technology.

One approach that we tried with some success was to show the user a gaze replay video at the start of the session, even before they had been calibrated. The moderator would explain how the eye tracker works, what the results would look like and how the user’s eye tracked data would be employed to discuss what they did and why. Pre-emptively showing the user what they would see removed some of the novelty, but it was an approach that only partially treated a symptom, it was not a cure. A bigger change was needed and I’m pleased to say that my Bunnyfoot co-founder Dr Jon Dodd had the insight to deliver just that.

Jon has been using eye tracking since he started his doctorate at Oxford in 1995 and any reverence to the technology he may have had, is long gone. I think this lack of reverence helped Jon to realise that we should stop showing users their gaze replay data in any form and instead use the intelligence and experience of the moderator to watch and diagnose the information they are receiving from the live viewer.

We call this methodology Post Experience Eye Tracked Protocol or PEEP. The protocol requires the moderator to take notes and then use these notes to guide the retrospective interview, without showing the user their own eye tracking data. The protocol requires highly skilled practitioners, capable of multitasking and thinking on their feet. Being a great usability practitioner is a highly skilled job!

It took me many years to get to this point, but as a result of this experience I’d like to offer a word of caution to usability practitioners starting out with eye tracking; just because the tools exist within Tobii Studio to allow you to share user’s eye tracking data with them, does not mean that you should!

To be a great usability practitioner, you need to realise that eye tracking is just a tool. A valuable one, but one that deserves no more reverence that Microsoft Office. That is why I’m not in love with eye tracking.

Thanks for reading: http://twitter.com/modestrobert