The Emperor’s New Sneakers

Posted on by
Share

EyeTrackShop claim to be able to eye track a home based panel of participants using standard low resolution webcams. It’s a big claim that could revolutionise market research.
We were interested to find out if EyeTrackShop really works so we replicated an EyeTrackShop study with two fundamental differences:

  • We used specialist eye tracking equipment costing over $20,000 each, as used by thousands of academics so we know the eye tracking results are accurate.
  • We recruited participants using an in-street intercept rather than a self-selected online panel.

The Results

EyeTrackShop report that online female panelists looked at sneakers in this creative and produced a heatmap to illustrate it:

GazeHawk; women looked at sneakers

EyetrackShop report women look at sneakers.

In the comparative study THiNK Eye Tracking found that women who have not self-selected into an online panel don’t look at the sneakers:

Eye Tracking heatmap show women not looking at shoes, indeed they pay very little attention to the ad.

THiNK's Eye Tracking's heatmap proves women ignore the sneakers, indeed they give the ad little attention.

EyeTrackShops’s findings could mislead a marketer into believing that the creative draws attention to the sneakers and so is an effective piece of branding communication.
The Results
GazeHawk reported that women looked at shoes whilst men don’t and produced a heatmap to illustrate this

Why is there is difference between EyeTrackShop and THiNK Eye Tracking’s findings?

EyeTrackShop participants self-select onto an online market research panel, in self-selecting the panelists bias the sample. Most market research sampling induces some bias but it’s a particular sort of person who seeks out online panels to do research for up to $4 per study.

I self-selected into being on the GazeHawk in home webcam panel and have been invited to take part in 11 studies in less than 6 months. Reused panelists exhibit fundamental changes in behaviour because they become familiar with the research methodology, this is known as the Subject Expectancy Effect.

As a practical demonstration of how the Subject Expectancy effect fundamentally changes behaviour we showed THiNK participants the same ad and asked them to look at it, thereby changing their behaviour from natural to cued:

Cued participants look at the sneakers

Cued women look at the sneakers.

When participants are cued they do look at the sneakers, these cued results are comparable to EyeTrackShop’s.

This practical demonstration of the Subject Expectancy effect has far reaching implications for quantitative online panel based research.

Footnote: There are notable difference between EyeTrackShop’s webcam heat maps and eye tracking results gained from Tobii and EyeTech eye trackers including attention on the face, tattoo and bum, we will explore these differences in a further blog.