When is enough enough? Empirical guidelines to determine participant sample size for scene viewing studies
A. Hoogerbrugge, Ignace T. C. Hooge, Roy S. Hessels, Christoph Strauch
Abstract
Abstract Eye tracking is widely used to study where spatial attention is allocated across stimuli. However, determining a sufficient and efficient number of participants for such studies remains a challenge. While clear guidelines have been established for many classical statistical tests, no straightforward participant sample size guidelines exist for the comparison of gaze distribution maps and area-of-interest analyses – two of the most prominent analyses in scene viewing studies. Just how many participants should be included for reliable and reproducible gaze estimations? We here utilized gaze data to a single static image, viewed by 1248 individuals (dataset 1), and gaze data to 200+ images, viewed by 84 participants each (dataset 2). Researchers can assess which of these datasets and analysis types most resemble their setup and determine their sample size accordingly. Although we cannot provide a one-size-fits-all sample size recommendation, we show progressively diminishing returns for a range of sample sizes and for two typical study types. For example, when using Normalized Saliency Score as a metric of distribution map similarity, a 5% relative increase requires increases in sample size from 13 $$\rightarrow $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>→</mml:mo> </mml:math> 20 $$\rightarrow $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>→</mml:mo> </mml:math> 34 participants (based on dataset 1) or from 10 $$\rightarrow $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>→</mml:mo> </mml:math> 16 $$\rightarrow $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>→</mml:mo> </mml:math> 32 participants (based on dataset 2). Alternatively, when analyzing the number of visits to certain areas of interest, a 25% decrease in outcome variance requires increases in sample size from 13 $$\rightarrow $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>→</mml:mo> </mml:math> 24 $$\rightarrow $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>→</mml:mo> </mml:math> 44. We provide easy-to-use guidelines and reference tables to determine scene viewing participant sample size for academics and industry professionals alike.