Salient visual foci on human faces in viewers' engagement with advertisements: Eye-tracking evidence and theoretical implications

In: Multimodality & society, Volume 2, Issue 1, p. 3-22

Abstract

Multimodal theories informed by Systemic Functional Linguistics involve postulations on users' interactions with multimodal artifacts which need to be tested with behavioral data. Studies empirically testing the postulations are of great theoretical and methodological significance for multimodal research but have so far remained nascent. To open up new avenues of research in this area, findings from an eye-tracking experiment designed to test social semiotic postulations are reported and discussed. Specifically, the experiment adopted a between-group matched design and involved 44 viewers and 10 trials in each of which a print advertisement was randomly presented to the viewers who were instructed to perform a group-specific ad viewing task. Eye movement metrics obtained from the experiment were analyzed by descriptive statistics based on Areas of Interest and the Scanpath Trending Algorithm to link fixation metrics to semantic components of the ads and to calculate the common viewing path per group and trial, respectively. Data analysis results revealed, among others, a concentration of visual attention on human faces placed at various regions of the ads. The findings are then channeled into discussions on information value theory in the grammar of visual design to explicate its implications for theorizing multimodal communication.