To Smiley, Or Not To Smiley? Considerations and Experimentation to Optimize Data Quality and User Experience for Contextual Product Satisfaction Measurement?

Aaron Sedley
https://docs.google.com/a/google.com/presentation/d/e/2PACX-1vQMmPQ6xeyUWbA_tey23GiXJ8SUdZWn8FiL5E5x7BGrKOLe7Im8UnXOfRxBkFB0OYo_7ioovOpVztB1/pub?start=false&loop=false&delayms=5000 (2017)

Abstract

Happiness Tracking Surveys (HaTS) at Google are designed to measure satisfaction with a product or feature in context of actual usage. Smiley faces have been added to a fully-labeled satisfaction scale, to increase discoverability of the survey and response rates. Sensitive to the potential variety of effects from images and visual presentation in online surveys (Tourangeau, Conrad & Couper, 2013), this presentation will describe research designed to inform and optimize Google's use of smileys in Happiness Tracking Surveys across products and platforms:

1) We explore construct alignment by capturing users' interpretations of the various smiley faces, via open-ended responses. This data shows meaningful variation across potential smiley images, which informed design decisions.
2) We assess scaling properties of smileys by measuring each smiley independently on a 0-100 scale, to calculate semantic distance between smileys in order to achieve equally-spaced intervals between scale points (Klockars & Yamagishi, 1988).
3) We describe considerations and evaluative metrics for a smiley-based scale with endpoint text labels, to be used with mobile apps and devices.