StrategicReading: Understanding Complex Mobile Reading Strategies via Implicit Behavior Sensing

Wei Guo
Byeong-Young Cho
Proceedings of ACM International Conference on Multimodal Interaction (ICMI 2020), ACM

Abstract

Mobile devices are becoming an important platform for reading. However, existing research on mobile reading primarily focuses on low-level metrics such as speed and comprehension. In particular, for complex reading tasks involving information seeking, source evaluation, and integrative comprehension, researchers still rely on the labor-intensive analysis of reader-generated verbal reports. We present StrategicReading, an intelligent reading system running on unmodified smartphones, to understand high-level strategic reading behaviors on mobile devices. StrategicReading leverages multimodal behavior sensing and takes advantage of signals from camera-based gaze sensing, kinematic scrolling patterns, and the evolution of cross-page behaviors. Through a 40-participant study, we found that gaze patterns, muscle stiffness signals, and reading paths captured by StrategicReading can infer both users' reading strategies and reading performance with high accuracy.