Tao Dong, PhD
Dr. Tao Dong is a Senior Staff User Experience Researcher at Google, based in the San Francisco Bay Area. His research currently focuses on human-AI collaboration in software development. Previously, Dr. Dong led the UX team for Google’s Programming Frameworks and Languages group, which includes open-source projects such as Flutter, Dart, and Go.
Dr. Dong holds a PhD in Human-Computer Interaction from the University of Michigan School of Information.
Research Areas
Authored Publications
Sort By
From Correctness to Collaboration: A Human-Centered Taxonomy of AI Agent Behavior in Software Engineering
Sherry Y. Shi
Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26), ACM, New York, NY, USA (2026)
Preview abstract
The ongoing transition of Large Language Models in software engineering from code generators into autonomous agents requires a shift in how we define and measure success. While models are becoming more capable, the industry lacks a clear understanding of the behavioral norms that make an agent effective in collaborative software development in the enterprise. This work addresses this gap by presenting a taxonomy of desirable agent behaviors, synthesized from 91 sets of user-defined rules for coding agents. We identify four core expectations: Adhere to Standards and Processes, Ensure Code Quality and Reliability, Solve Problems Effectively, and Collaborate with the User. These findings offer a concrete vocabulary for agent behavior, enabling researchers to move beyond correctness-only benchmarks and design evaluations that reflect the realities of professional software development in large enterprises.
View details
Towards AI as a Collaborative Partner: A Taxonomy of AI Agent Behavior in Software Engineering
Sherry Y. Shi
Proceedings of the 3rd ACM International Conference on AI-Powered Software (AIware '26), ACM, Montreal, QC, Canada (2026) (to appear)
Preview abstract
The ongoing transition of Large Language Models (LLMs) in software engineering from one-shot code generators into agentic partners requires a shift in how we define and measure success. While models are becoming more capable, the industry lacks a clear understanding of the behavioral norms that make an interactive software engineering (SWE) agent effective in collaborative software development in the enterprise. This work addresses this gap by presenting a taxonomy of desirable SWE agent behaviors, synthesized from 91 sets of developer-defined rules for SWE agents and validated through interviewing 15 experienced professional developers. In this taxonomy, we identify four core expectations: Adhere to Standards and Processes, Ensure Code Quality and Reliability, Solve Problems Effectively, and Collaborate with the Developer. These findings offer a concrete vocabulary for aligning SWE agent behavior with developer preferences, enabling researchers and practitioners to move beyond correctness-only benchmarks and start designing evaluations that reflect the socio-technical nature of professional software development in enterprises.
View details
Blending Accessibility in UI Framework Documentation to Build Awareness
The 25th International ACM SIGACCESS Conference on Computers and Accessibility, ACM (2023)
Preview abstract
The lack of accessibility awareness among industry professionals is one of the reasons for rampant inaccessible websites and applications. This problem is exacerbated by the industry norm of having a single place dedicated to accessibility in the documentation of UI frameworks, which makes accessibility difficult for developers to discover and implement as part of their workflows. This paper presents the Blended Approach (BA), a novel approach and framework for improving accessibility awareness through documentation. Unlike the conventional practice, it recommends sprinkling and repeating short snippets on accessibility throughout the documentation while linking developers to detailed explanations on the dedicated accessibility page. Thus, BA places the topic of accessibility on an equal footing as other common programming concerns such as performance, security, and UX. As a case study, we applied BA to the onboarding tutorial of Flutter, a popular UI toolkit. The positive feedback we received in our evaluation with 11 professional developers suggests BA can be a viable and effective approach.
View details
Synthesis-Assisted Video Prototyping From a Document
Brian R. Colonna
Christian Frueh
UIST 2022: ACM Symposium on User Interface Software and Technology (2022)
Preview abstract
Video productions commonly start with a script, especially for talking head videos that feature a speaker narrating to the camera. When the source materials come from a written document -- such as a web tutorial, it takes iterations to refine content from a text article to a spoken dialogue, while considering visual compositions in each scene. We propose Doc2Video, a video prototyping approach that converts a document to interactive scripting with a preview of synthetic talking head videos. Our pipeline decomposes a source document into a series of scenes, each automatically creating a synthesized video of a virtual instructor. Designed for a specific domain -- programming cookbooks, we apply visual elements from the source document, such as a keyword, a code snippet or a screenshot, in suitable layouts. Users edit narration sentences, break or combine sections, and modify visuals to prototype a video in our Editing UI. We evaluated our pipeline with public programming cookbooks. Feedback from professional creators shows that our method provided a reasonable starting point to engage them in interactive scripting for a narrated instructional video.
View details
An Early Rico Retrospective: Three Years Of Uses For A Mobile App Dataset
Biplab Deka
Forrest Huang
Jeffrey Nichols
Artificial Intelligence for Human Computer Interaction: A Modern Approach (2021)
Preview abstract
The Rico dataset, containing design data from more than9.7k Android apps spanning 27categories, was released in 2017. It exposes visual, textual, structural, and interactive design properties of more than72k unique UI screens. Over the years since its release, the original paper has been cited nearly 100 times according to Google Scholar and the dataset has been used as the basis for numerous research projects. In this chapter, we describe the creation of Rico using a system that combined crowdsourcing and automation to scalably mine design and interaction data from Android apps at runtime. We then describe two projects that we conducted using the dataset: the training of an autoencoder to identify similarity between UI designs, and an exploration of the use of Google’s Material Design within the dataset using machine learned models. We conclude with an overview of other work that has used Rico to understand our mobile UI world and build data-driven models that assist users, designers, and developers.
View details
The Impact of “Cosmetic Changes” on the Usability of Error Messages
Kandarp Khandwala
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM (2019), LBW0273
Preview abstract
Programmatic errors are often difficult to resolve due to poor usability of error messages. Applying theories of visual perception and techniques in visual design, we created three visual variants of a representative error message in a modern UI framework. In an online experiment, we found that the visual variants led to substantial improvements over the original error message in both error comprehension and resolution. Our results demonstrate that seemingly cosmetic changes to the presentation of an error message can have an oversized impact on its usability.
View details
The Moving Context Kit: Designing for Context Shifts in Multi-Device Experiences
Elizabeth Churchill
Jeffrey Nichols
Julia Haines
Michael Gilbert
Proceedings of the 2017 Conference on Designing Interactive Systems, ACM, New York, NY, USA, pp. 309-320
Preview abstract
Multi-device product designers need tools to better address ecologically valid constraints in naturalistic settings early in their design process. To address this need, we created a reusable design kit of scenarios, “hint” cards, and a framework that codifies insights from prior work and our own field study. We named the kit the Moving Context Kit, or McKit for short, because it helps designers focus on context shifts that we found to be highly influential in everyday multi-device use. Specifically, we distilled the following findings from our field study in the McKit: (1) devices are typically specialized into one of six roles during parallel use—notifier, broadcaster, collector, gamer, remote, and hub, and (2) device roles are influenced by context shifts between private and shared situations. Through a workshop, we validated that the McKit enables designers to engage with complex user needs, situations, and relationships when incorporating novel multi-device techniques into the products they envision.
View details
Understanding the Challenges of Designing and Developing Multi-Device Experiences
Elizabeth F. Churchill
Jeffrey Nichols
DIS '16 Proceedings of the 2016 Conference on Designing Interactive Systems
Preview abstract
As the number of computing devices available to users continues to grow, personal computing increasingly involves using multiple devices together. However, support for multi-device interactions has fallen behind users' desire to leverage the diverse capabilities of the devices that surround them. In this paper, we report on an interview study of 29 designers and developers in which we investigate the barriers to creating useful, usable, and delightful multi-device experiences. We uncovered three key challenges: 1) the difficulty in designing the interactions between devices, 2) the complexity of adapting interfaces to different platform UI standards, and 3) the lack of tools and methods for testing multi-device user experiences. We discuss the technological and business factors behind these challenges and potential ways to lower the barriers they impose.
View details
