Software Engineering
At Google, we pride ourselves on our ability to develop and launch new products and features at a very fast pace. This is made possible in part by our world-class engineers, but our approach to software development enables us to balance speed and quality, and is integral to our success. Our obsession for speed and scale is evident in our developer infrastructure and tools. Developers across the world continually write, build, test and release code in multiple programming languages like C++, Java, Python, Javascript and others, and the Engineering Tools team, for example, is challenged to keep this development ecosystem running smoothly. Our engineers leverage these tools and infrastructure to produce clean code and keep software development running at an ever-increasing scale. In our publications, we share associated technical challenges and lessons learned along the way.
Recent Publications
Towards A Human-in-the-Loop Framework for Reliable Patch Evaluation using an LLM-as-a-Judge
Renyao Wei
José Cambronero
AI-SQE '26: Proceedings of the 1st International Workshop on AI for Software Quality Evaluation - Judgment, Metrics, Benchmarks, and Beyond, ACM (Association for Computing Machinery), New York, NY, USA (2026), pp. 19 - 28
Preview abstract
Reliable evaluation is crucial for advancing Automated Program Repair (APR), but prevailing benchmarks that rely on execution-based evaluation methods (pass@k) often fail to capture the patch quality required for real-world adoption. This creates a significant gap between automated metrics and true patch validity (valid@k), a discrepancy observed across several state-of-the-art techniques.
To develop a scalable solution for measuring valid@k, we first study the human evaluation process itself. While manual assessment can determine validity, we find it suffers from poor inter-rater reliability (Fleiss' Kappa k=0.307). Our foundational insight is that this inconsistency is largely resolved when evaluators use a shared, high-quality rubric, which significantly improves agreement.
Building on this finding, we propose an LLM-as-a-Judge framework that operationalizes rubric-guided evaluation at scale. Our method employs a human-in-the-loop workflow where an LLM first generates a candidate rubric for a given bug, which a human expert then reviews and refines into a "golden" evaluation standard. This golden rubric is then used by an LLM judge to assess the validity of candidate patches. In an evaluation on 48 bugs and 115 patches, our LLM judge demonstrates substantial agreement with the consensus of human developers. This work contributes a scalable and reliable methodology for approximating valid@k, providing a much-needed high-fidelity signal for measuring true progress in the field of automated program repair.
View details
From Correctness to Collaboration: A Human-Centered Taxonomy of AI Agent Behavior in Software Engineering
Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26), ACM, New York, NY, USA (2026)
Preview abstract
The ongoing transition of Large Language Models in software engineering from code generators into autonomous agents requires a shift in how we define and measure success. While models are becoming more capable, the industry lacks a clear understanding of the behavioral norms that make an agent effective in collaborative software development in the enterprise. This work addresses this gap by presenting a taxonomy of desirable agent behaviors, synthesized from 91 sets of user-defined rules for coding agents. We identify four core expectations: Adhere to Standards and Processes, Ensure Code Quality and Reliability, Solve Problems Effectively, and Collaborate with the User. These findings offer a concrete vocabulary for agent behavior, enabling researchers to move beyond correctness-only benchmarks and design evaluations that reflect the realities of professional software development in large enterprises.
View details
Beyond CLEAN and MVP: Architecting an Offline-first Reactive Data Layer in Android (RDLA)
Mervyn Anthony
InfoQ (2026)
Preview abstract
While MVP and CLEAN Architecture are popular Android patterns, they often introduce boilerplate or lack reactivity. This article introduces the Reactive Data Layer Architecture (RDLA), an offline-first, push-based data layer pattern designed for modern Android apps using Jetpack Compose and Room. Using a heart rate tracking example, we demonstrate how RDLA provides robust local-remote synchronization, clean separation of concerns without Use Case overhead, and simplified unit testing via the TestExtensions pattern.
View details
Regression Test Selection at Scale
Laiba Mehnaz
Michelle Wang
Brandon Stewart
Daniel Lee
Charlie Song
Randall Parker
Ilya Kavalerov
2026
Preview abstract
Core libraries in massive codebases are dependency
hot-spots with high code churn and an immense number of
reverse dependencies. A small code change in these libraries
can impact a vast number of clients, creating a massive blast
radius of costly breakages if a bug or regression is submitted.
To prevent expensive post-submit breakages, it is not feasible to
test global dependencies during the pre-submit stage for each
code change because of resource limitations and the desire for
minimal developer friction.
In this paper, we propose a novel ranking-based approach that
utilizes a hybrid call graph to test global dependencies during the
pre-submit stage. Our method leverages the call graph to select a
per-commit subset of these global dependencies and to generate
inexpensive graph structural features that allow our machine
learning model to learn the complex relationships between library
code changes and client tests to predict failures. Evaluating our
approach on 10 high-impact core libraries with an average test
suite size of 220,000, we demonstrate that a fixed budget of 2,000
tests per commit, representing just 0.9% of the total execution
cost, effectively catches regressions for 40% of failing commits.
View details
OpenClaw in Practice: Building Laptop-Less Engineering Workflows with an Agent Harness
Hackernoon (2026)
Preview abstract
This article introduces OpenClaw, an AI-powered workflow harness designed to reduce operational friction for developers. Unlike traditional chatbots, OpenClaw manages and persists context across complex engineering tasks, enabling asynchronous operations and mobile-first interactions. The post explores practical use cases, including incident triage from mobile, asynchronous pull request reviews, quick infrastructure scripting, and automating routine operational tasks. It also delves into the key architectural layers of an OpenClaw-like system—Connectors, Gateway/Session Manager, Agent Runtime, Memory/Configuration, and Skills/Tools. The article emphasizes the importance of security, observability, and proper integration with existing developer ecosystems, positioning OpenClaw as a shift towards reducing context switching and enhancing developer productivity by automating the workflows around coding.
View details
Taming the Variants Multi-Architecture Continuous Testing at Google
Chandrakanth Chittappa
Ali Esmaeeli
Laura Macaddino
Sam Manfreda
David Margolin
Dharma Naidu
Sabuj Pattanayek
Sachin Sable
Ruslan Sakevych
Dushyant Acharya
Adrian Berding
Kevin Crossan
Wolff Dobson
Abhay Singh
19th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2026, Daejeon, Republic of Korea, IEEE
Preview abstract
Enterprises are increasingly adopting multiple general-purpose computer architectures in the data center. This leads to new testing challenges as it creates demand to qualify the software for the additional architectures. Naively double-testing all software for both architectures is costly and unnecessary. Further, reconfiguring CI/CD to take advantage of the new architecture can be non-trivial at scale. This paper introduces CI/CD variants and an optimized testing cycle to solve these twin challenges. We empirically evaluate our solution's impact on human and machine expenses using 44k projects at Google on real production data. First, we estimate saving ~25% of machine expenses at the negligible cost of a few delayed breakage detections per day. Second, we estimate a 90+% reduction in human cost for migrating the configuration. All features described in this paper are now Generally Available at Google and we report this as an empirical case study in scaling CI/CD to new architectures.
View details
×
![]()