Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 378 publications
Preview abstract
This talk addresses the challenges of operating Google's monitoring systems at scale, handling terabytes of telemetry data and preventing overload from diverse workloads. We'll explore how Google's internal client library and Monarch, its planet-scale time-series database, work together for cost-effective data collection. Key principles include a distributed push model, dynamic client-side data reduction, centralized retention, and periodic metric analysis. The session will then bridge these concepts to the open-source world, discussing our work with OpenTelemetry's OpAMP protocol to achieve similar scalable and efficient telemetry collection. Attendees will gain insights into adapting these principles for cost savings and learn about our collaboration with the OpAMP SIG to benefit the broader community.
View details
Preview abstract
Optimizing large-language model (LLM) training and serving on large-sacle distributed systems with hundreds and thousands of accelerators is always a challenging task due to the fast evloving LLMs, strong domain expertise required, and various optimization goals from different worklaods. Existing methods rely on either handcrafted optimization performed by human experts, which is tedious and time-consuming or resource-intensive black-box searches, which lack the extensibility to keep pace with evolving models and hardware. To address this, we introduce PROMPTS, a novel multi-agent framework that complements traditional search methods with expert-informed reasoning. It automates the diagnosis of performance bottlenecks by synthesizing profiler data and leverages a knowledge base to propose optimized sharding configurations with detailed justifications.
Across eight real-world production workloads, PROMPTS demonstrated remarkable efficiency and accuracy, delivering performance improvements of up to 434%. These workloads spanned diverse model architectures, hardware platforms, computational scales, and various stages of the machine learning lifecycle (pre-training, serving, and post-training). In every case, the configuration adopted by human engineers was identified within the agent's top three proposals from a single invocation. Furthermore, the agent's top-ranked recommendation was the one ultimately adopted in 87.5% of cases, showcasing its ability to not only find optimized solutions, but also to correctly prioritize them. Our work establishes PROMPTS as a scalable, extensible, and explainable methodology for AI-assisted performance engineering in large-scale ML systems.
View details
Preview abstract
This paper introduces Operationalized Temporal Entity Resolution, a distributed system architecture designed to resolve data consistency challenges in modern Security Information and Event Management (SIEM) environments. processing petabytes of high-velocity telemetry. We address the critical failure mode of ”State Smearing”—a temporal discrepancy between an entity’s state at event time versus analysis time—which frequently corrupts forensic timelines, particularly regarding ephemeral assets like containers and DHCP leases. Our approach coalesces heterogeneous data from diverse log sources into a single, canonical representation, processing over 2 billion entity fragments daily. By leveraging a deterministic Dynamic Graph Resolution via modified distributed connected components and a novel Density-Aware Temporal Checkpointing algorithm, we generate precise validity intervals. This method embeds temporal state directly into the resolution graph, eliminating the need for computationally expensive query-time joins. Ultimately, this architecture enables security analysts to perform ”time-travel” queries to reconstruct historical states accurately. Analysis of a production environment demonstrates that 8–16% of threat detection rules critically depend on this enriched temporal merging.
View details
Hybrid Reputation Routing for Decentralized Networks: A Blockchain-Enabled Secure and Scalable Protocol with On-Chain/Off-Chain Partitioning
Praveen Baskar
Om Narayan
Ramkinker Singh
2026
Preview abstract
Our suggestion is a hybrid reputation-based routing
protocol in the context of decentralized routing. networks which
partitions trust measurements to on-chain immutable logs and
off-chain dynamic computations to attain security and scalability.
The protocol incorporates a layer of sharded blockchain to store
critical. trust data and local reputation model to do real-time
updates, enabling routing choices that cannot be tampered with.
performance. The trust ledger which operated on-chain kept
the records of the impartiality of. Historical performance and
node identities, whereas the off-chain reputation engine uses
graph neural network to calculate dynamic. real-time scores like
packet delivery rate to be trusted. and latency. Additionally, it
uses rollup-based batches of synchronization off-chain. optimistic
updates to zero-knowledge proofs, making on-chain efficient.
checking with minimum overhead. The suggested approach
replaces conventional network discovery and forwarding modules
using trust-based adjacency lists and trust-based pathing, and
thus enhancing routing consistency over adversarial settings.
Moreover, the structure integrates Hyperledger Fabric with.
Graph Attention Network-based high-throughput sharded ledger
operations. to update reputation in a privacy-preserving manner,
proving to be linear. network size scalability. The experimental
findings indicate that the system supports 10,000 transactions per
shard and produces. Under 100 ms per ZK-Rollup proof, which
is appropriate to large-scale IoT. and DeFi deployments. This
publication fills the gap between pure on-chain. and off-chain
reputation systems, which provides a viable solution to scalable
and secure decentralized routing.
View details
Meta-Initialized Warm Pool Manager: Predictive Cold Start Mitigation for Serverless Functions via Adaptive Meta-Learning
Praveen Baskar
Om Narayan
Ramkinker Singh
2026
Preview abstract
Serverless computing platforms commonly suffer
from cold-start latency because of Cold-start problems dynamic
invocation patterns, but with current warm pool management
strategies face difficulty in effectively adjusting to changes in
workload. We propose the Meta-Initialized Warm Pool Manager
(MI-WPM), a new component which combines meta-learning and
adaptive control to address the problem of cold start efficiently.
The innovation is based on a meta predictor trained on diverse
invocation scenarios, which offers a generalized Initialization for
rapid fine-tuning based on detection of workload drift. Contrary
to traditional schemes based upon static heuristics or expensive
“retraining, MI-WPM adapts its warm pool sizes by combining
“ meta-initialized predictions with lightweight statistical drift
detection. The architecture uses a temporal fusion transformer
for modeling invocation patterns, Kolmogorov-Smirnov test for
monitoring drift, and feedback-controlled controller for opti-
mizing resource distribution. The experimental results show the
reduction in the Cold Start problem using MI-WPM. occurring
by 47% compared to state-of-the-art approaches while keeping
provisioning costs below 5%. Moreover, the Meta learning frame-
work supports sub-minute adaptation on unknown invocation
patterns, filling a gap in serverless computing. The proposed
approach is developed as a modular extension of the existing
serverless architectures without any infrastructure modifications.
This work fills the gap between offline-trained models and
reactive scaling, providing a principled approach to cold start
mitigation in production environments.
View details
QProf: Fleetwide Transitive Cost Profiling for Warehouse-Scale Computers
Sam Xi
Ali Sheikh
Vance Lankhaar
Symposium on Operating Systems Principles (2026)
Preview abstract
Warehouse-scale services form the backbone of major cloud services relied on by billions of users every day. As these systems grow increasingly complex, understanding them and finding opportunities for optimization becomes more and more difficult. In this paper, we present QProf, a distributed systems profiler built upon RPC tracing. Going well beyond prior work that focuses on fleetwide profiling of single programs, QProf focuses on cost profiling of entire services. By treating a distributed program as if it were a single process, QProf can produce call-graph profiles of entire systems, so it can measure the transitive cost of services through the entire stack of backend dependencies. QProf is backed by a novel tracing mechanism called skeletal tracing which is exceedingly low-overhead and invisible to applications, ensuring that tracing minimally perturbs application behavior. An ambient per-query CPU profiler ensures high profiling coverage across thousands of microservices without manual instrumentation, and new fan-in APIs provide support for representing batched requests in a tracing data model that is tree-centric. QProf processes billions of traces per day to produce aggregated datasets which can be queried by engineers. Despite the sampled nature of traces, QProf is highly accurate in measuring QPS and CPU across thousands of microservices in an extremely diverse datacenter fleet. QProf has been deployed for every job in Google’s production fleet for several years. It has a myriad of use cases, and we present several case studies showing how it has been used to find the “room-at-the-top” in complex systems and optimize them across many dimensions.
View details
Securing the Agentic Enterprise
Amazon Kindle (2026)
Preview abstract
Securing the Agentic Enterprise: Threat Modeling, Anomaly Detection, and Governing Autonomous Multi-Agent Systems addresses the critical security and governance gaps emerging as enterprises transition from human-supervised copilots to autonomous agentic workflows. As software processes gain the ability to reason, decompose natural language objectives, and execute multi-step tool calls at machine speed, traditional syntactic security boundaries (like firewalls and static analysis) become obsolete.
This book provides security architects, CISOs, and platform engineers with a practical, architecture-level blueprint for securing this new paradigm. It explores novel attack vectors such as indirect prompt injections and consumption-based economic threats and provides frameworks for robust mitigation. Key topics include modernizing agentic identity, implementing semantic firewalls, transition-state anomaly detection, and applying zero-trust principles to autonomous execution contexts. Bridging the gap between high-level ethical guidelines and isolated model safety, this guide prepares practitioners to confidently deploy and govern enterprise-grade autonomous systems.
View details
Preview abstract
Large-scale software systems frequently suffer from architectural rigidity caused by monolithic designs, tightly coupled integrations, and legacy technology stacks. Backend-forFrontend (BFF) architectures are increasingly adopted to address these challenges by decoupling frontend-specific requirements from backend domain services. However, designing a BFF layer requires a series of irreversible technology decisions across compute platforms, traffic routing, programming languages, frameworks, and API protocols. These decisions directly influence system latency, scalability, operational complexity, and long-term maintainability. This paper proposes a structured, metrics-driven decision framework to guide architects through foundational technology choices when designing BFF architectures. The framework decomposes the decision space into independent sub-problems, introduces weighted evaluation criteria, and applies quantitative scoring models to enable objective trade-off analysis. The approach is validated through a representative modernization scenario, demonstrating how systematic evaluation reduces architectural risk, resolves stakeholder disagreement, and improves performance and developer efficiency. The proposed framework is generic, repeatable, and applicable to a wide range of cloudnative system modernization efforts.
View details
Preview abstract
Serverless platforms offer compelling economics for low-traffic inference of small language models (SLMs), but cold start latency remains a critical barrier. We present a systematic empirical study of cold start performance for quantized SLMs served via llama.cpp on Google Cloud Run’s CPU-only infrastructure. We benchmark five models (270M– 3.8B parameters) across two memory tiers (4 GiB and 8 GiB), with a quantization sweep spanning five GGUF formats (Q2_K through Q8_0) for LLaMA 3.2 1B and three-point validation on Gemma 3 270M and Qwen3 0.6B. Our results reveal that model loading dominates cold start time (55–70%), the 8 GiB tier provides a hidden 2× vCPU advantage that nearly halves warm inference time, and Q4_K_M quantization achieves a Pareto-optimal tradeoff across latency, throughput, and model quality (confirmed via WikiText-2 perplexity). A threepredictor latency model (R 2 = 0.97) and break-even cost analysis provide practitioners a quantitative framework for SLM deployment decisions. Index Terms—small language models, serverless computing, cold start, quantization, llama.cpp, Cloud Run, GGUF, inference optimization
View details
The End of SaaS Dashboards: Why 2026 Belongs to Agents-as-a-Service
Hackernoon (2026)
Preview abstract
The traditional SaaS model, centered on human-operated dashboards, is being supplanted by Agents-as-a-Service (AaaS), where autonomous agents orchestrate outcomes. This article explores the architectural shift from request-response to reason-act loops, a reference "Agentic Stack," the rise of Agent-to-Agent (A2A) communication, and the implications for business models and system design, emphasizing the transition from User Experience (UX) to Agent Experience (AX).
View details
Preview abstract
This article presents a novel approach to automating operations tasks, particularly incident triage, by using AI agents defined entirely in Markdown. These agents orchestrate actions across various observability tools (e.g., Datadog, Splunk) and use the file system for state and communication, mimicking the Unix philosophy. The system enables parallel investigations, cross-tool validation, and structured reporting without traditional coding frameworks.
View details
Scalability With AI: Lessons From Real Production Systems
Hackernoon (2026)
Preview abstract
Traditional software scaling methods often fall short when applied to AI and Machine Learning systems, especially those involving Large Language Models (LLMs). This document explores the paradigm shift required to scale AI applications effectively in production. It highlights the challenges, such as non-deterministic inference latency and non-linear cost explosion, and contrasts AI systems with traditional web services. We present key architectural patterns and lessons learned from real-world production systems, including: Tiered Model Architectures for cost/latency optimization, Asynchronous Inference Pipelines to prevent blocking, Intelligent Semantic Caching using vector databases, and strategies for Graceful Degradation and Dynamic Batching. Additionally, we emphasize the need for an observability stack that goes beyond CPU and memory to monitor model behavior, data quality, and prediction drift. The goal is to provide practical guidance for engineers building and scaling robust and efficient AI-powered services.
View details
Fast ACS: Low-Latency File-Based Ordered Message Delivery at Scale
Anil Raghunath Iyer
Neel Bagora
Chang Yu
Olivier Pomerleau
Vivek Kumar
Prunthaban Kanthakumar
Usenix Annual Technical Conference (2025)
Preview abstract
Low-latency message delivery is crucial for real-time systems. Data originating from a producer must be delivered to consumers, potentially distributed in clusters across metropolitan and continental boundaries. With the growing scale of computing, there can be several thousand consumers of the data. Such systems require a robust messaging system capable of transmitting messages containing data across clusters and efficiently delivering them to consumers. The system must offer guarantees like ordering and at-least-once delivery while avoiding overload on consumers, allowing them to consume messages at their own pace.
This paper presents the design of Fast ACS (an abbreviation for Ads Copy Service), a file-based ordered message delivery system that leverages a combination of two-sided (inter-cluster) and one-sided (intra-cluster) communication primitives—namely, Remote Procedure Call and Remote Direct Memory Access, respectively—to deliver messages. The system has been successfully deployed to dozens of production clusters and scales to accommodate several thousand consumers within each cluster, which amounts to Tbps-scale intra-cluster consumer traffic at peak. Notably, Fast ACS delivers messages to consumers across the globe within a few seconds or even sub-seconds (p99) based on the message volume and consumer scale, at a low resource cost.
View details
Continuous Evaluation: Using CI Techniques For Experimentation At Scale
IEEE (2025), pp. 157-157
Preview abstract
Continuous Integration (CI) is an essential software development practice that establishes processes to minimize bugs and errors in production. In a similar vein, experimentation of software products is vital for evaluating user satisfaction, quality, performance and other key business metrics. Experimentation allows product owners to evaluate the user impact of changes. This can help make informed decisions regarding feature launches. Experimentation also allows developers to tweak internal processes and algorithms to maximize the impact of new features and changes. Additionally, it can sometimes detect errors not detected by CI.
Unlike CI systems, experimentation platforms are meant to closely imitate production and usually run the system under test (SUT) against a large scale of input. Despite this, experimentation platforms have a lot in common with CI systems. The mechanisms for continuously integrating and testing changes can be modified and applied to experimentation platforms.
Google Search's experimentation platform started as a command line tool many years ago. Over time, this tool has evolved into a platform that serves the evaluation needs for many of Google's products like Search, Assistant, YouTube, Play, Lens, etc., running thousands of large experiments every day.
In this workshop, we will present the evolution of Google Search's experimentation platform and how it was transformed from a simple CLI tool into a platform that works at scale, fulfills continuous experimentation needs and provides many CI-like functionalities to its users.
Note: This presentation was a part of CCIW workshop a ISCT 2025. Please download slides to see full presentation.
View details
The ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning
Michael Moffitt
Pratik Fegade
Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (2025), pp. 5-17
Preview abstract
A chief enabler of large-scale deep learning is the distribution of computation across multiple interconnected hardware accelerators. In order to unlock the maximum possible performance, a compiler must first select a reasonable strategy to parallelize a model's operations. Since neural network architectures admit multiple flavors of parallelism, determining the proper strategy for each instruction is a critical (albeit non-trivial) task. To solicit new ideas toward solving this challenging combinatorial optimization problem, we organized the ASPLOS 2025 / EuroSys 2025 Contest on Intra-Operator Parallelism for Distributed Deep Learning, a multi-month competition focused on advancing the state-of-the-art for model partitioning algorithms. In this paper, we offer a retrospective of this event, including the basic problem formulation, key challenges & opportunities, our new benchmark suite, and the quality of submissions received.
View details
