Alexey Alexandrov
Alexey Alexandrov has been working on performance and optimization since 2004. He received PhD in Computer Science in Saratov State Technical University in Russia in 2002. He joined Google in 2013 where he is working on fleet-wide performance efficiency tools since 2016. Before Google he spent ~10 years at Intel leading the VTune performance analyzer team.
Research Areas
Authored Publications
Sort By
QProf: Fleetwide Transitive Cost Profiling for Warehouse-Scale Computers
Sam Xi
Ali Sheikh
Vance Lankhaar
Symposium on Operating Systems Principles (2026)
Preview abstract
Warehouse-scale services form the backbone of major cloud services relied on by billions of users every day. As these systems grow increasingly complex, understanding them and finding opportunities for optimization becomes more and more difficult. In this paper, we present QProf, a distributed systems profiler built upon RPC tracing. Going well beyond prior work that focuses on fleetwide profiling of single programs, QProf focuses on cost profiling of entire services. By treating a distributed program as if it were a single process, QProf can produce call-graph profiles of entire systems, so it can measure the transitive cost of services through the entire stack of backend dependencies. QProf is backed by a novel tracing mechanism called skeletal tracing which is exceedingly low-overhead and invisible to applications, ensuring that tracing minimally perturbs application behavior. An ambient per-query CPU profiler ensures high profiling coverage across thousands of microservices without manual instrumentation, and new fan-in APIs provide support for representing batched requests in a tracing data model that is tree-centric. QProf processes billions of traces per day to produce aggregated datasets which can be queried by engineers. Despite the sampled nature of traces, QProf is highly accurate in measuring QPS and CPU across thousands of microservices in an extremely diverse datacenter fleet. QProf has been deployed for every job in Google’s production fleet for several years. It has a myriad of use cases, and we present several case studies showing how it has been used to find the “room-at-the-top” in complex systems and optimize them across many dimensions.
View details
Break Dancing: low overhead, architecture agnostic software branch tracing
22nd ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES ’21) (2021)
Preview abstract
Sampling-based Feedback Directed Optimization (FDO) methods like AutoFDO and BOLT that employ profiles continuously collected in live production environments, are commonly used in datacenter applications to attain significant performance benefits without the toil of maintaining representative load tests. Sampled profiles rely on hardware facilities like Intel’s Last Branch Record (LBR) which are not currently available even on popular CPUs from ARM or AMD. Since not all architectures include a hardware LBR feature, we present an architecture agnostic approach to collect LBR-like data. We use sampling and limited program tracing to capture LBR like data from optimized and unmodified applications binaries. Since the implementation is in user space, we can collect arbitrarily long LBR buffers, and by varying the sampling rate, we can adjust the runtime overhead to arbitrarily low values. We target runtime overheads of <2% when the profiler is on and zero when it’s off. This amortizes to negligible fleet-wide collection cost given the size of a modern production fleet.
We implemented a profiler that uses this method of software branch tracing. We also analyzed its overhead and the similarity of the data it collects to the Intel LBR hardware using the SPEC2006 benchmarks. Results demonstrate profile quality and optimization efficacy at parity with LBR-based AutoFDO and the target profiling overhead being achievable even without implementing any advanced tuning.
View details
