Data Management
Google is deeply engaged in Data Management research across a variety of topics with deep connections to Google products. We are building intelligent systems to discover, annotate, and explore structured data from the Web, and to surface them creatively through Google products, such as Search (e.g., structured snippets, Docs, and many others). The overarching goal is to create a plethora of structured data on the Web that maximally help Google users consume, interact and explore information. Through those projects, we study various cutting-edge data management research issues including information extraction and integration, large scale data analysis, effective data exploration, etc., using a variety of techniques, such as information retrieval, data mining and machine learning.
A major research effort involves the management of structured data within the enterprise. The goal is to discover, index, monitor, and organize this type of data in order to make it easier to access high-quality datasets. This type of data carries different, and often richer, semantics than structured data on the Web, which in turn raises new opportunities and technical challenges in their management.
Furthermore, Data Management research across Google allows us to build technologies that power Google's largest businesses through scalable, reliable, fast, and general-purpose infrastructure for large-scale data processing as a service. Some examples of such technologies include F1, the database serving our ads infrastructure; Mesa, a petabyte-scale analytic data warehousing system; and Dremel, for petabyte-scale data processing with interactive response times. Dremel is available for external customers to use as part of Google Cloud’s BigQuery.
Recent Publications
SemBench: A Benchmark for Semantic Query Processing Engines
Jiale Lao
Gerardo Vitagliano
Immanuel Trummer
H. V. Jagadish
Sebastian Schelter
Andreas Kipf
Matthew Russo
Kris Kissel
Michael Cochez
Andreas Zimmerer
Olga Ovcharenko
Thibaud Hottelier
Gautam Gupta
Tianji Cong
2026
Preview abstract
We present a benchmark targeting a novel class of systems: semantic query processing engines. Those systems rely inherently on zero-shot abilities of state-of-the-art large language models (LLMs). They extend SQL with semantic operators, configured by natural language instructions, that are evaluated via LLMs and enable users to perform various operations on multimodal data. Our benchmark provides variety along three axis: scenarios, modalities, and operators. Included are scenarios ranging from movie review analysis to medical question-answering. Within these scenarios, we cover different data modalities, including images, audio, and text. Finally, the queries involve a diverse set of operators, including semantic filters, joins, mappings, ranking, and classification operators.
We evaluate systems according to processing overheads and result quality. We present experimental results for an industrial semantic query processing engine (BigQuery), as well as academic systems (LOTUS, Palimpzest, and ThalamusDB). Our results shed light on the relative strengths and weaknesses of the evaluated systems, and hint at promising avenues for future research.
View details
Autonomous Semantic Harmonization and Decoupled Staging for Enterprise Cloud CRM and ERP Data Migration
Subhani Shaik
Universal Library of Engineering and Technology (2026)
Preview abstract
Large-scale enterprise mergers, acquisitions (M&A), and cloud digital transformations routinely face severe operational deadlocks during the migration of historical, transactional, and customer data across heterogeneous cloud software ecosystems. Specifically, migrating deep relational object hierarchies from Customer Relationship Management (CRM) platforms (e.g., Salesforce) into financial transaction ledgers in Enterprise Resource Planning (ERP) systems (e.g., NetSuite, SAP, Oracle ERP) introduces cross-system impedance mismatch, relational orphaning, and data drift. Standard industry data migration tools—such as Salesforce Data Loader, NetSuite Bulk Upload, and point-to-point Extract, Transform, Load (ETL) scripts—rely on rigid, manual one-to-one field mappings and synchronous write pipelines that necessitate disruptive production cutover windows.
In this paper, we present the Autonomous Semantic Harmonization Framework (ASHF), a formal architectural protocol designed to govern high-stakes cloud CRM and ERP data migrations. ASHF replaces point-to-point scripting (O(N²) integration complexity) with a centralized Knowledge-Graph-Driven Semantic Resolution Hub (O(N) complexity) that autonomously maps conflicting relational entities across source and target schemas. Furthermore, ASHF introduces a Decoupled Quarantined Staging Engine coupled with an Asynchronous Data Triage and Total Rollback Verification Protocol, enabling continuous extraction and mathematical referential verification without interrupting live source systems. Empirical benchmarks across multi-tenant enterprise migrations demonstrate that ASHF achieves 100% relational data integrity (zero data loss), reduces human-in-the-loop data mapping design effort by 50%, cuts financial audit reconciliation times by 50%, and compresses end-to-end migration build schedules by 30% compared to standard linear ETL tools.
Index Terms— Cloud Data Migration, Enterprise Systems Unification, Customer Relationship Management (CRM), Enterprise Resource Planning (ERP), Knowledge Graphs, Zero-Downtime Migration, Semantic Harmonization.
View details
Data-usage descriptors as search metadata: the case of food security data and the National Data Platform (2015-2025)
Julia Lane
Rafael Ladislau
Lauren Chenarides
Simon Porter
Manish Parashar
Scientific Data, 13 (2026)
Preview abstract
Scientific data is a critical input into scientific research. Yet the research data landscape is constantly changing as new datasets emerge, others are retired, or some disappear altogether. Without a systematic way to track how datasets are used across a research field, researchers have no reliable method for identifying relevant data resources or locating communities that work with them. Data-usage descriptors can substantially advance research productivity by reducing the time that researchers spend finding new and relevant datasets in their research field, and the communities that use them. This paper describes how to generate data-usage descriptors by finding how datasets are used in publications and then linking the dataset information to the publication metadata. It also shows how usage descriptors can be used to find other related datasets and their usage. It concludes by arguing that the approach represents a critical piece of foundational infrastructure that could be deployed in repositories as part of a referenceable, navigable, and contextual data framework. This article contains a reproducible workflow for constructing data-usage descriptors, based on analyzing the full text of publications in the Dimensions database. The illustrative use case is research on food security. The illustrative repository is the National Data Platform.
View details
Preview abstract
Semantic data models express high-level business concepts and metrics, capturing the business logic needed to query a database correctly. Most data modeling solutions are built as layers above SQL query engines, with bespoke query languages or APIs. The layered approach means that semantic models can’t be used directly in SQL queries. This paper focuses on an open problem in this space – can we define semantic models in SQL, and make them naturally queryable in SQL?
In parallel, graph query is becoming increasingly popular, including in SQL. SQL/PGQ extends SQL with an embedded subset of the GQL graph query language, adding property graph views and making graph traversal queries easy.
We explore a surprising connection: semantic data models are graphs, and defining graphs is a data modeling problem. In both domains, users start by defining a graph model, and need query language support to easily traverse edges in the graph, which means doing joins in the underlying data.
We propose some useful SQL extensions that make it easier to use higher-level data model abstractions in queries. Users can define a “semantic data graph” view of their data, encapsulating the complex business logic required to query the underlying tables correctly. Then they can query that semantic graph model easily with SQL.
Our SQL extensions are useful independently, simplifying many queries – particularly, queries with joins. We make declared foreign key relationships usable for joins at query time – a feature that seems obvious but is notably missing in standard SQL.
In combination, these extensions provide a practical approach to extend SQL incrementally, bringing semantic modeling and graph query together with the relational model and SQL.
View details
Preview abstract
Unifying query languages is key in reducing toil for app developers and end users to query and analyze observability data. A common query language that can leverage all observability data such as metrics, traces, profiles, events, logs to facilitate correlation, support trend analytics and provide end-to-end observability for AI applications. The Observability TAG QLS workgroup is finalizing a semantic query language spec in 2025 and is recommending SQL as a basis with further experimentation on syntaxes. This talk will explore the design principles, user research and challenges of creating a query language to support observability goals. It will delve into the core concepts, syntax, and semantics of SQL operators and its needed syntactic sugar, while addressing the unique requirements of observability data. It will also explore the trade-offs between simplicity, expressiveness, and performance. This query language convergence for end-to-end analytics could enhance reliability and operational efficiency for SREs and your app developers. A win-win for all.
View details
Preview abstract
Traditionally, quality management relies on siloed systems of record such as quality management system (QMS), application lifecycle management (ALM), and manufacturing execution system (MES) platforms. These systems are often static, passive repositories that require significant manual effort to connect disparate data and derive actionable insights. Fragmentation and lack of proactive intelligence can lead to delays in identifying quality issues, ensuring compliance, and accelerating innovation. This disclosure describes a quality management framework to provide collaboration between human experts and specialized artificial intelligence (AI) agents for proactive and semi-autonomous quality management. The framework provides a distributed, intelligent ecosystem where a central AI engine can delegate specific, complex quality workflows to specialized AI agents that operate continually and autonomously, with a human-in-the-loop for final approval. The framework is built on a three-layer architecture that can be powered by a cloud computing platform.
View details
Some of our teams
×
![]()