Neslihan Bulut
I’m a Software Engineer in Google Research, working on Graph Neural Networks, Submodular Optimization and Data Efficient ML modeling.
Authored Publications
Sort By
General Geospatial Inference with a Population Dynamics Foundation Model
Chaitanya Kamath
Prithul Sarker
Joydeep Paul
Yael Mayer
Sheila de Guia
Jamie McPike
Adam Boulanger
David Schottlander
Yao Xiao
Manjit Chakravarthy Manukonda
Monica Bharel
Von Nguyen
Luke Barrington
Niv Efron
Krish Eswaran
Shravya Shetty
(2024) (to appear)
Preview abstract
Supporting the health and well-being of dynamic populations around the world requires governmental agencies, organizations, and researchers to understand and reason over complex relationships between human behavior and local contexts. This support includes identifying populations at elevated risk and gauging where to target limited aid resources. Traditional approaches to these classes of problems often entail developing manually curated, task-specific features and models to represent human behavior and the natural and built environment, which can be challenging to adapt to new, or even related tasks. To address this, we introduce the Population Dynamics Foundation Model (PDFM), which aims to capture the relationships between diverse data modalities and is applicable to a broad range of geospatial tasks. We first construct a geo-indexed dataset for postal codes and counties across the United States, capturing rich aggregated information on human behavior from maps, busyness, and aggregated search trends, and environmental factors such as weather and air quality. We then model this data and the complex relationships between locations using a graph neural network, producing embeddings that can be adapted to a wide range of downstream tasks using relatively simple models. We evaluate the effectiveness of our approach by benchmarking it on 27 downstream tasks spanning three distinct domains: health indicators, socioeconomic factors, and environmental measurements. The approach achieves state-of-the-art performance on geospatial interpolation across all tasks, surpassing existing satellite and geotagged image based location encoders. In addition, it achieves state-of-the-art performance in extrapolation and super-resolution for 25 of the 27 tasks. We also show that the PDFM can be combined with a state-of-the-art forecasting foundation model, TimesFM, to predict unemployment and poverty, achieving performance that surpasses fully supervised forecasting. The full set of embeddings and sample code are publicly available for researchers. In conclusion, we have demonstrated a general purpose approach to geospatial modeling tasks critical to understanding population dynamics by leveraging a rich set of complementary globally available datasets that can be readily adapted to previously unseen machine learning tasks.
View details
Data Sampling using Locality Sensitive Hashing for Large Scale Graph Learning
Mohamed Farghal
Animesh Nandi
Sarath Shekkizhar
2023
Preview abstract
An important step in graph-based data analysis and processing is the construction of similarity graphs. Recent works have focused on the semi-supervised setting to learn an optimal similarity function for constructing a task-optimal graph. However, in many scenarios with billions of data points and trillions of potential edges, the run-time and computational requirements for training the similarity model make these approaches impractical.
In this work, we consider data sampling as a means to overcome this issue. Unlike typical sampling use-cases which only seek diversity, the similarity-learning for graph construction problem requires data samples that are both diverse and representative of highly similar data points. We present an efficient sampling approach by taking an adaptive partition view of locality sensitive hashing. Theoretically, we show that, though the samples obtained are correlated with sampling probabilities that do not sum to one, the training loss estimated for learning the graph similarity model using our approach is unbiased with a smaller variance compared to random sampling. Experiments on public datasets demonstrate the superior generalization of similarity models learned via our sampling. In a real large-scale industrial abuse-detection example, we observe ≈10× increase in identifying abusive items while having lower false positive rate compared to the baseline.
View details
SubMix: Learning to Mix Graph Sampling Heuristics
Josh Dillon
Johannes Gasteiger
2023
Preview abstract
Sampling subgraphs for training Graph Neural Networks (GNNs) is receiving much attention from the GNN community. While a variety of methods have been proposed, each method samples the graph according to its own heuristic. However, there has been little work in mixing these heuristics in an end-to-end trainable manner. In this work, we design a generative framework for graph sampling. Our method, SubMix, parameterizes subgraph sampling as a convex combination of heuristics. We show that a continuous relaxation of the discrete sampling process allows us to efficiently obtain analytical gradients for training the sampling parameters. Our experimental results illustrate the usefulness of learning graph sampling in three scenarios: (1) robust training of GNNs by automatically learning to discard noisy edge sources; (2) improving model performance by trainable and online edge subset selection; and (3) by integrating our framework into decoupled GNN models improves their performance on standard benchmarks.
View details
Leveraging Organizational Resources to Adapt Models to New Data Modalities
Yemao Zeng
Girija Narlikar
Chris Ré
Peter Bailis
Sahaana Suri
Raghuveer Chanda
Abishek Sethi
2020
Preview abstract
As applications in large organizations grow and evolve, the machine learning (ML) models that power them must adapt to new data modalities that arise over the application life cycle (e.g., a new video content launch in a social media application requires existing models apply to video).
To solve this problem, organizations typically create ML pipelines from scratch.
However, this fails to utilize the large volumes of organizational resources they possess in the form of existing services and models operating over related tasks, prior data modalities, aggregate statistics, and knowledge bases.
In this paper, we demonstrate how organizational resources can help construct a common feature space that enables teams across an organization to share data and resources for new tasks across different data modalities.
This allows teams to apply methods for training data curation (e.g., weak supervision) and model training (e.g., forms of transfer learning) across data modality.
We demonstrate how this improves end-model performance and time-to-deployment when creating cross-modal pipelines.
This serves as a case study in building a system to leverage resources from across an organization for each step of the ML pipeline, including feature generation, training data curation, and model training.
While techniques to use organizational resources at each step have been studied in isolation, we consider whether and how they compose at scale in a production setting.
View details
