When Recommendation Goes Wrong - Anomalous Link Discovery in Recommendation Networks
Abstract
We present a secondary ranking system to find and remove
erroneous suggestions from a geospatial recommendation system.
We discover such anomalous links by “double checking”
the recommendation system’s output to ensure that it
is both structurally cohesive, and semantically consistent.
Our approach is designed for the Google Related Places
Graph, a geographic recommendation system which provides
results for hundreds of millions of queries a day. We model
the quality of a recommendation between two geographic entities
as a function of their structure in the Related Places
Graph, and their semantic relationship in the Google Knowledge
Graph.
To evaluate our approach, we perform a large scale human
evaluation of such an anomalous link detection system. For
the long tail of unpopular entities, our models can predict
the recommendations users will consider poor with up to
42% higher mean precision (29 raw points) than the live
system.
Results from our study reveal that structural and semantic
features capture different facets of relatedness to human
judges. We characterize our performance with a qualitative
analysis detailing the categories of real-world anomalies our
system is able to detect, and provide a discussion of additional
applications of our method.
erroneous suggestions from a geospatial recommendation system.
We discover such anomalous links by “double checking”
the recommendation system’s output to ensure that it
is both structurally cohesive, and semantically consistent.
Our approach is designed for the Google Related Places
Graph, a geographic recommendation system which provides
results for hundreds of millions of queries a day. We model
the quality of a recommendation between two geographic entities
as a function of their structure in the Related Places
Graph, and their semantic relationship in the Google Knowledge
Graph.
To evaluate our approach, we perform a large scale human
evaluation of such an anomalous link detection system. For
the long tail of unpopular entities, our models can predict
the recommendations users will consider poor with up to
42% higher mean precision (29 raw points) than the live
system.
Results from our study reveal that structural and semantic
features capture different facets of relatedness to human
judges. We characterize our performance with a qualitative
analysis detailing the categories of real-world anomalies our
system is able to detect, and provide a discussion of additional
applications of our method.
Research Areas
