Consider the scenario where an algorithm is given a context, and then it must select a slate of relevant results to display. As four special cases, the context may be a search query, a slot for an advertisement, a social media user, or an opportunity to show recommendations. We want to compare many alternative ranking functions that select results in different ways. However, A/B testing with traffic from real users is expensive. This research provides a method to use traffic that was exposed to a past ranking function to obtain an unbiased estimate of the utility of a hypothetical new ranking function. The method is a purely offline computation, and relies on assumptions that are quite reasonable. We show further how to design a ranking function that is the best possible, given the same assumptions. Learning optimal rankings for search results given queries is a special case. Experimental findings on data logged by a real-world e-commerce web site are positive.
17/01/2020
11:00 am
Place: Aula Seminari, via Salaria 113, last floor
Charles Elkan is an adjunct professor of computer science at the University of California, San Diego (UCSD). His full-time position is as managing director and global head of machine learning at Goldman Sachs. From 2014 to 2018 he was the first Amazon Fellow, leading a team of over 30 scientists and engineers in Seattle, Palo Alto, and New York doing research and development in applied machine learning for both e-commerce and cloud computing. Before joining Amazon, Dr. Elkan was a tenured full professor of computer science at UCSD. His Ph.D. is from Cornell and his undergraduate degree from Cambridge, and he has held visiting positions at Harvard and MIT. Dr. Elkan's students have gone on to faculty positions at universities including Columbia, Carnegie Mellon, the University of Washington, and Stanford, and to leading roles in industry.