Position Bias Correction for Search Behavior Analysis

Author:Wei Min, Jason Wang

1. Introduction

Clicks contain information about user satisfaction with search results and provide a measurement of item relevance/quality. However, the observed click-through rate (CTR) is confounded with position bias, that is, users are more likely to click top-ranked items than lower ones regardless of relevance. This bias affects the click and sales statistics that have a profound implication for machine learned ranking. So discovering an effective way to offset the bias could have a big impact. The purpose of this research is to evaluate and compare three different methods of compensating for position bias. We use eBay search data to compare the performance of COEC (clicks over expected clicks), GaP (Gamma-Poisson) and Validated Impression. We recommend Validated Impression as a reasonable fix based on the experimentation.

 2. Model Introduction

2.1 Position models

 Introduce a hidden variable exam which is true if an item is examined. We make two assumptions:

• the probability of a click only depends on whether an item is examined
• the probability that an item is examined is independent of the particular item, and only depends on its position

From this it follows (see ref. [2]) that the probability of clicking an item i in position jis the product of the relevance-based probability p_i and the probability that position jgets examined q_j :

P(click = 1 | i, j) =  P(click = 1|exam=1, i)  × P(exam = 1|j) = p_i ×q_j .

2.1.1 COEC model

In COEC method estimate empirical CTR@position j q_j  as the sum of clicks over all items appearing in position j divided by sum of impressions over all items appearing in position j. Estimate p_i  as sum of clicks over all positions for item i divided by sum of “expected examinations” given the positions of each exposure of the item. For example, if item i has been exposed K tim