count crowd loi和roi,4 Crowdsourced Relevance · AI-Powered Search MEAP V08

4 Crowdsourced Relevance

This chapter covers:

How to harness your users' collective insights to improve the relevance of your search platform

Collecting and working with user behavioral signals

Leveraging Reflected Intelligence to create self-tuning models like signals boosting, collaborative recommendations and personalization, and machine-learned ranking

Building an end-to-end signals boosting model

Crowdsourced learning from content-based signals

4.1  Intro to Crowdsourced relevance

In chapter one, we introduced the dimensions of user intent as "content understanding", "user understanding", and "domain understanding". In order to create an optimal AI-powered search platform, we need to be able to combine each of these contexts to understand our users' query intent. The question, though, is how do we derive these understandings?

Many different sources of information may exist from which we can learn: documents, databases, internal knowledge graphs, user behavior, domain experts, and so on. Some organizations have teams that manually tag documents with topics or categories, and some even outsource these tasks using tools like Amazon Mechanical Turk, which allows them to crowdsource answers from people all around the world. For identifying malicious behavior or errors on websites, companies will often allow their users to report problems and even suggest corrections. All of these are examples of crowdsourcing - relying upon the inputs from many people to learn new information.

4.2  Working with User Signals

4.2.1  Signals vs. Content

4.2.2  Setting up our product and signals datasets (RetroTech)

4.2.3  Exploring the signals data

4.2.4   Modeling users, sessions, and requests

4.3  Introduction to Reflected Intelligence

4.3.1  What is Reflected Intelligence?

4.3.2  Popularized Relevance through Signals Boosting

4.3.3  Personalized Relevance through Collaborative Filtering

4.3.4  Generalized Relevance through Learning to Rank

4.3.5  Other reflected intelligence models

4.3.6  Crowdsourcing from content

4.4  Summary