ai 模型 管理框架_ai第一产品管理第1部分框架

ai 模型 管理框架

Expedia Group Technology —数据(EXPEDIA GROUP TECHNOLOGY — DATA)

If your company is developing its AI strategy and you are the product manager assigned to build one or more of its AI initiatives this post should help you. Do you have good knowledge and experience of the AI space already? Perhaps you worked as product manager on a couple of innovative initiatives, yet an AI-first product looks like an entirely different journey? I’ll lay out here the framework you need to move from concept to production.

如果您的公司正在制定其AI策略，并且您是负责构建一项或多项AI计划的产品经理，那么此职位应该会对您有所帮助。 您已经对AI空间有很好的知识和经验了吗？ 也许您曾在几个创新计划中担任产品经理，但以人工智能为先的产品似乎是完全不同的旅程？ 我将在这里列出从概念到生产所需的框架。

A little bit about myself: I am a PM at Expedia Group™ working on a Data Science and Algo team as part of the Marketing Organization. My day to day job is helping my business understand customers, reach out on the right channels and present them with the best travel options. For that we produce billions of predictions per day using AI, run A/B testing and apply statistical rigor in everything we do. I’ve also had the opportunity to work on other types of AI-first products in the past: conversation understanding as part of LivePerson Conversational Cloud and also ranking and recommendations engines for Azure Marketplaces. This has given me the opportunity to be involved in a good variety of AI products combining ML techniques, NLP pre-trained models, transfer learning and more.

关于我自己的一点点：我是Expedia Group™的PM，是数据科学和算法团队的一部分，是市场营销组织的一部分。 我的日常工作是帮助我的企业了解客户，联系正确的渠道并为他们提供最佳的旅行选择。 为此，我们每天使用AI进行数十亿次预测，进行A / B测试，并对我们所做的所有事情都应用严格的统计数据。 我过去也曾有机会研究其他类型的AI优先产品：作为LivePerson对话云的一部分的对话理解，以及Azure市场的排名和推荐引擎。 这使我有机会参与到很多结合了ML技术，NLP预训练模型，转移学习等等的AI产品中。

In this post I want to give an overview of the work involved and what can make or break your effort: bring the right expertise at the right time, how to drive ideation and most importantly how to create structure and clarity as product manager.

在这篇文章中，我想概述一下所涉及的工作以及可以做出或失败的事情：在正确的时间带来正确的专业知识，如何激发想法，最重要的是，如何以产品经理的身份建立结构和清晰度。

Executing an AI strategy takes a multiple quarter roadmap so getting your team to “Think Big and Small” instead of incubating is what it takes to succeed. Here’s what I’ll cover in this first part of this two-part series:

执行AI战略需要花费多个季度的路线图，因此要使您的团队“大大小小”而不是进行孵化是成功的前提。 这是由两部分组成的系列的第一部分中介绍的内容：

1. Focus on “AI First”
   专注于“人工智能优先”
2. Customer development
   客户发展
3. Business framing
   业务框架

Your journey gets the most exciting when you take your product from offline simulation into A/B testing to create a feedback loop to improve. Here’s what I’ll cover in the second part of this two-part series:

当您将产品从离线仿真引入A / B测试以创建反馈环以进行改进时，您的旅程将变得最令人兴奋。 这是由两部分组成的系列文章的第二部分：

1. Ideation and prototyping
   构想和原型制作
2. Test and learn
   测试和学习
3. Simplify to productize
   简化生产

AI-First助您一臂之力(AI-First comes to your rescue)

Let’s ask ourselves, what defines an AI-first product? In my mind there are three things that will set it apart:

让我们问问自己，什么定义了AI优先产品？ 在我看来，有三件事会使其与众不同：

1. Your business problem reached “the glass ceiling”: rules are unmaintainable, decision trees are not granular enough and human expertise can’t scale anymore.

1.您的业务问题达到了“玻璃天花板”：规则无法维护，决策树不够细致，人员的专业知识再也无法扩展。

For example: Can plain old analytics and forecasting meet business needs? How far off are the results with such a simplistic approach? Are you in the position of changing just one rule and at that point no human can tell if it still works? If yes, you are looking at an AI-first urgent step-in.

例如：简单的旧分析和预测能否满足业务需求？ 这样简单的结果有多远？ 您是否仅能更改一条规则，而此时还没有人知道它是否仍然有效？ 如果是，则您正在寻找AI优先紧急介入。

2. Your data is large and captured in great granularity. It deeply captures your business processes with individual interactions from start to end.

2.您的数据很大，并且捕获的粒度很大。 它从头到尾都通过单独的交互来深入捕获您的业务流程。

For example: Let’s imagine you have to create a recommendation engine. You’ll have to answer with yes to many questions before starting any work: What are all kinds of data you would need and are they available? Do they go far back enough? Can you use this data right away or does it need a few months to build a consumption pipeline? Do you have ethics practices in place for that data? If you answered yes to these questions, you’re ready to start.

例如：假设您必须创建一个推荐引擎。 开始任何工作之前，您必须对许多问题回答是：您将需要什么种类的数据，这些数据是否可用？ 他们走得足够远了吗？ 您可以立即使用此数据，还是需要几个月来建立消费渠道？ 您是否有针对这些数据的道德规范？ 如果您对这些问题的回答是“是”，那么您就可以开始了。

3. Your AI-first product integrates into a larger AI strategy. You have data engineering, a machine learning team and domain experts that pertain to your business problem: linguistics for NLP, business analysts for marketing etc.

3.您的AI至上产品将集成到更大的AI策略中。 您拥有与您的业务问题相关的数据工程，机器学习团队和领域专家：NLP的语言学，市场营销的业务分析师等。

For example: If you work in a large organization like Expedia Group you may have it all in place and you can get your roadmap and collaboration started. If you are working for a startup and can only have a tiny team, think how you will bring expertise: consulting, partnerships, etc.

例如：如果您在Expedia Group这样的大型组织中工作，则可能已经准备就绪，可以开始进行路线图和协作。 如果您在一家初创公司工作并且只能有一个很小的团队，请考虑如何带来专业知识：咨询，合作伙伴等。

Now you have got your three pillars in place: the challenge, the data and the talent. Time to think about your customers. We always start with them!

现在，您已经具备了三个Struts：挑战，数据和人才。 是时候考虑您的客户了。 我们总是从他们开始！

与客户交谈以了解哪些数据很重要 (Talk to customers to learn what data matters)

The first step goes without saying: know your customers. Is this a directly consumer facing solution or is it reaching them via a B2B process? Either way, you may learn your solution can power multiple products. Think model as a service when pinning down requirements.

第一步不言而喻：了解您的客户。 这是直接面向消费者的解决方案，还是通过B2B流程来解决？ 无论哪种方式，您都可以了解您的解决方案可以支持多种产品。 在确定需求时将模型视为服务。

Here’s an example of a directly consumer facing product: your recommendation engine shows personalized results on the homepage of your marketplace. You should not stop here. Can your Salesforce solution use the very same engine to find and prioritize proactive reachout? Expose the confidence score of your predictions, A/B test the optimal threshold and… voilà you are now powering two products.

这是一个直接面向消费者的产品的示例：您的推荐引擎会在市场首页上显示个性化结果。 您不应该在这里停下来。 您的Salesforce解决方案是否可以使用完全相同的引擎来查找主动宣传并确定其优先级？ 公开您的预测的置信度得分，A / B测试最佳阈值，然后……现在，您正在为两种产品供电。

For B2B products, your customer engagement can get even closer. Let’s imagine they are customer representatives in a care center. They have to audit conversations to improve or automate how are they are addressing the customer problem. Now just ask yourself, how do they get this done? There are two steps to find the answer:

对于B2B产品，您的客户参与度可以进一步提高。 假设他们是护理中心的客户代表。 他们必须审核对话以改善或自动化他们如何解决客户问题。 现在问问自己，他们如何做到这一点？ 有两个步骤可以找到答案：

Step 1: Shadow your customer for few hours. How do they analyze the data? Do they use regular expressions, excel spreadsheets, other tools?

第1步：为您的客户服务几个小时。 他们如何分析数据？ 他们使用正则表达式，excel电子表格或其他工具吗？

Step 2: Try to do the job yourself and see what it takes. Can you even do this work and maintain quality for more than few hours? As a bonus, you’ll learn about what types of data matters along the way.

第2步：尝试自己完成这项工作，然后看看需要什么。 您甚至可以完成这项工作并保持几个小时以上的质量？ 此外，您还将了解在此过程中哪些类型的数据很重要。

You’ve done all this awesome work and engaged with your customers. Now you’re ready to start framing your problem:

您已经完成了所有这些出色的工作，并与客户互动。 现在，您可以开始着手解决问题了：

是的，您的模型可以煮沸海洋……以后再做 (Yes, your model can boil the ocean… do that later)

We tend to think our AI product needs to solve all the customer problems at once like a magic wand. On the other hand, connecting to your customers has shown you can start with just one type of problem and do a good job. For example, you can recognize one type of conversation, image, etc. and deliver immediate value. If “intent to unsubscribe” is golden to your customer, start with that. This exercise will show your team where the bar is for winning against your customer’s substitute/workaround. You may be surprised how easy it is to have a quick win! Most likely you don’t need 99% accuracy on day one to double the efficiency of their business. On this point, I can’t stress enough the importance of bringing specific business requirements from your customer. What one type of prediction does your product needs right now? And what is an acceptably accurate prediction? To break this down further here’s a way for you to define the type of problem you’re solving and metrics that will define your success:

我们倾向于认为我们的AI产品需要像魔杖一样立即解决所有客户问题。 另一方面，与客户建立联系已表明，您可以仅从一种类型的问题入手并做好工作。 例如，您可以识别一种类型的对话，图像等，并立即产生价值。 如果“取消订阅”对您的客户来说是黄金，那就从此开始。 该练习将向您的团队展示在与客户的替代品/解决方法相对立的方面取得胜利的标准。 您可能会惊讶于获得快速胜利有多么容易！ 您很可能不需要在第一天就获得99％的准确性来使他们的业务效率提高一倍。 在这一点上，我不能太强调从客户那里提出特定业务需求的重要性。 您的产品现在需要哪种类型的预测？ 什么是可以接受的准确预测？ 为了进一步分解，这里提供了一种方法来定义要解决的问题的类型以及确定成功的指标：

Does your product need to understand existing data?

您的产品需要了解现有数据吗？

Example: NLP problems utilize a taxonomy/ontology to map natural language to establish concepts: intents, entities, etc. Image recognition tries similarly to classify. For all data that can’t be classified it will try to auto-cluster and help customers navigate it easier. If those clusters are unmanageably large, one way to explore them is by using a dimensionality reduction technique and visualize.

示例：NLP问题利用分类法/本体论来映射自然语言以建立概念：意图，实体等。图像识别尝试类似地进行分类。 对于所有无法分类的数据，它将尝试自动聚类并帮助客户更轻松地进行导航。 如果这些簇太大而无法管理，则探索它们的一种方法是使用降维技术并进行可视化。

If data was not properly classified you’ll have to explore the gap, create required data and reinforce your model. Sounds simple on paper yet finding the right data and eliminating bias in labeling will need rigor and great statistics skills on top of the domain experts that define your taxonomy. Engage all this expertise in the team early on!

如果数据分类不正确，则必须探索差距，创建所需数据并加强模型。 在纸上听起来很简单，但要找到正确的数据并消除标签上的偏差，将需要定义您的分类法的领域专家之上的严谨和出色的统计技能。 尽早将所有这些专业知识融入团队！

Does your product need to create new data?

您的产品需要创建新数据吗？

Example: In image processing field, it would mean to increase resolution into an image or fill in areas where light was too strong or not enough. For more complex problems like an industrial process or serving as an actor in an auction, we need an objective function/formulation, and the values generated may be on a continuous scale rather than discrete. In these problems, you only know your past product’s actions as an actor. You don’t know what all the other actors did and further will do or even what is their objective. This is why you can’t even go into the past data and simulate a full what-if scenario. There is also no human/auto-labeling or human crowdsourcing to leverage directly.

示例：在图像处理领域，这意味着要提高图像的分辨率或填充光线太强或不足的区域。 对于更复杂的问题，例如工业过程或在拍卖中充当参与者，我们需要目标函数/公式，并且所生成的值可能是连续的而不是离散的。 在这些问题中，您仅知道过去产品作为演员的行为。 您不知道其他所有参与者所做的事情以及进一步的行为，甚至他们的目标是什么。 这就是为什么您什至无法查看过去的数据并模拟完整的假设情景的原因。 也没有直接使用人工/自动标记或人工众包的优势。

Let me give you another example: you want to personalize the experience of a user and introduce recommendations. You’ll likely formulate this as an optimization problem yet you can never walk back in time and try again until the user engages. You’ll only know the error of your prediction. This becomes an interesting problem of what data signals you should bring in to know more about that context. This is where hypothesis based experimentation will get you moving!

让我再举一个例子：您想个性化用户的体验并提出建议。 您可能会将其表述为优化问题，但您永远无法退后并重试，直到用户参与。 您只会知道预测的错误。 这将成为一个有趣的问题，您应该输入哪些数据信号以进一步了解该上下文。 这是基于假设的实验将带您前进的地方！

At this point you may feel your AI-first strategy is just about to change the world. In my experience you should not try to “boil the ocean” yet. I’ll talk in Part 2 how to gradually increase scope and performance for your Ai-first product. For now close with three questions for you to think about:

此时，您可能会觉得以人工智能为先的策略即将改变世界。 以我的经验，您不应该尝试“煮沸海洋”。 我将在第2部分中讨论如何逐步增加Ai优先产品的范围和性能。 现在结束时，您需要考虑三个问题：

1. How much should your customers know about AI?
   您的客户应该对AI了解多少？
2. When should you bring your science team into customer sessions?
   您什么时候应该带您的科学团队参加客户会议？
3. What should your science team know about the business case before starting to work on the solution?
   在开始研究解决方案之前，您的科学团队应该对业务案例了解什么？

Learn more about technology at Expedia Group

在Expedia Group上了解有关技术的更多信息

翻译自: https://medium.com/expedia-group-tech/ai-first-product-management-part-1-framing-55eb998fe5a3

ai 模型 管理框架