查看-增强会话
Follow up on my previous post discussing the key technologies around the conversational AI solution, I will be dive into the typical challenges the AI Engineer team would encounter when building a virtual agent or a chatbot solution for your clients or customers.
跟进我之前的帖子，讨论 围绕对话式AI解决方案的关键技术 ，我将深入探讨AI工程师团队在为客户或客户构建虚拟代理或聊天机器人解决方案时遇到的典型挑战。
Let firstly define the scope and goal of the conversational application.
首先让我们定义对话应用程序的范围和目标。
The conversational agents can be categorized into two main streams. Open Domain Conversation and Task-Oriented Conversation.
会话代理可以分为两个主要流。 开放域对话和面向任务的对话 。
The typical agents for Open Domain Conversation are Siri, Google Assistant, BlenderBot from Facebook, Meena from Google. Users can start a conversation without a clear goal, and the topics are unrestricted. Those agents factor entertainments and emotional response into their design, and able to carry a long conversation with end-users.
开放域对话的典型代理是Siri，Google助手，Facebook的BlenderBot和Google的Meena。 用户可以在没有明确目标的情况下开始对话，并且主题不受限制。 这些代理商将娱乐性和情感React纳入其设计之中，并能够与最终用户进行长时间的交谈。
Most of the commercial used virtual agents are Task-Oriented. They include the chatbot you saw on your bank’s website or the virtual agent who greets you when you call the flight center hotline. Those agents were built to serve specific goals and objectives. They focus on close domain conversation and typically would fulfill your requests with a response. The dialog in task-oriented agent tends to be short and logical.
大多数商业使用的虚拟代理都是面向任务的 。 它们包括您在银行网站上看到的聊天机器人，或者当您拨打飞行中心热线电话时与您打招呼的虚拟代理。 这些代理旨在满足特定的目标。 他们专注于近距离对话，通常会通过响应满足您的请求。 面向任务的代理中的对话框往往简短而合乎逻辑。
Now we have a good definition of agent type, let’s explore the challenges in the realm of Task-Oriented Conversation. (the open domain conversation is beyond today’s topic, and will be covered in the future post)
现在我们已经对代理类型有了一个很好的定义，让我们探讨面向任务的对话领域中的挑战。 (开放域对话超出了今天的主题，将在以后的帖子中介绍)
面向任务的对话 (Task-Oriented Conversation)
(Please refer to the previous post for a detailed explanation of the below architecture diagram)
(有关以下架构图的详细说明，请参阅上一篇文章)
1.整合，整合，整合！ (1.Integration, Integration, and Integration!)
Building a stand along chatbot is easy. If you take the online course or follow up on the tutorial notebook, you can probably set up a voice agent in a few hours.
沿chatbot建立站很容易。 如果您参加在线课程或在教程笔记本上进行跟进，则可能可以在几个小时内设置语音代理。
But in the real world, the conversational component needs to be seamlessly integrated with the existing system and infrastructure. The virtual agent’s architecture varies significantly depending on the digital maturity of your client’s tech stack.
但是在现实世界中，会话组件需要与现有系统和基础架构无缝集成。 虚拟代理的体系结构根据客户技术堆栈的数字成熟度而有很大不同。
If you are using the popular conversational service on the cloud platform (Dialogflow, Lex, Azure bot service, etc.), and your virtual agent is exposed to the public, think security first. Essentially, bot or virtual agent is a series of API calls to your core conversational services, make sure the solution has a robust authentication and authorization mechanism, and sensitive information is encrypted. If the bot/virtual agent solution needs to interact with your existing system or database, set up an external TCL/SSL proxy to interpret and normalize the request, instead of letting bot directly send an unserialized message to your core system. Another potential benefit of using an external proxy is, you can build a generic RESTful API that will enable you to scale up quickly.
如果您正在云平台上使用流行的会话服务(Dialogflow，Lex，Azure bot服务等)，并且您的虚拟代理公开给公众，请首先考虑安全性 。 本质上，机器人或虚拟代理是对核心对话服务的一系列API调用，请确保该解决方案具有可靠的身份验证和授权机制，并且敏感信息已加密 。 如果漫游器/虚拟代理解决方案需要与您现有的系统或数据库进行交互，请设置外部TCL / SSL代理来解释和规范化请求，而不是让漫游器直接向您的核心系统发送未序列化的消息。 使用外部代理的另一个潜在好处是，您可以构建通用的RESTful API，使您能够快速扩展。
If you are using conversational service on the cloud and building a virtual agent for the internal users, those agents are most likely to be information retrieval heavy. Internal users or employees would like to use the virtual agent as a knowledge search engine and ask the chatbot to find the last meeting notes, tax invoice, specific company policy, or a proposal from last year. The key to this type of solution is to create and maintain a consistent knowledge base and dynamic indexing strategy. The knowledge base is the ground truth of the agent’s search space, and it correlated with the answer accuracy. However, documents and content are scattered in the organization, and lack of management and ownership. So the first step of building this type of virtual agent should be designing comprehensive data ingestion, management, and governance pipeline. Be careful if the data you need to collect and query are both online and offline, or in multiple cloud platforms.
如果您在云上使用会话服务并为内部用户构建虚拟代理，则这些代理很可能会占用大量信息 。 内部用户或员工希望将虚拟代理用作知识搜索引擎，并且 请聊天机器人查找上次会议记录，税务发票，特定的公司政策或去年的提案。 这种解决方案的关键是创建并维护一致的知识库和动态索引策略。 知识库是座席搜索空间的基础事实，它与答案的准确性相关。 但是，文档和内容分散在组织中，并且缺乏管理和所有权。 因此，构建此类虚拟代理的第一步应该是设计全面的数据接收，管理和治理管道 。 如果您需要收集和查询的数据是联机和脱机的，或者是在多个云平台中的，请务必小心。
Another critical part of the above solution is maintaining the indexing and knowledge ontology for the agent to query the knowledge base. In the simplest version, the virtual agent was backed by the curated Q&A sets and matching similar questions to the predefined answers. A more advanced agent would have semantic search capability (can be implemented via elastic search), it can understand natural language and perform query search on the knowledge base. The ultimate solution involves machine comprehension, making machines understand the corpus and long questions, thus find the span of the answer in the relevant document. A mature virtual agent solution would usually stack those components to increase the robustness.
上述解决方案的另一个关键部分是维护索引和知识本体，以供代理查询知识库。 在最简单的版本中，虚拟代理由精选的问答集支持，并将相似的问题与预定义的答案进行匹配。 更高级的代理将具有语义搜索功能(可以通过弹性搜索实现)，它可以理解自然语言并在知识库上执行查询搜索。 最终的解决方案涉及机器理解 ，使机器理解语料库和冗长的问题，从而在相关文档中找到答案的范围。 成熟的虚拟代理解决方案通常会堆叠那些组件以增加健壮性。
2.如何处理嵌套或复合意图匹配 (2. How to deal with Nested or Compound Intent matching)
In the perfect scenario, the user would ask a question with single and straight forward intent. But when humans communicate, we prefer to combine several intentions into one dialog.
在理想情况下，用户会直接提出一个简单的问题。 但是，当人们进行交流时，我们更喜欢将多种意图组合成一个对话。
Question: Can you check the weather for me? If it is good, help me to book a ticket for XXX.
问题：你能帮我看看天气吗？ 如果不错，请帮我预订XXX的门票。
Intent: [“check weather” , “book ticket”]
目的：[“查询天气”，“预订机票”]
In the virtual agent solution, the dialog management component needs to track the conversation’s status and identify multiple intents from the user expressions (or questions). Once multiple intents have been matched, the next step is to determine the execution or fulfillment priority.
在虚拟代理解决方案中，对话框管理组件需要跟踪对话的状态并从用户表达式(或问题)中识别多个意图。 一旦多个意图已经匹配，下一步就是确定执行或实现优先级。
Potential solutions can be:
可能的解决方案可以是：
Main Topic and parallel intent with priority scores
主要主题和与意图得分平行的意图
If, in your user scenario, intents that are naturally clustered into a topic, you can use the top-down setup like the above image. The main topic classifier will help you drill down to a subset of intents, then match to individual intent by considering pre-defined priority scores.
如果在您的用户场景中，意图自然地聚集到一个主题中，则可以使用如上图所示的自上而下的设置。 主要主题分类器将帮助您深入分析意图的子集，然后通过考虑预定义的优先级评分来匹配各个意图。
Main intent with follow up intent
主要意图与跟进意图
Suppose your user scenario has a natural logic flow that can be converted into a sequence of intents. Then a better configuration is to set up the main intent with follow up intents similar to a logic decision tree. The virtual agent will then guide the user through the predefined logic.
假设您的用户场景具有自然的逻辑流程，可以将其转换为一系列意图。 然后，更好的配置是使用类似于逻辑决策树的跟进意图来设置主意图。 然后，虚拟代理将通过预定义的逻辑指导用户。
Trigger confusion and error detection engine, and use recovering policy to clarify the intent.
触发混乱和错误检测引擎，并使用恢复策略来阐明意图。
This option will be discussed in section 4. manage confusion and ambiguity.
此选项将在第4节中讨论。 管理混乱和歧义 。
3.将对话映射到预定义任务中的挑战。 (3. The challenge of mapping conversation into pre-defined tasks.)
Machine and humans have a different logic framework. Human conversation is intuitive and non-linear, but the machine program is linear, logical, and strictly defined. When designing your conversational agent, think about the user scenarios and what questions would user normally asked.
机器和人有不同的逻辑框架。 人机对话是直观且非线性的，但是机器程序是线性，逻辑且严格定义的。 在设计会话代理时，请考虑用户方案以及用户通常会问哪些问题。
Once you have understood your customers’ behavior, try to find the key drivers and topics of their actions, then develop a task flow for each type of the topic(user scenario). Please keep it simple, easy to follow, and flexible to expand. (main branch and optional child-nodes)
了解了客户的行为后，请尝试查找其行为的关键驱动因素和主题，然后针对每种主题类型(用户场景)开发任务流。 请保持简单，易于遵循且灵活扩展。 (主分支和可选的子节点)
Your conversational agent needs to be configured based on the task flow, and keep the master task flow updated and resolve conflicted logic points at an earlier stage.
您需要根据任务流配置会话代理，并保持主任务流更新并在较早阶段解决冲突的逻辑点。
Suppose you are working on a mega-client project, where there are multiple parallel business divisions, and potentially hundreds of intents clustered into a range of topics. In that case, you can adopt the multi-agent architecture. A master agent controls the sub-agent, and the sub-agent has its own knowledge space and intent configuration.
假设您正在从事一个大型客户项目，该项目中有多个并行的业务部门，并且可能有数百个意图聚集在一系列主题中。 在这种情况下，您可以采用多代理架构。 主代理控制子代理，子代理具有自己的知识空间和意图配置。
[ Be careful if you choose to implement the above architecture, it is not suitable for PoC or quick experiments. Always start with a simple solution that can be easily integrated and deployed. Get some quick wins to build up the momentum, when the development cycle is more mature and stable, pivot to the fine-grained architecture design.]
[如果选择实现上述架构，请当心，它不适用于PoC或快速实验。 始终从一个易于集成和部署的简单解决方案开始。 在开发周期更加成熟和稳定时，获得一些快速的胜利来建立动力，转向细粒度的架构设计。]
4.我们应该如何涵盖长尾意图？ (4. How should we cover long-tail intent?)
Long-tail intents refer to the low-frequency and high variance questions that your virtual agent would receive over the time of service.
长尾意图是指您的虚拟代理在服务时间内会收到的低频和高方差问题。
It might cause the “out of intent” or “out of vocabulary” error.
这可能会导致“意图不足”或“词汇不足”错误。
The standard way of solving this issue is by building a knowledge base and use information retrieval techniques to generate answers for the unseen questions (instead of creating additional Q & A pairs or defining new intents mapping).
解决此问题的标准方法是建立知识库，并使用信息检索技术为看不见的问题生成答案(而不是创建其他问答对或定义新的意图映射)。
The long tail intents will always happen no matter how well you design your agent’s conversation and the underlying task flow. Human likes improvisation and abstraction, hence a prepare for the unseen and unknown.
无论您如何设计座席的对话和基本任务流程，长尾意图总是会发生。 人类喜欢即兴创作和抽象创作，因此为看不见和未知的事物做准备。
5.如何处理谈话中的混乱和模棱两可？ (5. How to manage confusion and ambiguity in conversation？)
Human has a robust communication protocol and able to clarify ambiguity and confusion quickly.
人类具有强大的通信协议，并且能够快速澄清歧义和混乱。
But for machines to clear up ambiguity, we need to design a mechanism to trigger the recovery and fallback policy (the rules defined by AI Engineer). A user’s expression(question) can be put into three groups, 1) clean and straight intent (easy to understand and process), 2) unknown intent (out of scope questions), and 3) uncertain intent (have potential matching but need additional clarification)
但是，对于要消除歧义的机器，我们需要设计一种机制来触发恢复和后备策略(由AI工程师定义的规则)。 用户的表达(问题)可以分为三类：1)清晰明确的意图(易于理解和处理)； 2)意图不明(超出范围的问题)； 3)意图不确定(具有潜在的匹配性，但需要附加条件)澄清)
The dialog management engine should be able to determine which group should the current user expression falls into. The general idea is that we assume the common user questions would typically follow the normal distribution. And the DM(dialog management) is using the classification threshold to triage the received questions.
对话框管理引擎应该能够确定当前用户表达式应属于哪个组。 一般的想法是，我们假设常见的用户问题通常遵循正态分布。 DM(对话管理)使用分类阈值对收到的问题进行分类。
predict_proba(expression)< ∂1, [Reject], trigger [Fallback] response
预言_proba(表达式)<∂1，[拒绝]，触发[回退]响应
∂1<predict_proba(expression)< ∂2, [Uncertain], trigger [Recovery] response
∂1<predict_proba(表达式)<∂2，[不确定]，触发[恢复]响应
predict_proba(expression)> ∂2, [Accept], trigger [Match Intent] response
预言_proba(表达式)>∂2，[接受]，触发[匹配意图]响应
6.如何设计A / B测试以改善您的虚拟代理？ (6. How to design the A/B testing to improve your virtual agent?)
In my humble view, A/B testing is the silver bullet to improve the usability and adoption rate of your virtual agent.
在我看来，A / B测试是提高虚拟代理的可用性和采用率的灵丹妙药。
Before conducting A/B testing, the agent is only being used in a sanctuary environment where testers and developers might have an unconscious bias in the development cycle. Instead of making an arbitrary decision on feature roadmap, build priority should be guided by the A/B testing result.
在进行A / B测试之前，该代理仅在避难所环境中使用，在该环境中，测试人员和开发人员可能在开发周期中存在无意识的偏差。 代替对功能路线图做出任意决定，构建优先级应以A / B测试结果为指导。
Below are a few design principles for the agent A/B testing:
以下是代理A / B测试的一些设计原则：
- Running the test over a period of time, and design a time interval for on/off test features. 在一段时间内运行测试，并为开/关测试功能设计一个时间间隔。
- Collect a significant size of samples to run the hypothesis test. 收集大量样本以运行假设检验。
- Build testing groups on different demographics. 在不同的受众特征上建立测试小组。
- Separate the UI change the virtual agent function change in testing. 单独的UI更改将在测试中更改虚拟代理功能。
Key metrics to track are:
要跟踪的关键指标是：
Activation Rate: The ratio of a random user would open the chatbot on your client application.
激活率 ：随机用户的比例将在客户端应用程序上打开聊天机器人。
Confusion Trigger: What’s the percentage dose the dialog agent classifies the user expression to ‘uncertain’ and trigger recovery policy.
混乱触发： 对话代理将用户表达式分类为“不确定”并触发恢复策略的剂量百分比是多少。
Fall Back Rate: The ratio of the fallback response being triggered in the conversation session.
回退率：在会话会话中触发的回退响应的比率。
Goal Completion Rate: How many percentages of the conversation lead to successful task completion.
目标完成率：多少百分比的对话可以成功完成任务。
Retention: How long does a user stay in the conversation session with the virtual agent.
保留时间 ：用户在与虚拟代理的对话会话中停留多长时间。
Self-Serve Rate: How often can the agent fulfill the user’s requests independently (without trigger fallback and need a human intervention).
自助服务率：座席可以独立执行用户请求的频率(不触发回退并且需要人工干预)。
User Satisfaction: The user feedback that rate the satisfaction of the experience in using the virtual agent service.
用户满意度：评估使用虚拟代理服务的体验满意度的用户反馈。
7.通过不断学习来改善您的对话代理人 (7. Improve your conversational agent via continuous learning)
Most of the engineering team wouldn't need to use this approach to improve their agent. The conversational cloud platform (Dialogflow or Lex) will do the heavy lifting for you.
大多数工程团队不需要使用这种方法来改进他们的代理。 对话云平台(Dialogflow或Lex)将为您完成繁重的工作。
However, if you would like to build your conversational system from scratch, below is a way to architect the continuous learning pipeline to improve your dialog system.
但是，如果您想从头开始构建会话系统，下面是一种构建持续学习管道以改进对话框系统的方法。
To train your dialog model, you need to prepare a relevant dataset. If you have exposed your agent to the end-user, build a feedback loop to collect the chat logs, transcripts, transactions, weblogs, etc. Then transform and normalize those data into a training database.
要训练对话框模型，您需要准备一个相关的数据集。 如果您已将代理暴露给最终用户，则建立一个反馈循环以收集聊天记录，成绩单，交易，网络日志等。然后将这些数据转换并标准化为培训数据库。
If you don’t have user interaction data, use a user simulator (e.g. .. ) to chat with your agent and collect the labeled data from the simulated conversation (e.g., classified or misclassified). Those data can be used to train the recovery policy and dialog state tracking model. The next step is to sample user goals (tasks users want to accomplish in the chat session) and use it on the following reinforcement learning phase as a reward evaluator.
如果您没有用户交互数据，请使用用户模拟器(例如..)与您的代理聊天，并从模拟对话中收集标记的数据(例如，分类或错误分类)。 这些数据可用于训练恢复策略和对话状态跟踪模型。 下一步是对用户目标(用户希望在聊天会话中完成的任务)进行抽样，并在随后的强化学习阶段将其用作奖励评估者。
Lastly, expose your agent online and enable online learning mechanism. Ensure your agent will get smarter and smarter when more and more people are using it.
最后，在线公开您的代理并启用在线学习机制。 确保当越来越多的人使用它时，您的代理将变得越来越聪明。
(This is an advanced topic, and I won’t be going too deep in this post. Do contact me if you are interested in exploring more.)
(这是一个高级主题，在本文中我不会做得太深。如果您有兴趣探索更多内容，请与我联系。)
What has been discussed in this post only represents a fraction of the problems you would face in real life. And each point that has been mentioned can be expanded into a research post.
本文中讨论的内容仅代表您在现实生活中将面临的部分问题。 并且已提到的每个要点都可以扩展为研究职位。
We are exploring what is possible and building the future while we are learning it. Try to take your client on the journey with you and communicate authentically and transparently.
我们正在探索什么是可能的，并在我们学习的同时建设未来。 尝试使您的客户与您一起旅行，并进行真实，透明的沟通。
Remember, we are building a conversational AI solution, so why shouldn’t we first communicate better with our clients before thinking about how to make our virtual agent interact more smoothly with end-users.
请记住，我们正在构建一个对话式AI解决方案，所以为什么不首先考虑与客户进行更好的沟通，然后再考虑如何使我们的虚拟代理与最终用户更流畅地交互。
Thanks for reading, until next time.
感谢您的阅读，直到下一次。
About me, I am a 👧🏻 who is living in Melbourne, Australia. I studied computer science and applied statistics. I am passionate about general-purpose technology. Working in a Global Consulting firm as an AI Engineer lead👩🏻🔬, helping the organization to integrate AI solutions and harness its innovation power. See more about me on LinkedIn.
关于我，我是a ，住在澳大利亚墨尔本。 我学习了计算机科学和应用统计学。 我对通用技术充满热情。 在全球咨询公司担任AI工程师领导 👩🏻🔬，帮助组织集成AI解决方案并利用其创新能力。 在LinkedIn上查看有关我的更多信息。
翻译自: https://towardsdatascience.com/conversational-ai-key-technologies-and-challenges-part-2-b8dea63aeafc
查看-增强会话
833
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
