在接口中定义有效验证函数
You may wonder why?
您可能想知道为什么？
In theory, it should be simple to come up with an idea, gather feedback and prototype to get a demo and validate the concept.
从理论上讲，提出一个想法，收集反馈和原型以进行演示并验证该概念应该很简单。
However, in reality, the majority of data science and machine learning professionals fail at every step of the process and as a result deliver solutions that rarely achieve production stage or post launch and are of no value whatsoever (do not increase productivity, revenue or simply do not entertain).
但是，实际上，大多数数据科学和机器学习专业人员都会在流程的每个步骤中失败，因此所提供的解决方案很少达到生产阶段或发布后的结果，而且毫无价值(不要提高生产力，收益或仅仅是不要招待)。
Just one out of every 10 data science projects actually make it into production. (Source)
实际上，每10个数据科学项目中就有1个投入生产。 ( 来源 )
One could argue that data science and machine learning projects are only of a research nature and it is therefore obvious that most of them will fail.
有人可能会争辩说，数据科学和机器学习项目只是研究性质的，因此很明显，大多数项目都会失败。
Of course this is true, but how to structure the process of use case definition and validation to cut time and costs and increase the success rate? Building a prototype is usually a significant investment, requires effort and time and some of them are built for months… with an army of ML engineers onboard.
当然这是正确的，但是如何构造用例定义和验证过程以减少时间和成本并提高成功率呢？ 建造原型通常是一笔巨大的投资，需要大量的精力和时间，而其中的一些则需要数月的建造时间……机上有ML工程师大军。
Also, with COVID-19 restrictions in place some organizations struggle with remote collaboration and productivity (although some are proving to be even more effective than before!).
同样，由于实施了COVID-19限制，一些组织仍在为远程协作和生产力而苦苦挣扎(尽管事实证明某些组织比以前更有效！)。
Definitely there is no golden rule here and the process will differ between companies. However, this article aims to provide data science and ML professionals guidance and techniques that can help them to navigate between obstacles and enable them to launch a production solution that meets users’ expectations and generates benefits for the company (revenue, effectiveness, agility).
绝对没有黄金法则，并且公司之间的过程将有所不同。 但是，本文旨在为数据科学和ML专业人员提供指导和技术，以帮助他们在障碍之间导航，并使他们能够推出满足用户期望并为公司带来收益(收入，效率，敏捷性)的生产解决方案。
Regardless of how the process of use case validation is structured it will embed the following two main phases:
无论用例验证过程的结构如何，它将嵌入以下两个主要阶段：
● Ideation — use case definition and preparation for prototyping
● 构思 —用例定义和原型准备
● Validation — building a prototype and measuring results
● 验证 -建立原型并测量结果
如何定义用例？ (How to define a Use Case?)
Use case is a description/ an opportunity of how AI/ML may be applied to solve a particular problem.
用例是如何应用AI / ML解决特定问题的描述/机会。
Everything starts with an idea and such ideas may come from various sources, sometimes even unusual ones. Perhaps a VP saw a post on LinkedIn and became inspired, a data scientist may have read an interesting paper or an ML engineer came across a new algorithm and wants to explore its potential.
一切都始于一个想法，这样的想法可能来自各种来源，有时甚至来自不寻常的来源。 一位副总裁可能在LinkedIn上看到了一篇文章并受到启发，一位数据科学家可能已经读了一篇有趣的论文，或者ML工程师遇到了一种新算法，并想发掘其潜力。
Oftentimes, use cases originate from product management, UX team or HW teams, who just prototyped a new sensor or who would like to bring an awesome new feature to a product.
用例通常来自产品管理，UX团队或HW团队，他们只是为新传感器制作原型，或者想为产品带来很棒的新功能。
Each of the above examples may generate interesting use cases but at the same time may also only lead to spectacular and costly failure. For example, a frequent mistake is applying cool, complex deep learning algorithms (that require large amounts of data that is not available) to every problem, even those that could be successfully solved with heuristics or simple ‘traditional’ models.
上面的每个示例都可能产生有趣的用例，但同时也可能仅导致严重且代价高昂的失败。 例如，经常犯的错误是对每个问题都应用了很酷，复杂的深度学习算法(需要大量不可用的数据)，即使那些可以通过启发式或简单的“传统”模型成功解决的算法也是如此。
好的，如何开始？ (Okay, how to start?)
Invest in creativity & research!
投资创意与研究！
The best way to start is to organize a cross-team brainstorm session that includes not only your data science/ ML team but also other stakeholders from such teams as product management, user experience, hardware as well as any others who may contribute to use case definition and later validation.
最好的开始方法是组织跨团队的头脑风暴会议，不仅包括您的数据科学/ ML团队，而且包括产品管理，用户体验，硬件以及可能对用例做出贡献的其他团队中的其他利益相关者定义和以后的验证。
This mixture of different perspectives + creativity is the key to success!
不同观点和创造力的融合是成功的关键！
Data science is all about creativity and building meaningful solutions from bytes of data that make our world a better place!
数据科学就是关于创造力和从字节数据中构建有意义的解决方案的一切，这些数据使我们的世界变得更加美好！
Creativity can be boosted by sending interesting examples of use cases (papers, videos etc.) to involved parties prior to the session.
通过在会议之前向参与方发送有趣的用例示例(纸张，视频等)，可以提高创造力。
Foster research by studying the competition and their products, use cases from other industries and also applicable research papers.
通过研究竞争及其产品，其他行业的用例以及适用的研究论文来促进研究。
The current pandemic and need to collaborate remotely can also be a challenge. Be sure to focus on proper facilitation and provide ample opportunity for all participants to share their ideas and provide feedback.
当前的流行病和远程协作的需求也可能是一个挑战。 确保专注于适当的便利，并为所有参与者提供充分的机会分享他们的想法并提供反馈。
List all ideas that come up during the session. Select the 10 or so most interesting ones and have the team go over each one using the following list of questions to prioritize use cases and choose the most promising for prototyping:
列出会议中提出的所有想法。 选择10个左右最有趣的问题，并让团队使用以下问题列表对每个问题进行研究，确定用例的优先级，并选择最有前途的原型：
Problem — how can it be solved with data science and machine learning?
问题 -如何通过数据科学和机器学习解决？
Impact — how will users benefit from the suggested solution?
影响 -用户将如何从建议的解决方案中受益？
Value/ benefits — tangible & intangible; cost reduction, effectiveness, agility or other?
价值/收益 -有形和无形； 降低成本，有效性，敏捷性或其他？
Innovation — how unique/novel is the idea? How much research is involved? Is something similar available? Can any components be reused?
创新 -这个想法有多独特/新颖？ 涉及多少研究？ 有类似的东西吗？ 可以重用任何组件吗？
Data — is data available or is there the need to generate/collect it for prototyping?
数据 -数据是否可用，或者是否需要生成/收集以进行原型制作？
Validation — how easy is it to validate?
验证 - 验证有多容易？
Complexity — how difficult is it to build a solution and how long may it take to release the solution for users?
复杂性 –构建解决方案有多困难，以及为用户发布解决方案需要多长时间？
At this point, it is not necessary to be concerned with the absolute accuracy of the answers, as they are only rough estimates that support prioritization of use cases. The above list is only an example of important questions which can, of course, be adjusted/expanded to your specific needs.
在这一点上，不必担心答案的绝对准确性，因为它们只是支持用例优先级的粗略估计。 上面列出的只是重要问题的示例，可以根据您的特定需求进行调整/扩展。
Nevertheless, keep it short and simple. Apply predefined answers where possible. Such answers can be used further to apply weights and implement a scoring algorithm.
但是，请使其简短而简单。 尽可能应用预定义的答案。 这样的答案可以进一步用于施加权重并实施评分算法。
The final output from the above exercise should be three use cases shortlisted for the next phase.
上述练习的最终输出应是在下一阶段入围的三个用例。
Select the most promising one (that will be prototyped first) in terms of tangible business benefits, impact on users and time to market to conduct a simplified feasibility study.
根据有形的商业利益，对用户的影响和上市时间，选择最有前途的(将首先原型化)进行简化的可行性研究。
The outcome should be a short summary outlining the scope of work for the next phase (validation) and cover at least the following 6 areas: 1) Goal, 2) Business opportunity, 3) Technology, 4) Validation approach, 5) Data, 6) Team, 7) Schedule, 8) Risks.
结果应为简短摘要，概述下一阶段(验证)的工作范围，并至少涵盖以下六个领域：1)目标，2)商业机会，3)技术，4)验证方法，5)数据， 6)团队，7)时间表，8)风险。
如何验证用例？ (How to validate a Use Case?)
The answer is rapid prototyping.
答案是快速成型。
Do not spend time on Proof of Concept since your main goal should not be to verify if it is feasible to build a solution from a technological perspective. The answer is always yes (ok, almost always…). In most reasonable cases it comes down to a matter of data availability and choosing the right algorithms.
不要花时间在概念验证上，因为您的主要目标不应是从技术角度验证构建解决方案是否可行。 答案永远是肯定的(好的，几乎总是……)。 在最合理的情况下，这取决于数据可用性和选择正确的算法。
However, your goal for validation is not only technical feasibility, but foremost building a prototype to assess potential impact, usefulness and benefits of the solution aimed for productization. Additionally, assess effort, resources, timeframe and risk associated with developing the ultimate solution.
但是，验证的目标不仅是技术可行性，而且最重要的是构建原型来评估针对产品化的解决方案的潜在影响，有用性和益处。 此外，评估与开发最终解决方案相关的工作量，资源，时间表和风险。
A few of my personal hints for rapid prototyping:
我对快速原型制作的一些个人提示：
Clearly define and obtain the team’s understanding of the goal, scope, output and validation criteria!
明确定义并获得团队对目标，范围，输出和验证标准的理解！
Organize work in weekly sprints (max 4–5, longer may lead to scope creep and failure).
每周进行一次冲刺 (最多4到5次，时间更长可能会导致示波器蠕变和失败)。
Build the prototype incrementally, start with the basic model (even heuristics) and then enhance with additional improvements, switching to more advanced approaches.
逐步构建原型 ，从基本模型(甚至启发式)开始，然后通过其他改进进行增强，切换到更高级的方法。
Small agile cross functional teams (1–2 data scientists, 1–2 SMEs, + lead). Allocating many ML engineers (a common practice) to the same use case is a roadmap to failure.
小型敏捷的跨职能团队 (1-2个数据科学家，1-2个中小型企业和领导)。 将许多ML工程师(通常的做法)分配给相同的用例是失败的路线图。
Include SMEs that will be able to validate the use case (UX/HW/Product Management or other).
包括能够验证用例的中小企业 (UX / HW /产品管理或其他)。
Look for the simplest approach, neural nets require lots of data that you may not have and oftentimes the same results can be obtained with classical algorithms, even simple linear regression or a combination of statistical model and heuristics.
寻找最简单的方法 ，神经网络需要大量您可能没有的数据，并且通常可以通过经典算法(甚至是简单的线性回归或统计模型和启发式算法的组合)获得相同的结果。
Try different algorithms — do not stick to one, try others for the same problem to assess performance.
尝试不同的算法 -不要坚持使用一种算法 ，而要针对同一问题尝试其他算法来评估性能。
Invest time into research — reuse available architectures, models and packages.
花时间进行研究 -重用可用的架构，模型和软件包。
Ensure quality of data and apply necessary data cleansing.
确保数据质量并进行必要的数据清理。
Identify your key stakeholders and ensure ongoing communication.
确定您的主要利益相关者，并确保持续的沟通。
Collaboration in case of distributed teams, especially in light of the current pandemic restrictions may be difficult since rapid prototyping is performed most efficiently in collocated teams. Nevertheless, there are many different technologies and techniques that you can apply, ranging from task management tools, Kanban boards, to catch ups via video and ad hoc messaging. I will not promote any specific tools here since each company uses their own suite of tools that may be applied effectively for the intended purpose.
在分布式团队的情况下，尤其是鉴于当前的流行病限制，可能很难进行协作，因为在并置的团队中，快速原型制作最有效。 但是，您可以应用许多不同的技术，包括任务管理工具，看板，以通过视频和临时消息进行补充。 我不会在这里推广任何特定的工具，因为每个公司都使用自己的工具套件，这些工具可以有效地用于预期目的。
验证，最重要的部分！ (Validation, the most important part!)
Obviously, our goal here is not to validate the model (this is still important) but to ensure that the prototype/demo built corresponds to the use case that we defined, has the potential to make an impact for users and is able to generate tangible business value.
显然，我们的目标不是验证模型(这仍然很重要)，而是要确保构建的原型/演示与我们定义的用例相对应，有可能对用户产生影响并能够产生有形的影响。商业价值 。
During this phase you should validate the prototype against quantitative and qualitative success criteria that you defined with the team & stakeholders prior to starting the project.
在此阶段中，您应该在开始项目之前根据与团队和利益相关者定义的定量和定性成功标准来验证原型。
Quantitative criteria can be an error threshold (MSE, recall or any other relevant ones) that the model has to achieve. It could also be a combination of different metrics related to performance of the model or prototype.
定量标准可以是模型必须达到的错误阈值(MSE，召回率或任何其他相关标准)。 它也可以是与模型或原型性能相关的不同指标的组合。
This part is easy.
这部分很容易。
The most challenging part is qualitative, where you cannot measure aspects such as value for users and benefits that the prototype may generate. Of course, you could run a focus group or usability study, however, such techniques require time and significant cost investment. Therefore, leave them for further phases and for the time being rely on your SMEs — UX and Product management’s expert knowledge. Also engage other teams in your company, share the prototype and gather feedback.
最具挑战性的部分是定性 ，在这里您无法衡量诸如用户价值和原型可能产生的收益等方面。 当然，您可以运行焦点小组或可用性研究，但是，此类技术需要时间和大量的成本投资。 因此，请将它们留作进一步的阶段，并暂时取决于您的SME-UX和产品管理的专家知识。 还请公司中的其他团队参与，共享原型并收集反馈。
However, remember — in the end it is all about money…
但是，请记住-最终，一切都与金钱有关……
If you want to convince your business stakeholders that your demo is valuable and may deliver financial profits you should perform a cost benefit analysis. List all direct, indirect and also intangible costs and benefits first, including those associated with performance increase and customer satisfaction. Apply monetary measurement to all positions, aggregate and compare. If you struggle to show benefits, work with your team to identify and quantify them.
如果您想让业务利益相关者相信您的演示很有价值，并且可以带来财务利润，则应该执行成本效益分析 。 首先列出所有直接，间接和无形的成本和收益，包括与绩效提高和客户满意度相关的成本和收益。 将货币计量应用于所有头寸，进行汇总和比较。 如果您难以显示出收益，请与您的团队一起确定并量化收益。
When should I start with the validation?
我什么时候应该开始验证？
The answer is — as soon as you start rapid prototyping. This should be a continual process that aims to validate usefulness of the use case.
答案是-一旦您开始快速制作原型。 这应该是一个连续的过程，旨在验证用例的有用性。
During rapid prototyping, communicate results to your stakeholders for buy-in and agreement of the next steps. This can be in the form of a short weekly meeting and/or daily update.
在快速原型制作过程中，将结果传达给您的利益相关者，以接受并同意后续步骤。 这可以是每周一次的简短会议和/或每日更新的形式。
Regardless of the output of validation — summarize findings.
不管验证的结果如何，都要总结发现。
The summary can be a document or presentation and should include not only results but also aspects related to productization, such as identified issues and risks that need to be considered when moving to the next phase and developing the solution.
摘要可以是文档或演示文稿，不仅应包括结果，还应包括与产品生产相关的各个方面，例如在进入下一阶段并开发解决方案时需要考虑的已确定问题和风险。
A summary doc is also useful when the project is put on hold due to lack of capacity or other company priorities at that particular time. Such a summary may serve as a good starting point even for other similar use cases that are considered for development.
当项目由于在那个特定时间缺乏能力或其他公司优先事项而被搁置时，摘要文档也很有用。 即使对于考虑开发的其他类似用例，此类摘要也可以作为一个良好的起点。
If particular use case after validation is not planned for further development review your backlog and pick up the next one and kick off the prototyping again.
如果没有计划在验证后进行进一步开发的特定用例，请查看您的积压订单，选择下一个积压订单，然后重新进行原型设计。
In the end, you will never get to the point that all your use cases pass validation and are scheduled for productization.
最后，您将永远无法达到所有用例都通过验证并计划进行生产的程度。
And frankly speaking, you should not expect them to. Why?
坦白说，您不应期望他们这样做。 为什么？
Because prototyping and validation are part of research and are intended to ensure that only the most promising use cases are converted into solutions and to avoid failure.
因为原型制作和验证是研究的一部分，目的在于确保仅将最有前途的用例转换为解决方案，并避免失败。
Hopefully, this article will provide you with some tried and proven tips and guidance on how to structure your use case definition, prototyping and validation process to maximize your success ratio. But as always, it is up to you, how you organize and apply these suggested guidelines within your own company and embed them within processes and tools already set in place.
希望本文将为您提供一些久经考验的技巧和指导，以帮助您构建用例定义，原型设计和验证过程，以最大程度地提高成功率。 但是，与以往一样，您将如何在自己的公司内组织和应用这些建议的准则，以及如何将其嵌入到已设置的流程和工具中，由您自己决定。
Wishing you good success! 🙂
祝您成功！ 🙂
翻译自: https://towardsdatascience.com/how-to-effectively-define-and-validate-use-cases-932c284837fe
在接口中定义有效验证函数
5875
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
