生命周期模型_建立从思想到价值的生命周期模型

生命周期模型

数据科学家来自火星，软件工程师来自金星(第4部分)(DATA SCIENTISTS ARE FROM MARS AND SOFTWARE ENGINEERS ARE FROM VENUS (PART 4))

In Part 1 of this series we examined the key differences between software and models; in Part 2 we explored the twelve traps of conflating models with software; and in Part 3 we looked at the evolution of models. In this article, we go through the model lifecycle, from the initial conception of the idea to build models to finally delivering the value from these models.

在本系列的第1部分中，我们研究了软件和模型之间的主要区别； 在第2部分中，我们探讨了将模型与软件合并的十二个陷阱。 在第3部分中，我们研究了模型的演变。 在本文中，我们经历了模型生命周期，从最初的构想到构建模型，再到最终从这些模型中提供价值。

We breakdown the entire lifecycle of models into four major phases — scoping, discovery, delivery, and stewardship. While there are many similarities between this model lifecycle and a typical software lifecycle, there are significant differences as well, stemming from the differences between software and models that we started this series with. Here we go over the four phases and the nine steps within these phases.

我们将模型的整个生命周期细分为四个主要阶段-范围界定，发现，交付和管理。 尽管此模型生命周期与典型的软件生命周期之间有许多相似之处，但也存在显着差异，这是由于我们开始本系列文章时所使用的软件和模型之间存在差异。 在这里，我们介绍了四个阶段以及这些阶段中的九个步骤。

价值范围 (Value Scoping)

This is arguably the most critical phase of the model lifecycle; this is the phase that determines if there is any business value in building the model, the success metrics for the model, and how we intend to use the model. The scoping phase is composed of two steps:

可以说，这是模型生命周期中最关键的阶段。 该阶段确定在构建模型时是否具有任何业务价值，模型的成功指标以及我们打算如何使用模型。 范围界定阶段包括两个步骤：

业务与数据理解 (Business & Data Understanding)

The idea of building a model to automate a task or to make a decision or take an action can come from a variety of different sources. It can be a ‘pull’ coming from the business to improve productivity, reduce time, or cut costs (the so called automation path) or make better decisions (the so called analytics path). It can also be a ‘pull’ from the software or IT organization that has an existing software or is building one that they want to make ‘smart’ by adding unique personalization, optimization, recommendation or a host of other characteristics offered by models.

建立模型以自动执行任务，做出决定或采取行动的想法可能来自多种不同的来源。 它可以是来自企业的“拉动”，以提高生产力，减少时间或削减成本(所谓的自动化路径)或做出更好的决策(所谓的分析路径)。 它也可以是来自具有现有软件的软件或IT组织的“拉”，或正在通过添加独特的个性化，优化，推荐或模型提供的许多其他特征来构建他们希望使其“智能”的软件。

It can also be a ‘push’ from the automation, analytics, or the AI organization to exploit the unique characteristics of machine learning models or bots. It can also be a ‘push’ from the data organization that sees value in its unique data assets that can be exploited for a competitive advantage.

它也可以是自动化，分析或AI组织的“推动力”，以利用机器学习模型或机器人的独特特征。 它也可能是数据组织的“推动力”，它看到了其独特数据资产的价值，这些价值可以被利用来获得竞争优势。

For example, the automation or AI group that has built a chatbot and proven its benefits in enhancing customer engagement in one business unit may want to build similar chatbots across multiple areas of the business. Similarly, the analytics or AI group that has built a NLP model to extract structured information in one function (e.g., finance) may push to build similar models in other functions (e.g., legal).

例如，已经构建了聊天机器人并证明其在增强一个业务部门中的客户参与方面的优势的自动化或AI小组可能希望跨业务的多个领域构建类似的聊天机器人。 类似地，已经建立了NLP模型以在一个功能(例如财务)中提取结构化信息的分析或AI小组可以推动在其他功能(例如法律)中建立类似的模型。

Irrespective of whether there is a ‘push’ or a ‘pull’ to build models here are some of the key questions that businesses must seek to answer:

无论在此处构建模型是“推”还是“拉”，企业都必须寻求回答以下一些关键问题：

1. What is the business objective — business activity, decision, or action that needs to be made more efficient or effective?
   需要使业务活动，决策或行动变得更加有效的业务目标是什么？
2. Can the business objective be achieved using existing tools, software, people, process?
   是否可以使用现有工具，软件，人员，流程来实现业务目标？
3. Do we have sufficient data that is labelled to build the model? If not, can data be annotated and collected over a period of time?
   我们是否有足够的数据被标记以构建模型？ 如果不是，是否可以在一段时间内对数据进行注释和收集？
4. What is the desired model performance criteria for business acceptance?
   商业接受所需的模型性能标准是什么？
5. How will the model be used when the business criteria are met?
   当满足业务标准时将如何使用该模型？

Answering these questions requires a combination of different groups — business, data, analytics, and software professionals. If the answer to question 2 is in the affirmative, there is no need to build a model (using ML, NLP, computer vision). If the answer to question 2 is negative then it makes sense to go further down the list of questions. If answers to 3–5 are not available the team should strive to obtain the answers before initiating the next step in the process. To really realize the full value of an AI/ML model we would argue that questions 1–5 must be answered before proceeding further — even if the answer may be revised based on further investigations of the following steps.

要回答这些问题，需要组合不同的团队-业务，数据，分析和软件专业人员。 如果对问题2的回答是肯定的，则无需构建模型(使用ML，NLP，计算机视觉)。 如果对问题2的回答是否定的，则在问题列表中进一步查找是有意义的。 如果无法获得3–5的答案，则团队应努力在开始下一步之前获得答案。 为了真正实现AI / ML模型的全部价值，我们认为在继续进行之前必须回答问题1-5，即使可以根据对以下步骤的进一步研究来修改答案。

解决方案设计 (Solution Design)

Once we have a specification of a model from the business we can go on to design the solution. Here we do not mean just a classical definition of solution where one typically looks at the IT stack or a technology tool or vendor to realize the specification of what the business wants. Solution design, to us should encompass how data, models, and software interact with each other to satisfy the needs of the business. It is not just the storage and compute infrastructure, but it should also encompass how applications deliver the business functionality as the environment changes including the continuous integration and delivery of the software, the continuous monitoring and learning of the models, and the data pipelines that feed the models.

一旦我们获得了业务模型的规范，就可以继续设计解决方案。 在这里，我们不仅意味着对解决方案的经典定义，通常人们会着眼于IT堆栈或技术工具或供应商以实现业务需求的规范。 对我们而言，解决方案设计应涵盖数据，模型和软件如何相互交互以满足业务需求。 它不仅是存储和计算基础架构，还应包括应用程序如何随着环境变化而交付业务功能，包括软件的持续集成和交付，模型的持续监视和学习以及提供的数据管道。模型。

Traditional software engineers and technical architects look at build vs buy vs lease decisions to deliver the required business functionality. With models we should also add an additional choice of assemble. There are two reasons why one should consider the assemble option. First, models typically make a prediction or recommendation or automate a specific task. A single vendor solution is likely to be too narrow to address all the needs of an enterprise. So assembling multiple models from the same platform, or multiple vendor provided models, or a combination of vendor and open-source or proprietary models is often necessary. Second, models from large ML platforms and vendor solutions get trained on data outside of your enterprise. Frequently, when these models get trained on your own dataset you are likely to see improved model performance. As a result retraining models and assembling them together is often better.

传统的软件工程师和技术架构师会着眼于构建，购买与租赁的决策，以交付所需的业务功能。 随着车型还应加上组装的额外选择。 为什么要考虑使用汇编选项有两个原因。 首先，模型通常会做出预测或推荐或使特定任务自动化。 单一供应商解决方案可能太狭窄，无法满足企业的所有需求。 因此，通常需要从同一平台组装多个模型，或者组装多个供应商提供的模型，或者组装供应商与开源或专有模型。 其次，大型ML平台和供应商解决方案中的模型接受了企业外部数据的培训。 通常，当这些模型在您自己的数据集中进行训练时，您可能会看到改进的模型性能。 因此，重新训练模型并将它们组装在一起通常会更好。

The solution design and the key questions that one should ask depends on the phase of model evolution. For a standalone model phase the emphasis is more on getting the data from other enterprise data warehouses or data lakes provisioned to the data scientists for them to demonstrate model performance. Using open-source alternatives or pre-packaged models from ML platforms, may be quick ways of determining initial feasibility. For a prediction as a service phase we need to worry about security, how frequently the models need to be retrained and redeployed, and ensuring the traceability of data, models, and software. For a model factory phase, the emphasis of solution design is much broader; the solution designers need to worry about how multiple groups of people with varying levels of expertise and skills can build, train, test, consume, and monitor models. They need to provide a secure ‘sandbox’ as well as a production infrastructure for multiple groups to operate in. Selection of the right cloud ML platform or vendor ML solutions and policies around use of open-source software become critical.

解决方案设计和应提出的关键问题取决于模型演化的阶段。 对于独立模型阶段，重点更多是从其他企业数据仓库或数据湖中获取数据，这些数据或数据湖已调配给数据科学家以演示模型性能。 使用ML平台的开源替代方案或预打包模型，可能是确定最初可行性的快速方法。 对于作为服务的预测阶段，我们需要担心安全性，需要重新训练和重新部署模型的频率以及确保数据，模型和软件的可追溯性。 对于模型工厂阶段，解决方案设计的重点要广泛得多。 解决方案设计者需要担心，具有不同专业知识和技能水平的多组人如何建立，训练，测试，使用和监视模型。 他们需要为多个小组提供安全的“沙盒”以及生产基础架构。选择合适的云ML平台或供应商ML解决方案以及有关使用开源软件的策略变得至关重要。

The popular CRISP-DM methodology splits the business and data understanding as two distinct steps. The solution design phase is what we have added as an additional step. As we have seen so far the value scoping phase is a collaborative effort between the business, IT, and data science groups. Once there is a reasonable specification of what the business wants and how the model will be used the data scientists and data engineers can move on to the next phase.

流行的CRISP-DM方法将业务和数据理解分为两个不同的步骤。 解决方案设计阶段是我们作为附加步骤添加的。 到目前为止，我们已经看到，价值界定阶段是业务，IT和数据科学团队之间的协作。 一旦有了关于业务需求以及如何使用模型的合理规范，数据科学家和数据工程师便可以进入下一阶段。

价值发现 (Value Discovery)

This phase has been well studied in data mining and data science literature. This is the phase that determines if there is likely to be any value in building and deploying a model. The data science team should be building and evaluating a number of models at any given instance. The models should be managed as a ‘portfolio’ with the expectation that a proportion of models will be able to demonstrate the performance criteria set by the business and some others will fail. As we have discussed in our earlier blog failing to treat this as a portfolio with an experimentation mindset could kill the entire AI/ML and data science endeavor.

在数据挖掘和数据科学文献中已经对该阶段进行了充分的研究。 这个阶段确定在构建和部署模型中是否可能具有任何价值。 数据科学团队应在任何给定实例上建立和评估许多模型。 应将模型作为“组合”进行管理，并期望一部分模型能够证明企业设定的绩效标准，而其他一些模型则会失败。 正如我们在之前的博客中所讨论的那样，如果不将其视为具有实验思维方式的投资组合，则可能会扼杀整个AI / ML和数据科学事业。

数据提取 (Data Extraction)

This step involves deciding the internal and external data sources that can inform the model and then obtaining the data from these sources. Additional dimensions to consider here are the variety (e.g., structure or unstructured data), volume, velocity, and veracity of data. In the unstructured data domain one needs to consider text-based data and how they will be ingested (e.g., streaming data from social media or static documents), audio data, image, video, and sensor data.

此步骤涉及确定可以通知模型的内部和外部数据源，然后从这些源中获取数据。 这里要考虑的其他维度是数据的多样性(例如，结构或非结构化数据)，数据量，速度和准确性。 在非结构化数据域中，需要考虑基于文本的数据以及如何提取它们(例如，来自社交媒体或静态文档的流数据)，音频数据，图像，视频和传感器数据。

Unlike traditional data mining we also need to consider if we need labelled data, and how we can facilitate better ‘in-process’ labelling. We have seen many data science projects falter due to lack of adequate labelled data. Creating a pipeline for data annotation, as part of the ongoing process of a domain expert, might be one of the most valuable initiatives within an enterprise. While it may not yield immediate value, it will set the stage to collect the right labelled data.

与传统数据挖掘不同，我们还需要考虑是否需要标记数据，以及如何促进更好的“过程中”标记。 由于缺乏足够的标记数据，我们已经看到许多数据科学项目步履蹒跚。 作为域专家正在进行的过程的一部分，创建数据注释管道可能是企业内最有价值的计划之一。 尽管它可能不会立即产生价值，但它将为收集正确标记的数据奠定基础。

The task of data extraction from different sources is something that falls on a data engineer and/or a data scientist. The smaller and localized the data extraction, the better it is for the data scientist to handle it on her own; as the data extraction involves enterprise data warehouse or transaction systems, the better it is for data engineers to handle such requests. As enterprises adopt this journey along the data, automation, analytics, and AI continuum their data engineers should be able to handle both structured and unstructured data.

从不同来源提取数据的任务由数据工程师和/或数据科学家负责。 数据提取的规模和范围越小，数据科学家自己处理数据的能力就越好。 由于数据提取涉及企业数据仓库或交易系统，因此数据工程师更好地处理此类请求。 随着企业在数据，自动化，分析和AI连续性方面采用这一旅程，其数据工程师应能够处理结构化和非结构化数据。

预处理 (Pre-Processing)

This step involves taking the ingested data and running some pre-processing on the data and making it ready for building machine learning models. The pre-processing steps depend on where we are obtaining the data from (i.e., internal vs external), what type of data we are processing (i.e., text, audio, image etc), and the speed at which we will be receiving the data.

此步骤涉及获取摄取的数据并对数据进行一些预处理，并使其准备好构建机器学习模型。 预处理步骤取决于我们从何处获取数据(即内部还是外部)，我们正在处理什么类型的数据(即文本，音频，图像等)以及接收数据的速度。数据。

Data cleansing is a sub-step within pre-processing. The amount of data cleansing needed depends on the quality of the data. It also varies by the type of data.

数据清理是预处理中的一个子步骤。 所需的数据清除量取决于数据的质量。 它也因数据类型而异。

For example, in text documents one needs to go through a series of pre-processing steps, such as, punctuation removal, stemming, lemmatization etc., before running through NLP tasks like sentiment analysis. In a more structured dataset the cleansing might take the form of removing NAs (fields not available) or imputing values for missing cells.

例如，在文本文档中，需要先执行一系列预处理步骤，例如标点符号去除，词干，词形修饰等，然后再执行诸如情感分析之类的NLP任务。 在结构更完整的数据集中，清除可能采取删除NA(字段不可用)或为缺失的单元格估算值的形式。

Data enrichment is another pre-processing step that is gaining more ground with all the external data sources and synthetic data sources that one has access to these days. This process requires statistically matching records from two datasets and then enriching the original dataset with additional variables.

数据充实是另一个预处理步骤，正在利用当今人们可以使用的所有外部数据源和综合数据源获得更多的基础。 此过程需要统计匹配来自两个数据集的记录，然后使用其他变量丰富原始数据集。

For example, we can take the customer data from a bank and enrich it with external information around their online behavior or purchase patterns. This allows the bank now to target their marketing campaigns better based on the channel preferences and attributes of the customer.

例如，我们可以从银行获取客户数据，并使用有关其在线行为或购买方式的外部信息来丰富这些数据。 这使银行现在可以根据渠道偏好和客户属性更好地针对营销活动。

Exploratory data analysis and feature engineering can also be considered as part of the pre-processing step. They provide useful information on what data is useful within the data collected and also what types of models need to be built.

探索性数据分析和特征工程也可以视为预处理步骤的一部分。 它们提供了有关哪些数据在收集的数据中有用以及哪些模型需要建立的有用信息。

As one moves through these steps the tasks move from a data engineer skill set to a data scientist skill set. It is often useful to have a team of at least two members to work on data science projects and also to have a blend of data engineering and data science skills, preferably with some level of domain knowledge.

随着这些步骤的完成，任务将从数据工程师技能组转移到数据科学家技能组。 通常，由至少两个成员组成的团队来从事数据科学项目，同时融合数据工程和数据科学技能(最好具有一定的领域知识)通常很有用。

建筑模型 (Model Building)

This is one of the most challenging phases of the entire discovery process. CRISP-DM and other methodologies often split the model building and model evaluation into two separate phases. However, we have found it useful to combine these two as we feel that a data scientist should be continually building and testing or evaluating their models.

这是整个发现过程中最具挑战性的阶段之一。 CRISP-DM和其他方法通常将模型构建和模型评估分为两个单独的阶段。 但是，我们发现将两者结合起来很有用，因为我们认为数据科学家应该不断构建和测试或评估他们的模型。

The techniques that can be used to build models depend on the objectives of the model, and the 4 V’s (variety, volume, velocity, veracity) of the data. The model could be for a variety of different purposes including, predicting, recommending, summarizing, etc. The criteria for model evaluation could be many as well — performance, fairness, explainability, robustness, safety, etc. In addition, it has been shown that having an ensemble of models will yield better accuracy and could be more robust. Given all these factors selecting the techniques to build models and the way one exploits certain features within the dataset to build these models is more of an art than a science. In addition, the way models are built and evaluated can also be parameterized, often called hyper-parameters. Given the breadth and depth of this step (which deserves an entire book as opposed to a single blog) we will not explore all the details here.

可以用来构建模型的技术取决于模型的目标以及数据的4 V(多样性，体积，速度，准确性)。 该模型可以用于各种不同的目的，包括预测，推荐，总结等。模型评估的标准也可以很多—性能，公平性，可解释性，鲁棒性，安全性等。此外，它已经显示拥有完整的模型将产生更高的准确性，并且可能更健壮。 考虑到所有这些因素，选择构建模型的技术以及利用数据集中的某些特征构建这些模型的方式更多的是艺术而不是科学。 此外，模型的建立和评估方式也可以参数化，通常称为超参数。 鉴于此步骤的广度和深度(值得一整本书而不是一个博客)，我们将不在这里探索所有细节。

These three steps of model discovery are iterative. In our experience, we have seen data scientists getting obsessed with these steps and strive to build better and better models with better techniques, better data, and better engineering of the data. While this may be a worthwhile academic endeavor, it could prove quite expensive and a deathknell for an enterprise data science team. Baselining models as we discussed elsewhere and time boxing modeling sprints are essential best practices that we will come back to in a future blog.

模型发现的这三个步骤是迭代的。 根据我们的经验，我们看到数据科学家沉迷于这些步骤，并努力通过更好的技术，更好的数据和更好的数据工程来构建更好的模型。 尽管这可能是值得进行的学术努力，但对于企业数据科学团队而言，这可能是相当昂贵的，并且是丧钟。 我们在其他地方讨论过的基准化模型和时间框建模冲刺是必不可少的最佳实践，我们将在以后的博客中再次介绍。

价值传递 (Value Delivery)

Organizations should enter this phase only if they feel that the model discovery phase has resulted in a model that has met the model performance criteria established by the business or is confident that it can be met when the model is deployed and is allowed to improve its performance after deployment. As enterprises have evolved in how they build and use models this phase has gained increasing importance. It has also highlighted the need for new skills like Model Operations, ML Operations, and ML engineers.

组织应该进入这个阶段，只有当他们觉得模型发现阶段已经导致该已符合企业建立的模型的性能标准或有信心可以在模型部署得到满足，并且被允许以提高其性能的模型部署后。 随着企业在构建和使用模型方面的发展，这一阶段变得越来越重要。 它还强调了对诸如模型操作，机器学习操作和机器学习工程师等新技能的需求。

模型部署 (Model Deployment)

In this step the model built by the data scientist, typically in a sandbox environment, gets deployed in a production environment. If the deployment is just ‘deploy-once-and-forget’ there is not much to do in this step. However, models by their very nature consume data and this data changes as time passes. As a result, the model that was built originally using historical data starts deteriorating in performance. The pace of deterioration really depends on how fast the environment is changing and how much data the model is consuming.

在此步骤中，通常在沙盒环境中由数据科学家构建的模型将被部署到生产环境中。 如果部署只是“一次性部署而忘了”，那么在此步骤中没有太多工作要做。 但是，模型本质上会消耗数据，并且该数据会随着时间的流逝而变化。 结果，最初使用历史数据构建的模型的性能开始下降。 恶化的速度实际上取决于环境变化的速度和模型消耗的数据量。

For example, a telco provider built a chatbot as a first-line of support to handle customer queries; if the chatbot is unable to answer the queries the chat session will be directed to a human representative. When the model was deployed after extensive training and testing, it had an excellent performance and handled 95% of all queries. However, its performance start deteriorating every week and in a few months it could only handle 80% of the queries. On further investigation it was clear that the model needed to be retrained at least on a weekly or monthly basis to account for new makes, models, and accessories of devices being released and the latest marketing campaigns of the company.

例如，一家电信运营商将聊天机器人作为支持客户查询的第一线支持。 如果聊天机器人无法回答查询，则聊天会话将定向给人类代表。 在经过大量培训和测试后部署该模型时，该模型具有出色的性能，可以处理所有查询的95％。 但是，它的性能每周都会开始下降，并且在几个月内只能处理80％的查询。 经过进一步调查，很明显，该模型至少需要每周或每月进行一次培训，以考虑所发布设备的新品牌，新模型和新配件以及公司的最新营销活动。

Model deployment is concerned with modularizing the model code (like software code) to separate out the parameters of the model, versioning the parameters, hyper-parameters, the data, and the open-source packages that went into training and testing the models, and finally automating the experimentation, training, testing, and back-testing of the models. Depending on the data that is required for the retraining one has to also consider the creation of a data pipeline to feed the retraining of the model.

模型部署涉及将模型代码(例如软件代码)模块化以分离出模型的参数，对参数，超参数，数据以及用于训练和测试模型的开源软件包进行版本控制，以及最终使模型的实验，训练，测试和回测自动化。 根据重新训练所需的数据，还必须考虑创建数据管道以提供模型的重新训练。

Given the heavy software engineering and data engineering aspects of this step, you need a mix of data science and software engineering skills. This has resulted in ML engineers and ML operations emerging as new roles to address the deployment of models on an ongoing or even continuous basis.

鉴于此步骤涉及繁重的软件工程和数据工程方面，您需要结合使用数据科学和软件工程技能。 这导致机器学习工程师和机器学习操作作为新角色出现，以解决持续甚至连续的模型部署问题。

过渡与执行 (Transition and Execution)

As an organization transitions to model deployment it needs to ensure that there is adequate governance in place to monitor the performance of the model and also its appropriate use. Most models alter the flow of work in a process or augment the decisions of a human. In either case, users have to be retrained or reskilled on how to work with models. Humans need to build confidence in the predictions or recommendations being made by the model, and at the same time also be alert to potential errors in the model. Change management and specifically the data science culture of when to trust the model and when to use human judgement becomes critical.

当组织过渡到模型部署时，它需要确保有适当的治理来监视模型的性能及其适当使用。 大多数模型会更改流程中的工作流程或增强人员的决策。 无论哪种情况，都必须对用户进行模型使用方面的培训或技能培训。 人们需要建立对模型所做的预测或建议的信心，同时也要警惕模型中的潜在错误。 变更管理，尤其是何时信任模型以及何时使用人工判断的数据科学文化变得至关重要。

In this step the model leaves the ‘tender and loving care’ of the data scientists and software engineers and gets used by different business stakeholders with varying digital or data science skills. The enterprise starts reaping the value of the models and hence there should be mechanisms for collecting and reporting on the ROI of the models.

在这一步骤中，模型离开了数据科学家和软件工程师的“温柔关怀”，并被具有不同数字或数据科学技能的不同业务涉众使用。 企业开始收获模型的价值，因此应该有收集和报告模型ROI的机制。

价值管理 (Value Stewardship)

This is the last and probably the longest phase of the process. The model has been delivered and the business is using the model — potentially embedded in other application systems. This phase needs to make sure that the value being generated is being captured and reported to the senior management on an ongoing basis and also that the value is not degenerating.

这是该过程的最后阶段，可能也是最长的阶段。 该模型已交付，企业正在使用该模型-可能嵌入其他应用程序系统中。 此阶段需要确保不断捕获正在生成的价值并将其报告给高级管理层，并且还确保该价值不会退化。

持续监控 (Ongoing Monitoring)

We have seen above as well as in our earlier blog that the value of the models could deteriorate. So we need to continuously monitor the results of the model, understand any deviations from the past, and report on the value being generated. In some regulated industry sectors, like financial services, there are stringent requirements for monitoring and reporting on models to regulators. As a result, they have extensive processes, governance and structures to govern models. However, in many of the unregulated sectors there is very little of ongoing monitoring of models. We will get back to this step when we discuss responsible AI in one of the future blogs.

在上面以及我们先前的博客中，我们已经看到模型的价值可能会下降。 因此，我们需要持续监控模型的结果，了解与过去的任何差异，并报告所产生的价值。 在某些受监管的行业部门(例如金融服务)中，对监控和向监管机构报告模型有严格的要求。 结果，他们拥有广泛的流程，治理和结构来治理模型。 但是，在许多不受监管的部门中，很少有正在进行的模型监视。 当我们在未来的一个博客中讨论负责任的AI时，我们将回到这一步。

评估和签到 (Evaluation and Check-in)

There will be a time when the useful life of a model will come to an end. This might be due to a change in the business or a change in technology or the availability of new data. As we monitor and report the value of a model or a portfolio of models we need to keep track of the overall portfolio value. We need to determine when and how we will retire or retrain or build new models. This is akin to your financial advisor selling certain stocks that are underperforming and buying certain stocks that have the future potential for growth to ensure that your portfolio value is protected or is growing.

有时模型的使用寿命将要结束。 这可能是由于业务变化，技术变化或新数据的可用性。 当我们监视和报告模型或模型组合的价值时，我们需要跟踪整体组合价值。 我们需要确定何时以及如何退休，重新培训或建立新模型。 这类似于您的财务顾问出售表现不佳的某些股票并购买具有未来增长潜力的某些股票，以确保您的投资组合价值得到保护或正在增长。

Both these steps of value stewardship should be done by the business — especially the finance organization with the help of the business sponsors, data scientists and the software organization. Also, this step should be planned when the model lifecycle starts and should not be an afterthought when the models have started deteriorating.

价值管理的这两个步骤都应该由企业来完成，特别是在财务组织的帮助下，由企业赞助商，数据科学家和软件组织来完成。 同样，应该在模型生命周期开始时计划此步骤，而不应该在模型开始恶化时再考虑。

Interestingly, the CRISP-DM methodology stops with model deployment and does not include some of the additional steps that we have outlined above. Our nine step process is illustrated below.

有趣的是，CRISP-DM方法随模型部署而停止，并且不包括我们上面概述的一些其他步骤。 我们的九步过程如下所示。

Nine Step Model Lifecycle (Souce: PwC Analysis)

九步模型生命周期(来源：普华永道分析)

迭代过程 (Iterative Process)

Although the detailed nine-step process was outlined sequentially, the process itself is far from a ‘waterfall’ approach to model development and deployment. The life cycle is iterative with three specific feedback loops.

尽管按顺序概述了详细的九步过程，但是该过程本身并不是用于模型开发和部署的“瀑布式”方法。 使用三个特定的反馈循环来迭代生命周期。

The first feedback loop is in the value discovery phase. The three steps in the process are iterative and could also result in the re-examination of the business objectives. The second feedback process is in the value delivery phase. This phase could result in going back to the value discovery phase or could even trigger a value scoping phase. The final feedback comes through the value stewardship process. This can result in going back to the value delivery, value discovery or value scoping process. The rich interplay between all these steps in the four phases results in a very complex management process for models, data, and software.

第一反馈回路处于价值发现阶段。 该过程中的三个步骤是重复的，也可能导致对业务目标的重新检查。 第二反馈过程处于价值传递阶段。 此阶段可能导致返回到价值发现阶段，甚至可能触发价值范围确定阶段。 最终反馈来自价值管理过程。 这可能导致返回到价值交付，价值发现或价值范围界定过程。 四个阶段中所有这些步骤之间的丰富相互作用导致模型，数据和软件的管理过程非常复杂。

The iterative model lifecycle process and the different roles involved in the process are captured in the diagram below.

下图捕获了迭代模型生命周期过程以及该过程涉及的不同角色。

Feedback Loops in the nine-step model lifecycle (Source: PwC Analysis)

九步模型生命周期中的反馈循环(来源：普华永道分析)

Although, we have detailed the model lifecycle process and its iterative nature we have not compared it with the agile software development process. This we will do in the next blog.

虽然，我们已经详细描述了模型生命周期过程及其迭代性质，但我们并未将其与敏捷软件开发过程进行比较。 我们将在下一个博客中做到这一点。

Authors: Authors: Anand S. Rao, Joseph Voyles and Shinan Zhang

作者：作者： Anand S. Rao ， Joseph Voyles和Shinan Zhang

翻译自: https://towardsdatascience.com/model-lifecycle-from-ideas-to-value-14e654b7d4a4

生命周期模型