AI Safety 之 AI Standard

Part1：Standards about AI and functional safety

1. AI General standards

ISO 22989 - Information technology — Artificial intelligence — Artificial intelligence concepts and terminology

• Provides standardized concepts and terminology to help AI technology to be better understood and used by a broader set of stakeholders.
• Including:
  Chapter 5 AI concepts
  Chapter 6 AI system life cycle
  Chapter 7 AI system functional overview
  Chapter 8-AI ecosystem
  Chapter 9-Field of AI
  Chapter 10-Application of AI systems

ISO/IEC JTC1 TR 5469 on “Artificial intelligence — Functional safety and AI systems”

• Now the Draft technical report stage.
• The purpose of this document is to enable the developer of safety-related systems to appropriately apply AI technologies as part of safety functions by fostering awareness of the properties, functional safety risk factors, available functional safety methods and potential constraints of AI technologies.
• This document also provides information on the challenges and solution concepts related to the functional safety of AI systems.

2. Automotive/Road vehicles

• ISO 26262-6, Road vehicles — Functional safety — Part 6: Product development at the software level
  The SW development process to avoid the systematic failures.
• ISO 21448:2022, Road vehicles — Safety of the intended functionality – Annex D.2–“Implications for Machine Learning”
• ISO/TR 4804:2020, Road vehicles — Safety and cybersecurity for automated driving systems — Design, verification and validation. --Annex B–Using deep neural networks to implement safety-related elements for automated driving systems.
  • The aim of this annex is to provide an overview of the challenges for achieving and assuring the safety of DNNs in automated driving, propose solutions that address the safety challenges, and conduct a brief survey on the current state-of-the-art regarding these challenges.
  • To convert Safety First White Paper into an ISO/TR creating an early 1st edition to worldwide address this field by an ISO standardization activity.
• ISO/CD TS 5083, Road vehicles — Safety for automated driving systems — Design, verification and validation
  ISO TS 5083 evolves from ISO TR 4804, and go for necessary enhancements and extensions to cover scope in width and depth.
• ISO/CD PAS 8800, Road Vehicles — Safety and artificial intelligence
  The purpose of this document is to provide industry-specific guidance on the use of AI and, in particular, ML-based systems involved in safety-related functions of road vehicles.

Relationship between AI related standards

Part 2： ISO 8800 Road Vehicles - Safety and artificial intelligence

ISO 8800 current status

仍在开发阶段，暂时还未正式发布，预计24年正式发布，目前手里的也是行业专家review版本，很多内容处于Work in Progress，待完善

ISO 8800 structure & overview

Clause 1：Scope
Clause 2：Normative reference
Clause 3：Terms and definitions
Clause 4：Abbreviations
Clause 5：Relation to other safety and AI related standards
Clause 6：AI within the context of road vehicles systems safety engineering 车辆系统中的AI系统，定义了AI系统的参考架构及风险行为的因果模型，以及与ISO 26262及ISO 21448之间的关系
Clause 7：Safety lifecycle for AI systems
Clause 8：Derivation of safety requirements on AI systems
Clause 9：Selection of AI-Measures and design-related considerations
Clause 10：Data-related considerations
Clause 11：Verification and validation of the AI system
Clause 12：Risk evaluation and safety analysis
Clause 13：Assurance arguments for AI systems
Clause 14：Measures during operation and continuous assurance
Clause 15：Confidence in use of AI-related software tools
Annex A: Example assurance argumentation structure for an AI-based vehicle function (Work-in-Progress)
Annex B：ISO 26262-6:2018 Gap Analysis for ML
Annex C: Detailed considerations on safety-related properties (Work-in-Progress)

AI技术尤其是ML在汽车行业应用越来越广泛，但是关于AI及ML相关的指导规范仍然缺失，如，详细规范，设计，V&V方案等；

本标准的目的也是为了AI在汽车行业的应用提供指导规范，不局限于ML；

Clause5 Relation to other safety and AI related standard

1. This document provides specific guidance for providing an assurance argument that a tolerable level of residual risk has been achieved for the use of AI in safety-related vehicle functions.|
   Assurance argument关于在安全相关的车辆功能中使用AI，已经达到了可容忍的残余风险水平，本标准提供了具体的指导；

2. 与车辆安全相关的标准的关系：同样关注于两个方面：

• 功能安全相关的风险–参考ISO26262：通过提供相关条款的解释来实现；– Clause 6.4
• 性能局限–参考ISO21448 ：提出一个因果模型causal model来提出相应的安全需求及风险消除措施；-- Clause 6.5
• 本标准主要关注于指导如何将TSR转化成AI系统的安全相关的需求，聚焦于设计design及实施operation阶段

3. 与ISO 5469

• ISO TR5469提供了通用的指导指南，关于AI 技术作为安全功能的一部分的应用，没有局限于具体的行业类别；
• ISO TR 5469提供了分类方案（Classification schemes）来确定AL/ML功能的安全需求；包括usage level 和AI technology class；
• ISO TR 5469提供了Class I, Class II, Class III 三种类别，在ISO 8800中主要提供具体的指导规范关于Class II，对应的Usage level是 A，B1，B2，C1，C2；
• General classification scheme for applicability of AI in the safety related E/E/PE system: (From ISO TR 5469)