软件质量2010年： 关于现有软件技术状况的调查（U.S）

由于全文是英文PPT，我把关键的要点转帖上来。

 

 

SOFTWARE QUALITY IN 2010:

A SURVEY OF THE STATE OF THE ART 

Author：Capers Jones

Founder and Chief Scientist Emeritus 

数据来源：

Data collected from 1984 through 2010

• About 675 companies (150 clients in Fortune 500 set)

• About 35 government/military groups

• About 13,500 total projects

• New data =  about 50-75 projects per month

• Data collected from 24 countries

• Observations during more than 15 lawsuits 

软件行业类型与弊病（已省略）

INDUSTRY HAZARD

Airlines Safety hazards

INDUSTRY HAZARD

Defense Security hazards

INDUSTRY HAZARD

Finance Financial transaction hazards

INDUSTRY HAZARD

Health Care Safety hazards

INDUSTRY HAZARD

Insurance Liability, benefit hazards

INDUSTRY HAZARD

State, Local Governments Local economic hazards

INDUSTRY HAZARD

Manufacturing Operational hazards

INDUSTRY HAZARD

National Government Citizen record hazards

INDUSTRY HAZARD

Public Utilities Safety hazards

INDUSTRY HAZARD

Telecommunications Service disruption  hazards

美国平均软件质量

U.S. AVERAGES FOR SOFTWARE QUALITY

Defect Removal Delivered

Defect Origins Potential Efficiency Defects

Requirements 1.00 77% 0.23

Design 1.25 85% 0.19

Coding 1.75 95% 0.09

Documents 0.60 80% 0.12

Bad Fixes 0.40 70% 0.12

TOTAL 5.00 85% 0.75

最好的软件质量

BEST IN CLASS SOFTWARE QUALITY

   Defect  Removal  Delivered

Defect Origins  Potential  Efficiency  Defects

  Requirements 0.40 85% 0.08

Design 0.60 97% 0.02

Coding 1.00 99% 0.01

Documents 0.40 98% 0.01

Bad Fixes 0.10 95% 0.01

TOTAL 2.50 96% 0.13

 

通常在通过CMMI 3级水平系统软件公司发现

 

Most often found in systems software > SEI CMM Level 3

 

 

差的软件质量

 

POOR SOFTWARE QUALITY - MALPRACTICE

 

 

Defect  Removal  Delivered

Defect Origins  Potential  Efficiency  Defects

 

 

Requirements 1.50 50% 0.75

Design 2.20 50% 1.10

Coding 2.50 80% 0.50

Documents 1.00 70% 0.30

Bad Fixes 0.80 50% 0.40

TOTAL 8.00 62% 3.05

 

 

通常在大型C/S项目发现（大于5000功能点）

Most often found in large client-server projects (> 5000 FP).

上述所有包含以功能点作为单位的所有软件缺陷源

(Function points show all defect sources - not just coding defects)

 

各类软件类型的质量情况

 

	System  Software	Commercial  Software	Information  Software	Military  Software	Outsource  Software
Defect Potentials	6.0	5.0	4.5	7.0	5.2
Defect Removal	94%	90%	73%	96%	92%
Efficiency Delivered Defects	0.4	0.5	1.2	0.3	0.4
First Year Discovery Rate	65%	70%	30%	75%	60%
First Year Reported  Defects	0.26	0.35	0.36	0.23	0.30
	Web Software	Embedded  Software	SEI-CMMI3  Software	SEI-CMM1  Software	Overall average
Defect Potentials	4.0	5.5	3.0	5.5	5.1
Defect Removal	72%	95%	95%	73%	86.7%
Efficiency Delivered Defects	1.1	0.3	0.15	1.5	0.68
First Year Discovery Rate	95%	90%	60%	35%	64.4%
First Year Reported  Defects	1.0	0.27	0.09	0.52	0.43

 

 

 

 

良好质量结果帮助90%的提升

 

GOOD QUALITY RESULTS > 90% SUCCESS RATE

 

 

• Formal Inspections (Requirements, Design, and Code)

•   Static analysis (for about 25 languages out of 2,500 in all)

• Joint Application Design (JAD)

• Software Six-Sigma methods (tailored for software projects)

• Quality Metrics using function points

• Quality Metrics using IBM’s Orthogonal classification

• Defect Removal Efficiency Measurements

• Automated Defect tracking tools

• Active Quality Assurance (> 5% SQA staff)

• Utilization of TSP/PSP approaches

• => Level 3 on the SEI capability maturity model (CMMI)

• Virtualization for reuse and debugging

• Quality Estimation Tools

• Automated Test Support Tools + testing specialists

• Root-Cause Analysis

实际可操作的软件质量定义

A PRACTICAL DEFINITION OF SOFTWARE 

QUALITY 

 

•         Low Defect Potentials (< 2.5 per Function Point)

 

•         High Defect Removal Efficiency (> 95%)

 

•         Unambiguous, Stable Requirements (< 2.5% change)

 

•         Explicit Requirements Achieved (> 97.5% achieved)

 

•         High User Satisfaction Ratings (> 90% “excellent”)

 

       - Installation

 

       - Ease of learning

 

       - Ease of use

 

       - Functionality

 

       - Compatibility

 

       - Error handling

 

       - User information (screens, manuals, tutorials)

 

       - Customer support

 

       - Defect repairs

 

 

 

 

Quality Measurements Have Found:

 

 

 

• Individual programmers -- Less than 50% efficient  in finding bugs in their own software

• Normal test steps -- often less than 75% efficient  (1 of 4 bugs remain)

• Design Reviews and Code Inspections -- often more than 65% efficient; have topped 90%

• Inspections, static analysis, virtualization, plus formal testing – are often more than 95% efficient; have hit 99%

• Reviews, Inspections, static analysis, and virtualization  -- lower costs and schedules by as much as 30 %

 

 

 

 

• 1) Requirements:      Hardest to prevent and repair

• 2) Design:      Most severe and pervasive

• 3) Code:      Most numerous; easiest to fix

• 4) Documentation:   Can be serious if ignored

• 5) Bad Fixes:      Very difficult to find

• 6) Bad Test Cases:   Common and troublesome

• 7) Data quality:      Common but hard to measure

• 8) Web content:      Unmeasured to date

 

 

Severity 1:    TOTAL FAILURES               1% at release

Severity 2:    MAJOR PROBLEMS          20%  at release

Severity 3:    MINOR PROBLEMS          35%  at release

Severity 4:    COSMETIC ERRORS         44%  at release

 

INVALID       USER OR SYSTEM ERRORS    15% of reports

DUPLICATE  MULTIPLE REPORTS        30% of reports

ABEYANT     CAN’T RECREATE ERROR      5% of reports