Why Software Development Methodologies Suck

原文

Published: Aug 2, 2012

There’s a lot of dogma in the religious wars around software development practices and methodologies. Are phase-gate methodologies effective at managing the risk of software development, or just risk management kabuki? Does TDD really make for higher quality software? Is pair programming a superior replacement for code review or just a way to inflate consulting rates? I’m going to argue that while scientific evidence to decide these claims is lacking, there are two general principles which can help us choose good practices while at the same time improving the value of the software we deliver: reduce cycle time and increase feedback.

Michael Feathers makes the following observation:

I think that, in the end, we just have to accept that developer skill is a far more significant variable than language choice or methodological nuances1. Frankly, I think we all know that, but we seem to suffer from the delusion that they are the primary knobs to tweak. Maybe it’s an extension of the deeply held view that from an economic viewpoint, it would be ideal if people were interchangeable.

The problem is, how do we get skilled developers? Since the concept of individual productivity in IT has never been satisfactorily defined, this is a particularly hard problem to solve. Lines of code – still a popular measure – suffers from the devastating flaw that a line of code is a liability, not an asset as is often thought. Measuring number of hours worked encourages heroic behavior – but experience shows that the “heroes” are usually the same people that cause projects to become late through taking unacceptable risks early on, and working long hours makes people stupid and leads to poor quality software. There is still no generally accepted set of professional standards or chartering system for IT professionals, and recruiting good people is very much an art rather than a science.

Psychologists have at least addressed the problem of why it is so difficult to acquire and measure skill in IT. As Daniel Kahneman says in Thinking Fast and Slow, there are “two basic conditions for acquiring a skill: an environment that is sufficiently regular to be predictable; [and] an opportunity to learn these regularities through prolonged practice.”

But traditional software projects are the opposite of a regular, predictable environment. The only good measure of success of a project – did the end result create the expected value over its lifetime? – is so distant from the critical decisions that caused that success or failure that it’s rare for anybody from the original team even to be present to get the feedback. It’s practically impossible to determine which of those decisions led to success or failure (in artificial intelligence, this is known as the credit-assignment problem).

These factors make it very hard for IT professionals to acquire the skills that lead to successful products and services. Instead, developers acquire the skills that allow them to most efficiently reach the goals they are incentivized by – usually declaring their work “dev complete” as rapidly as possible irrespective of whether the functionality is integrated and production-ready – and similar problems arise in other functional areas too.

The fact that software projects are complex systems rather than regular environments leads to another problem – the extreme difficulty of gathering data on which techniques, practices, and methodologies are actually effective, and the near impossibility of generalizing this data outside the context in which it was gathered.

In his excellent book The Leprechauns of Software Engineering Laurent Bossavit executes a devastating attack on software development folklore such as the “cost of change” (or “cost of defects”) “curve”, the claim that the variance in developer productivity is an order of magnitude, the idea of the cone of certainty, and many other cornerstones of methodological lore in software development. He shows that these theories – and many others – depend on very small sets of data that are gathered either from informal experiments run on computer science students, or projects which cannot possibly have been effectively controlled. The organization of the studies that form the basis of these claims is often methodologically unsound, the data poorly analyzed, and – most egregiously – the findings generalized well beyond their domain of applicability2.

As a result, it’s not possible to take seriously any of the general claims as to whether agile development practices are better than waterfall ones, or vice-versa. The intuitions of “thought leaders” are also a poor guide. As Kahneman says, “The confidence that people have in their intuitions is not a reliable guide to their validity… when evaluating expert intuition you should always consider whether there was an adequate opportunity to learn the cues, even in a regular environment.” As Ben Butler-Cole points out in his companion post, “why software development methodologies rock”, the very act of introducing a new methodology can generate some of the results the adopters of the methodology intend to bring about.

You might think that puts us in an impossible position when it comes to deciding how to run teams. But consider why software development is not a regular environment, and why it is so hard to run experiments, to acquire skills, and to measure which practices and decisions lead to success, and which to failure. The root cause in all these cases – the reason the environment is not regular – is that the feedback loop between making a change and understanding the result of that change is too long. The word “change” here should be understood very generally to mean change in requirements, change in methodology, change in development practices, change in business plan, or code or configuration change.

There are many benefits to reducing cycle time – it’s one of the most important principles that emerges when we apply Lean Thinking to software development. Short cycle times are certainly essential for creating great products: as Bret Victor says in his mind-blowing video Inventing on Principle, “so much of creation is discovery, and you can’t discover anything if you can’t see what you’re doing.”

But for me this is the clincher: It’s virtually impossible for us to practice continuous improvement, to learn how to get better as teams or as individuals, and to acquire the skills that enable the successful creation of great products and services – unless we focus on getting that feedback loop as short as possible so we can actually detect correlations, and discern cause and effect.

In fact, the benefits of having a short cycle time from idea to feedback are so important that they should form one of the most important criteria for your business model. If you have to decide between creating your product as a user-installed package or software-as-a-service, this consideration should push you strongly in the direction of software-as-a-service (I speak from experience here). If you’re building a system which involves hardware, work out how you can get prototypes out as quickly as possible, and how you can modularize both the hardware and the software so you can update them fast and independently. 3D printing is likely to make a huge impact in this area since it allows for the application of software development practices to the evolution of hardware systems. Working in cross-functional teams is more or less a requirement if you want to achieve a sufficiently short cycle time.

Software methodologies – even the “hire a bunch of awesome people and let them self-organize” methodology – suck because they so often lead to cargo-cult behaviour: we’re doing stand-ups, we have a prioritized backlog, we’re even practicing continuous integration for goodness’ sake – why is the stuff we make still shitty and late? Because you forgot the most important thing: building an organization which learns and adapts as fast as possible.

1 Although as Laurent Bossavit points out (private communication) “A developer’s skill is in part the method he/she knows and his/her reasons for preferring one language over another.”

2 I am not suggesting that we give up on running experiments to learn more about what works and what doesn’t in software development, and the contexts in which such claims are valid – quite the contrary, I’m saying we’re not trying nearly hard enough.

This post is from Continuous Delivery by Jez Humble. Click here to see the original post in full.

翻译

   围绕软件开发实践和方法的宗教战争中有很多教条。阶段门方法在管理软件开发的风险方面有效吗，还是仅仅是风险管理歌舞伎？TDD真的有利于更高质量的软件吗？结对编程是代码审查的高级替代品，还是只是提高咨询率的一种方式？我要说的是，虽然缺乏科学证据来决定这些主张，但有两个一般原则可以帮助我们选择好的实践，同时提高我们交付的软件的价值:减少周期时间和增加反馈。
   迈克尔·费哲做了如下观察:
   我认为，最终，我们不得不承认开发人员的技能是一个比语言选择或方法上的细微差别重要得多的变量。坦率地说，我想我们都知道这一点，但我们似乎有一种错觉，认为它们是调整的主要旋钮。也许这是一种根深蒂固的观点的延伸，即从经济的角度来看，如果人们可以互换，那将是理想的。
   问题是，我们如何获得熟练的开发人员？由于信息技术中个人生产率的概念从未得到令人满意的定义，这是一个特别难以解决的问题。代码行——仍然是一种流行的衡量标准——存在一个毁灭性的缺陷，即代码行是一种负债，而不是人们通常认为的资产。衡量工作时间鼓励英雄行为——但经验表明，“英雄”通常是那些通过在早期冒不可接受的风险而导致项目延迟的人，长时间工作会让人变得愚蠢，并导致软件质量低下。对于信息技术专业人员来说，仍然没有一套普遍接受的专业标准或包租系统，招募优秀人员与其说是一门科学，不如说是一门艺术。
   心理学家至少解决了为什么获取和衡量信息技术技能如此困难的问题。正如丹尼尔·卡内曼在《快速思考和慢速思考》中所说，“获得一项技能有两个基本条件:一个足够有规律、可以预测的环境；[和]通过长期练习学习这些规律的机会。”
   但是传统的软件项目与常规的、可预测的环境相反。一个项目成功的唯一好的衡量标准——最终结果是否在其生命周期内创造了预期价值？–距离导致成功或失败的关键决策如此之远，以至于原始团队中的任何人都很少到场获得反馈。几乎不可能确定这些决定中的哪一个导致了成功或失败(在人工智能中，这被称为信用分配问题)。
   这些因素使得信息技术专业人员很难获得导致成功产品和服务的技能。相反，开发人员获得的技能使他们能够最有效地实现他们所激励的目标——通常是尽可能快地宣布他们的工作“开发完成”，而不管功能是否集成和生产就绪——类似的问题也会出现在其他功能领域。
   事实上，软件项目是复杂的系统，而不是常规环境，这导致了另一个问题——收集技术、实践和方法实际有效的数据极其困难，以及在收集数据的环境之外推广这些数据几乎是不可能的。
   在他的优秀著作《软件工程的妖精》中，洛朗·博萨维特对软件开发民间传说进行了毁灭性的攻击，如“变化的成本”(或“缺陷的成本”)“曲线”，声称开发人员生产率的变化是一个数量级，确定性锥的思想，以及软件开发方法学的许多其他基石。他表明，这些理论——以及许多其他理论——依赖于非常小的数据集，这些数据集要么是从对计算机科学学生进行的非正式实验中收集的，要么是从不可能得到有效控制的项目中收集的。构成这些主张基础的研究组织通常在方法上不合理，数据分析不佳，最令人震惊的是，这些发现远远超出了它们的适用范围。
   因此，不可能认真对待关于敏捷开发实践是否比瀑布开发实践更好的任何一般说法，反之亦然。“思想领袖”的直觉也是一个糟糕的向导。正如卡尼曼所说，“人们对直觉的信心并不能可靠地指导他们的有效性……在评估专家直觉时，你应该始终考虑是否有足够的机会学习线索，即使是在正常的环境中。”正如本·巴特勒-科尔在他的配套文章《为什么软件开发方法会失败》中指出的那样，引入一种新方法的行为可以产生该方法的采纳者想要带来的一些结果。
   你可能会认为，在决定如何管理团队时，这让我们处于一个不可能的境地。但是考虑一下为什么软件开发不是一个常规的环境，为什么运行实验、获取技能以及衡量哪些实践和决策导致成功，哪些导致失败是如此困难。所有这些情况的根本原因——环境不规则的原因——是做出改变和理解改变的结果之间的反馈回路太长。这里的“变更”一词应该理解为需求的变更、方法的变更、开发实践的变更、业务计划的变更、代码或配置的变更。
   缩短周期时间有很多好处——当我们将精益思想应用于软件开发时，这是最重要的原则之一。短周期时间对于创造伟大的产品当然是至关重要的:正如布雷特·维克多在他令人惊叹的视频《发明原理》中所说，“如此多的创造是发现，如果你看不到你在做什么，你就不能发现任何东西。”

   但对我来说，这是决定性的:对我们来说，实践持续改进，学习如何作为团队或个人变得更好，以及获得能够成功创造伟大产品和服务的技能几乎是不可能的——除非我们专注于让反馈回路尽可能短，以便我们能够实际检测相关性，并辨别因果关系。
   事实上，从想法到反馈周期短的好处是如此重要，以至于它们应该成为你的商业模式最重要的标准之一。如果您必须决定是将您的产品创建为用户安装的软件包还是软件即服务，这种考虑应该会将您推向软件即服务的方向(我在此根据经验发言)。如果您正在构建一个包含硬件的系统，请找出如何尽快获得原型，以及如何模块化硬件和软件，以便快速独立地更新它们。3D打印可能会在这一领域产生巨大的影响，因为它允许软件开发实践应用于硬件系统的发展。如果你想实现足够短的周期时间，在跨职能团队中工作或多或少是一个要求。
   1尽管正如洛朗·博萨维特(私人交流)所指出的，“开发人员的技能部分是他/她知道的方法和他/她喜欢一种语言而不是另一种语言的原因。”

2我并不是建议我们放弃运行实验，去了解软件开发中什么可行，什么不可行，以及这种说法有效的环境——恰恰相反，我是说我们还不够努力。