What Project Management Practices Lead to Success?

From June M. Verner, William M. Evanco.  What Project Management Practices Lead to Success? IEEE SOFTWARE 2005

conslusion:

Requirements elicitation and management 

Good project management necessitates complete, consistent requirements.6 Although gathering requirements with a specific methodology (R1) was significantly associated with  requirements being complete and accurate at  the project’s start (R2), it wasn’t significantly associated with project success. However, in  54 percent of our projects, respondents didn’t know what requirements methodology their project’s systems analysts used. Again, this didn’t surprise us, because most respondents had no interaction with the systems analysts.

Requirements-gathering methods. 

Among the  46 percent of respondents who knew about requirements gathering, four projects used prototyping and nine used JAD (joint application design) sessions with prototyping. Eleven of these 13 projects were successful. Interviews and focus groups were the remaining projects’ main requirements-gathering methods. Eight projects used UML to document requirements, but only three of these were successful. Practitioners commented that there were “too many new things without a pilot” and “unfamiliar methods.” However, the failed UML projects had other problems such as poor estimates and no risk management, so their failures weren’t necessarily due to using UML.

Managing changing requirements. 

Nearly half the projects began with incomplete requirements (R2); predictably, the scope changed for many of these projects during development (R6). A 2 test of R2 with R6 was significant. The scope was more likely to change for larger projects. Given that an organization needs control over the requirements function to advance from the lowest CMMI (Capability Maturity Model Integration, see www.sei.cmu. edu/cmmi) level, it’s obvious that many of our sample’s organizations are still at that level. These results agree with a survey Colin J. Neill and Philip A. Laplante conducted in 2002,7 whose respondents thought that their companies didn’t do enough requirements engineering. The number of projects that began with poor requirements suggests that these organizations should develop their projects using methodologies designed to deal with unclear requirements. However, this isn’t the case.

The importance of requirements management. 

Our results, shown in Table 1, demonstrate that requirements continue to be an enormous problem for IS development and one of the most common causes of runaway projects.8 Consistent with Robert Glass’s observations,9 we found that good requirements (R4) that were complete and accurate at the project’s start (R2) with a well-defined project scope (R5), resulting in well-defined software deliverables (R9), were positively associated with project success. The importance of user involvement in  requirements gathering (R7) supports observations by Glass9 and Carl Clavedetschers.10 

Although Barry Boehm includes a “continuing stream of requirements changes” in his top 10 risk items,11 we didn’t find an association between changing the scope during the project (R6) and project success. Instead, we found that if requirements were initially incomplete, completing them during the project (R3) was positively associated with success, as was being able to manage requirements and any changes to them through a central repository (R8). Only 60 percent of our projects used a central repository, supporting Clavedetscher’s suggestion that software developers “fail to use requirements management to surface (early) errors or problems. 

When a project’s size impacted requirements gathering (R10), project success was negatively influenced. This result agrees with another of Glass’s observations,9 suggesting that large projects hamper requirements gathering and lead to unclear, incomplete, and potentially unstable requirements.  Using logistic regression, we found that the best risk management predictor of project success was R4 (the requirements were good), which predicted 89 percent of successes, 58 percent of failures, and 78 percent correctly overall. 

Cost and effort estimation and scheduling

Good cost and schedule estimates (C4) affect project success.5 As early as 1975, Frederick Brooks stated that more projects have gone awry for lack of calendar time than from all other causes combined.12 Optimistic estimation is still one of the two most common causes for runaway projects,8 with cost and schedule failures exceeding any other kind of software failures in practice.13 Boehm includes unrealistic schedules and budgets in his top 10 risk items.11 However, having a schedule (C3) wasn’t associated with project success; not having a schedule means you don’t have to meet one.

Who’s making the estimates?

We initially assumed that the PM would be involved in deciding the delivery date because he or she would likely know the project’s technology, participants, and development practices better than anyone else. This assumption was mainly incorrect. Higher-level management, marketing, or the customer or user generally made
initial estimates. The PM was involved in making initial cost and effort estimates for only 33 percent of the projects (C1), but responses to this question weren’t associated
with project success. PMs were able to negotiate the schedule in just under half (48 percent) of the projects where they weren’t included in the initial delivery date or budget decisions. 

In these organizations, the wrong people are making the estimates. This observation agrees with Glass.9 When the PM had no say in project estimates, our respondents indicated that only 31 percent of those estimates were good. Given the lack of PM participation in estimation, it’s understandable that only about half of the projects’ delivery dates were made with appropriate requirements information (C2). Our result agrees with Glass8 that most software estimates are performed at the beginning
of the life cycle before the requirements phase and thus before the problem is understood. It’s noteworthy that 76 percent of the estimates considered to be above average had some kind of PM input.

Estimate accuracy. 

Seventy-four percent of the projects were underestimated, 36 percent were accurately estimated, and no projects were overestimated. Overall, respondents thought
that 38 percent of the estimates were either poor or very poor (33 percent of these projects were successful), 27 percent were average (66 percent successful), and 35 percent were either good or very good (86 percent successful). Many estimates initially thought to be of average quality were underestimates. Developers were more likely to contribute to project estimates when the requirements were incomplete, and developer involvement in making the estimates (C5) was negatively associated with project success. Furthermore, developers might not have a global perspective of the project, which could handicap them in producing projectwide estimates. 

Project size was significantly related to whether a project had a schedule. Larger projects were more likely to have a schedule but were also more likely to have inadequate staffing levels, staff added late, and estimates that didn’t take staff leave into account. Finally, adding staff late to meet an aggressive schedule (C7) was negatively related to project success, in agreement with Brooks’s findings in The Mythical Man Month .

Using logistic regression, we found that C2 (making delivery decisions with the appropriate requirements information) was the best C predictor of project outcomes, predicting 73 percent of successes, 83 percent of failures, and 77 percent correctly overall. 

Risk assessment and postmortem reviews

Unfortunately, most developers and PMs perceive risk management processes and activities as creating extra work and expense, and risk management is the least-practiced project management discipline.14 Respondents indicated that 33 percent of the projects had no risks, even though 62 percent of these projects failed. As Tom DeMarco and Timothy Lister noted, “if a project has no risks, don’t do it.”5 Large projects were significantly less likely to have risks incorporated into their project
plans. Active risk management characterizes software project management quality,6 and managing risks throughout the project (A3) was significantly associated with project success. 5 The association between responses to questions A1, A2, and A3 and the PM being above average (M1) supports this observation 

Using logistic regression, we found that A3 (risks were managed throughout the project)  predicted project success the best, predicting 60 percent of successes, 80 percent of failures, and 69 percent correctly overall. 

Postmortem reviews are important for process improvement,8 but companies seldom perform them. As a result, they tend to repeat the same mistakes project after project.1,15 Few organizations conducted project postmortems, and those that conducted them didn’t do so consistently (for our first 42 projects from the same organization, only 33 percent had postmortem reviews). Holding postmortem reviews (P1) was significantly associated with good requirements (R4) and managing risks

throughout the process (A1, A2, and A3).