Adult Behavior on Projects

My Advice: If there is only one piece of advice I could give to everyone in the software development business these days, it is to experiment seriously with the family of techniques that have come to be called the Agile Methods. We are all in the high-risk system development and delivery business, and these methods fit our real world.


Tim Lister

The Agile Methods arose in the second half of the 1990's, and Addison Wesley published their seminal book, Extreme Programming Explained: Embrace Change, by Kent Beck, in 2000. I come to this advice from consulting and writing in another area, software project risk management. My partner, Tom DeMarco, and I have written Waltzing With Bears: Managing Risk on Software Projects. Viewing the Agile Methods from the risk management perspective gives them valuable context. I've often said that risk management is project management for adults, and now I'll add that the Agile Methods address many of the most common software risks.

A Few Words on Risk Management: Risk Management is not risk avoidance when it comes to software projects. All the software projects with high value and low risks were done decades ago. Almost by the simple definition of your mission, software efforts at your organization are going to be high risk. To avoid the risk on these projects one often ends up devaluing the system, which undermines the decision to try to build the system in the first place. Sadly, many organizations believe the following diagram that comes from the CMM is true.

What worries me here is the notion that risk completely evaporates as one gets more adept at building software. I would agree with the S.E.I. that as your software processes improve you lower a certain class of risk, that of unpredictability of schedule and outcome due to variation by groups and individuals.

The risks don't go away as we improve, the bar, as in high jumping, just gets higher. In 2005, the expectations for software projects have never been higher: more, faster, with less. Raise expectations and you raise risk. It is just that simple.

You also cannot directly control many risks on a software project. For example, you may have a dependency on the timely delivery for a key piece of the system from a vendor or a sub-contractor. You can monitor their progress, and you can make contingency plans for late, or heaven forbid, non delivery of their product, but there is no way you can remove the risk of late delivery impacting your critical path from the realm of possibility. It's never delivered until it's delivered.

A Journey To...Someplace: I think of software projects as journeys into the uncertain. As we begin we may not know exactly what we are to build, we may not know what is the most appropriate technology to build it with, and therefore we aren't sure what skills we'll need, and add that up and we are very unsure of how long a journey we will take. The odd thing is that many people are very certain of when we will be done. The deadline is very often set before the system is specified. At face value, this is comical, but what is happening is that the organization is trying rather clumsily to set a project goal or a project constraint. I think that there is a lovely analogy to the journey of a software project. It is:

hurricane isabel Here our project is Hurricane Isabel. We know exactly where it is now, on September 5, 2003, at 11 AM EDT. As we look into the future our project could speed up, slow down, veer east or west, so as we look out three days we see landfall somewhere between South Carolina and the Delmarva Peninsula. Not too precise. I can hear management urging the forecasters to tell them exactly when and where it will make landfall. My point is that The National Weather Center is very explicit about their uncertainty here. We only get into trouble by not declaring our own uncertainties and risks.

A Risk is: My working definition of a risk is any variable on your project that within its normal distribution of possible values, could take on a value that is detrimental or even fatal to your project. Here is a risk curve:

risk curve

So if you have a drop-dead delivery date of July 1, you have risk. It's not a problem, you have about a 50% chance of delivering by then, but you have a 50% chance of missing the date.

The Good News About Software Project Risk: The good news is that as a consultant for the past 30 years, I've never met an unlucky project. That is I have never found a project that had a catastrophic failure that was unforeseeable. There are always risk signs detectable by project members long before the problem materializes. For example, no software manager can claim that they awoke one morning and came to the sudden realization that the project was six months behind schedule. As the saying goes:

Q: "How does a project get to be six months late?"

A: "One day at a time."

Software risks are so commonplace that there is actually a very useful software risk taxonomy from the S.E.I. Go to: www.sei.cmu.edu/publications/documents/93.reports/93.tr.006.html and you can download the entire technical report.

What Makes The Agile Methods So Valuable For High Risk Projects: From my perspective the Agile Methods have inculcated some risk mitigation strategies for common risks into their techniques.

  • Schedule risk in our business is huge. The majority of projects either delivers later than desired, or delivers on time with less functionality than expected. The goal-deadline is most often set before the actual system definition is agreed upon. The Agile Methods attack this risk through tight iterative development. In XP for instance, iterations are most often about two weeks, and the delivery is tested code. The client, along with the project team, decides what is in each iteration. The project continues to iterate with demonstrable progress at the end of each one until the client accepts the product. If the deadline-goal turns out to be unattainable, it is recognized early since the progress is visible and can be mapped to the plan.
  • Integration risk, the likelihood that pieces of a system will not fit together smoothly and therefore will require re-work, basically disappears with the Agile Methods. Iterations are tight, and builds are often daily. Interfaces can be tested and verified long before all the code for any one component is written.
  • Quality of product risk, the likelihood that the code will have an unacceptable defect density, again gets attacked by the Agile Methods. Rather than attempting to catch bugs through a costly (and deadly boring) review process, XP moves to pair programming where two programmers sit and team together to produce a piece of code. Reviewing is so important that it becomes an organic part of writing code.
  • Specification risk, the chance that what is currently specified is not complete or not acceptable to some of the client community, is minimized by two actions. First, the project life cycle is truly iterative. No spec is ever "done" before construction starts; only the details needed to build the next iteration are spelled out. This allows lots of time for some parts of the system to be agreed to before their iteration comes up. Second, the client is expected to be onboard, participating face-to-face with the development team on a daily basis. Specifying becomes a partnership between analysts and client representatives. If the clients can't decide, it won't make that iteration.
  • Test/Validation risk, the chance that test has not revealed some serious product defects gets minimized by the iterative and cumulative nature of the Agile - Test Driven Development strategy. Tests are devised prior to code construction, so therefore by definition the current iteration of the system fails those tests. Only when code for the next iteration is complete, and all tests are passed is the iteration considered accomplished. Test first creates a growing regression test bed for the product as it is fleshed out in the cumulative iterations.

The Agile Methods Are Worth a Serious Try... I was honored to be a Fellow at the Agile Development Conference held in Denver Colorado this summer. What was so heartening to me was to sit with the participants and hear how each company was tailoring the Agile Methods to best fit their kind of work. These ideas are bursting forth because they can be molded to fit projects so the projects can run both efficiently and effectively. If you need a place to start, here are two:

http://www.agilealliance.org/home

http://en.wikipedia.org/wiki/Agile_software_development

Or drop me a line.

Tim Lister lister@acm.org