Here’s some depressing news: The majority of IT projects fail. They either fail outright, come in wildly over budget, finished much later than planned, or don’t deliver the business value that was originally planned.
Anyone who’s been it IT for very long has certainly seen more than a few such “less than perfect” projects. While it’s depressing to see confirmation of just how bad the situation is, it’s also a bit of a relief to know the situation is universal and not just local.
The good new is, it’s just not you, and it’s not just your team. Industry wide, the percentage of “successful” IT projects is quite low. How low? In 2004, only 29% of US IT projects were successful. That’s according to the 2004 Standish Group Chaos Report, the largest and most comprehensive research report on IT project success in the US. Of the remaining non-successful projects, 18% were abject failures. They were either canceled or delivered no value. The remaining 53% were considered “Challenged”. These were projects that delivered *something*, but were either overdeadline, overbudget, delivered fewer features than planned, or a combination of the three. Other surveys and studies have reported similarly bleak statistics.
So what can be done? How can you prevent your projects from failing? Here are 5 things that can dramatically increase the odds that your IT project will be a success.
1. Keep your project size/duration small:
According to the Standish Group, the three key metrics for predicting the success or failure of an IT project are: project size, project duration, and team size. Simply put, size matters, but not in the way you might think. Big is bad. Small projects are much more likely to succeed. Consider the success probabilities in the chart below:
I’m still not excited by the slightly better than 50/50 odds for projects under $750K, I like it a lot better than the odds if you’re stuck on a $10Million or larger project. You’re going to literally need a miracle for your project to succeed!
Limiting the size and duration of your project is the #1 thing you can do to make your project successful. It’s so important that the next two items in this list are ways to help achieve this goal.
2. Use Agile development methods:
Agile processes are listed as #5 in the Standish groups list of Top 10 Reasons For Project Success. Personally, I think it should be higher on the list. If you develop and deliver incrementally, each iteration can be treated as a close ended “mini-project” that delivers immediate value to the users and enabling feedback from the users than can guide the next iterations development. This feedback loop is one of several ways that agile development keeps the users involved throughout the project, which incidentally is #1 in the Standish groups list of Reasons for Project Success.
Agile development methods also emphasis integrated testing methods, like Test Driven Development (TDD), and Continuous Integration. Integrating testing throughout the development process lets you catch and fix bugs earlier. Under the old waterfall project methodology, QA testing and final project integration was one of the riskiest parts of an IT project and was often where the project deadline and project costs would spiral out of control. Finding and fixing bugs and integration problems early allows problems to be discovered earlier when they are less costly to fix. This is similar to the “Do it right the first time” rule used in Lean Manufacturing.
3. Leverage a Service Oriented Architecture(SOA):
I’ve posted here before about some of the advantages of an SOA. As I stated in that post, SOA enables you to reduce the size of IT projects by allowing you to reuse functionality from other IT systems. For example, if one project has already exposed something like sales tax calculation or credit card processing as a service, future projects won’t have to write code to handle those functions. They can simply call the existing service. Less code to write/test/debug means a smaller project and less risk!
SOA can also helps reduce the size of a project by giving you new ways to divide up a big project into multiple smaller projects. Often a seemingly large, monolithic application can be re-designed into a number of services and much smaller application that consumes those services.
Here’s what Werner Vogels, the CTO of Amazon.com, had to say about Amazon’s use of SOA in a recent ACM Queue interview:
We went through a period of serious introspection and concluded that a service-oriented architecture would give us the level of isolation that would allow us to build many software components rapidly and independently. By the way, this was way before service-oriented was a buzzword. ….
If applied, strict service orientation is an excellent technique to achieve isolation; you come to a level of ownership and control that was not seen before. A second lesson is probably that by prohibiting direct database access by clients, you can make scaling and reliability improvements to your service state without involving your clients.
4. Use open standards and commodity software/hardware:
The history of IT has been a steady march of open standards displacing proprietary standards, and inexpensive commodity components displacing expensive proprietary components. How many SNA or IPX networks have you seen recently? They’ve all been displaced by IP networks, just as proprietary mainframes and minicomputers have mostly been replaced by more open Unix servers, which themselves are now being replaced by Linux servers running on commodity Intel hardware. Remember the old proprietary online services like Genie and Prodigy? Where are they now? Gone. Wiped out by the open architecture and commodity protocols of the Internet.
Open always wins. Open standards and commodity components are less expensive, more interoperable, more flexible, have greater vendor independence, and are just plain less risky than proprietary alternatives. Next time a slick sales rep tries to convince you to adopt a proprietary standard instead of an open alternative, ask your self whether you’d about to buy into the next Internet, or the next Prodigy. I know which side I’ll be betting on.
5. Find good people for your team and treat them well:
Programmers, sysadmins, and other technical team members are not interchangeable. This should be common sense, but for some reason people continue to get seduced by the idea that a wizbang new product, new language, or new development methodology will allow their project to be successful with a team of low payed, low skilled numskulls. It doesn’t work like that, and it never will.
For a project to be successful, you need a team with the appropriate skill levels and you need to provide them with a productive environment to work in. Studies have proven that factors like level of experience, tool familiarity, and motivation exert a massive influence on productivity.
Standish group lists “Skilled Resources” as #8 in their reasons for project success, but I would go a bit further and say that having skilled people is necessary for success, but not sufficient. You also have to put them in the right environment where they can be positively motivated, and most importantly NOT INTERRUPTED!
That last recommendation might seem strange to a lot of people, but it’s true. Most technical tasks require flow, which is what some people call “being in the zone”. Interruptions like phone calls, meetings, email or noise from adjacent cubicles break concentration and break the flow state. One study showed that developers with quieter workspaces and fewer interruptions were more than two and a half times more productive than other developers!
Interruptions aren’t just a problem for technical workers either. Another study showed that interruptions from email and instant messaging lowered effective IQ points more than smoking marijuana! Clearly this is not conducive to project success!
Building successful IT projects is hard because IT projects are fundamentally unlike any other kind of engineering, manufacturing or construction projects. Projects in the non-IT world often involve building a copy (perhaps with minor changes or tweaks) of something that’s been built before. This almost *never* happens in software! Unlike bridges, cars, or kitchen cabinets, software can be copied an unlimited number of times at the touch of a button.
IT projects are always experimental. That’s why they’re complex and risky. That’s why so many of them fail. That’s why it pays to have the best, most experience team you can afford. And that’s why it makes sense to keep the experiment (your project) as small and self contained (loosely coupled) as you can.