Research: How to Get Better at Killing Bad Projects

Summary.   

Most companies and innovation managers know that few of their initiatives will succeed, so they keep multiple projects running at the same time and create processes for quickly separating winners from losers. One popular way to make decisions about what stays and what goes is the use of stage gates. Yet, even with stage gates, firms struggle to kill bad projects. 

A recent research paper authors undertook a decade-long review of the product development portfolio at former handset maker Sony Ericsson. They found that that the conventional use of stage gates can actually be part of the problem, impeding project discontinuation in counterintuitive ways.

Fail fast, the adage goes, and move on to the next big idea. Most innovation managers know that few of their initiatives will succeed, so they keep multiple projects running at the same time and create processes for quickly separating winners from losers.

One popular way to make decisions about what stays and what goes is the use of stage gates. This is when project leaders present their progress and findings to date, alongside updates to commercial expectations. Executives then decide whether to unlock funds for further development. For example, a project might have to pass reviews at one, three, and six-month milestones to determine whether it continues to promise return on the innovation investment.

Stage-gate processes improve on more laissez-faire steering methods in several ways: They aim to improve innovation effectiveness by separating project leadership from resource decision making to avoid conflicts of interest; formalize points at which discontinuation decisions can be made; and nudge executives to critically compare projects with others.

Yet, even with stage gates, firms struggle to kill bad projects. Anyone who has ever had to pull the plug on a colleague’s work knows how difficult it can be. Even supposedly ruthless venture capitalists often struggle to end projects at the right time.

Our research shows that the conventional use of stage gates, it turns out, can be part of the problem, impeding project discontinuation in counter-intuitive ways. To reach this conclusion, we undertook a decade-long review of former handset maker Sony Ericsson. From its inception in 2001 to its dissolution into the Japanese parent in 2009, Sony Ericsson pursued about 200 handset projects. 

We tracked and compared their decision-making.

This unique historical analysis of the entirety of a firm’s innovation portfolio reveals, among other things, the opportunity costs of not failing fast enough. Only one-sixth of projects were discontinued before launch. Sony Ericsson had some notable successes among the rest, but many phones brought in lackluster returns. With the development of flops drowning out more promising projects, the firm could not muster enough innovation firepower to respond to the smartphone trend that eventually sealed its fate.

In our analysis, we found that Sony Ericsson initially misestimated the revenue from its projects by €182m on average. That figure is not far from what phones would typically make over the product’s lifetime. Over and underestimating happens in many companies, and scarce resources go to the most promising projects. And we know imprecision is par for the course in fast-changing markets with long development periods.

The problem begins when these misestimations lead to a flawed rank order of project candidates vying for development (Sony Ericsson ran about 20 innovation projects at a time). Project ideas with early promise might get prioritized for funding, but go on to disappoint. Meanwhile, others that look less promising in early stages miss out on funding but go on to become market hits. Stage gates, if used effectively, can help decision makers notice and act on such changes to project expectations.

Over the span of a year of product development, Sony Ericsson indeed gained new information — like customer-preference changes, competitors’ moves, or technological shifts — that reduced its project misestimations to €66m on average. But despite this new information, Sony Ericsson struggled to correct its rank order of projects when making stage-gate decisions. The company continued to fund projects whose business cases no longer looked as great as initially thought, thus preventing investments in projects whose business cases might have ranked higher.

How can the organization of your innovation function act quickly on information gains? From our research, we recommend three modifications to your stage-gate approach to ensure that you’re stopping projects efficiently.

Forego Proof of Failure

When you’re dealing with the uncertainty of a new product or market, there is no reliable proof that a project is going to fail — and no stage gate will offer you that proof. What may ultimately be more useful for making continued go or no-go decisions is a qualitative assessment of changes to the main assumptions underlying the business case that led you to invest in the project in the first place.

Sony Ericsson typically used seven stage gates, beginning with “concept” and concluding with “ship.” Few projects offered sufficient visibility to track reliable financial KPIs in the early stages. Concrete figures were often either not provided or reliable. In interviews we conducted as part of our research, we found that few placed trust in such early estimates. A lack of solid figures does not mean a lack of reasons to discontinue projects, however. 

For example, Sony Ericsson could quickly notice changes in customer preferences, such as the extent to which a bigger camera would still distinguish a new handset, even if exact revenue figures were hard to come by. Trying to quantify the possible negative revenue implications of such trends for mass-market handset returns would mean months of resources kept from other, and often more promising, endeavors.

The qualitative insights available about customer-preference changes could have permitted a reallocation of development resources from cameras to other improvements even if quantifying the deterioration of camera-phone prospects was not yet possible. In a hunt for conclusive proof that something would fail, resources were locked up in ultimately often failing projects, starving others of much needed support.

Sleuth the Business Case

It’s easier to refine project-return estimations as a project nears launch. The unfortunate reality at many firms, including Sony Ericsson, is that near launch, attention shifts to delivery — and few like to disrupt execution. As a result, project managers often do not feel the need to bother with updating business cases with the latest insights. Sony Ericsson all but ceased project discontinuations about halfway through its development process.

Even if it’s late in the game, discontinuation remains hugely important, considering that most projects consume the majority of their development resources in those later stages, as things move towards mass production. A single late-stage project can prevent dozens of alternative early-stage ideas from being funded. Failing to update business cases near launch, and thus missing signals of failure, ends up disproportionately expensive.

To counteract the shift in priorities in the later part of stage-gate processes, it may be advisable to create the roles of business case sleuths. Free from the pressures of project execution, and answerable to portfolio performance only, such detectives could go after changes to business case assumptions when others have lost interest in evaluation and focus on getting across the finish line. Independent sleuths allow decision makers to build on new information about technological advancements, customer preferences, competitors’ moves, or other factors with bearing on project business cases when these have the greatest resource implications. Averting one expensive fail stands to more than pay for the extra business-case detective on your team.

Don’t Sweat the Kill

The most troubling insight from our research was that stage-gates’ well-intentioned focus on project performance can create decision paralysis. Once a project showed a clearly deteriorating business case, Sony Ericsson spent a disproportionate amount of time discussing it. The handset maker often postponed and revisited decisions. Instead of discontinuing its worst performers, Sony Ericsson more likely downgraded expectations and thus made it look as if targets were met. Ironically, the firm had a much easier time discontinuing projects that did not show business case deteriorations. 

For example, if a novel swivel functionality for a clamshell phone proved hard to technically implement, executives shut the project down more quickly than if clamshells themselves looked to be falling out of customers’ favor, reducing sales expectations.

The heightened attention to bad projects would be better placed on more promising alternatives. There is also the question of how much better a flagging business case can become, even if you look at it long and hard. Such attentional inertia can be reduced by minimizing the scope for interpretation and discussion. Setting clear discontinuation criteria beforehand ensures swifter, more automatic responses, preserving stage-gate decision makers’ emotional energy for worthier pursuits.

Overall, our unique analysis of the entire new-product development portfolio of a firm’s life offers a cautionary tale. Using stage-gate processes has on occasion been criticized for the tendency to bias against more daring innovation; less known has been the potential to escalate commitment, the very bias stage-gates are intended to avoid. As a resource allocator, you should understand the impossibility to reduce commercial uncertainty in early stages as well as your staff’s natural reluctance to reduce uncertainty in later stages.

Finally, don’t let decision paralysis set in when performance lags. Selective project progression is key in markets where investment occurs prior to knowing, and where learning during development determines the chances of success.