Innovation still more art than science

In a recently published survey from the Boston Consulting Group on Innovation (Innovation 2006), the survey's editor concludes that innovation is one of the top strategic priorities for many companies, with 72% of executives surveyed ranking it as a top 3 priority for 2006. The same percentage said they pland to increase spending on innovation in 2006. Yet almost 50% of the executives surveyed were unsatisfied with the financial returned on the investments made so far in innovation. Why does innovation hold so much promise, yet seem to underperform on results?

You can read the entire report. As usual, BCG has done a great job collecting and analyzing a lot of information about innovation. The consultants at BCG seem convinced that innovation is an exceptionally important strategic initiative that all firms must embrace. And, as the survey notes, that message is filtering down to the senior executives of many corporations. Across all industries, senior executives expect to increase spending on innovation. Yet most of these executives can't put their fingers on real substantial metrics and returns from the innovation investment. The survey provided several reasons, which should be familiar to anyone trying to improve innovation: inability to track returns from innovation, cultural issues and process issues.

The editors raise a great question, and one that could be easily missed without careful reading: "What steps can, and should, companies be taking to optimize their innovation spending to ensure that little, if any, of the money is wasted?" Later in the survey the authors point to common problems in organizations which are trying to improve innovation. "The most frequently cited hurdles were development times which are too long, a lack of coordination within the company, not enough insight into customers and a risk adverse corporate culture. Difficulty in selecting the right ideas and a shortage of great ideas were also common laments."

Most of these failures are due to a lack of strategic direction, culture or process within the firm. Long development times and a lack of coordination seem to be mutually reinforcing. Also, poor requirements for new products and services will increase development times, since we can't be sure what customers want or need. A common complaint with many firms is that innovation involves risk, and we've been trained to remove and eliminate risk where ever possible. Introducing risk taking into the organization flies in the face of accepted wisdom. The difficulty in selecting ideas springs from the fact that there's no clear strategic direction and no clear customer requirements. How does one choose a feature or attribute in the absence of good strategic direction and the lack of customer demands?

The issue around a "shortage of great ideas" is one that really puzzles me. In a knowledge based economy, when the majority of our workers in pharmaceutical and high tech and consumer packaged goods have advanced degrees and are highly compensated, how can it be that we lack for great ideas? I suspect the real issue is not a lack of ideas, but the absence of a system to allow people to generate and submit good ideas that are then acted on by the organization. If there's no place to submit ideas, and if those ideas aren't acted on, then there will be a shortage of ideas. It's not the ideas, it's the process and the system that's missing.

Finally, the authors point out that most firms do a miserable job managing the metrics around innovation. Less than one quarter of the firms surveyed admit to using consistent metrics to measure the results of innovation, and even those firms aren't quite sure they are using the "right" metrics. So, if your organization isn't measuring the results, how can it say that innovation is a good investment or a poor investment? Clearly there are resources flowing to generate ideas and bring new products to market. What are the measurements and return on investment?

What's clear from the survey is that the senior management teams are getting the innovation religion. Now they need to provide the strategic direction, cultural change and processes that are necessary to implement innovation as a consistent, repeatable process within the business, and set specific and clear goals to measure the outcomes. Product life cycles are shrinking and competition is fierce. Firms that differentiate themselves with a consistent stream of new products and services will be successful. The study shows that firms with a differentiated innovation strategy consistent outstrip their peers in terms of profit and market share.

Innovation is moving from a "nice to have" to a "must have". As it makes that transition, innovation must move from a serendipitous art to a more mundane science. Our organizations need to be able to plan and execute innovation strategies and processes much like we execute other project plans and measure the results.