The Problems measuring Innovation

There are tasks we want to do that are reasonable to do and are practical to do.  There are activities we want to do that are sensible but difficult to accomplish.  Measuring innovation, which sounds important and reasonable, often falls in the second category.  It sounds reasonable and important to measure innovation, but is often very difficult to do well.  Sort of like measuring sand with a ruler, we may have the wrong tools, or the tools may not exist, or, quite possibly, there are some things we simply don't know how to measure.  As Einstein said, not every thing that counts is countable, and not everything that can be counted counts.  Let's look at some of the challenges in measuring innovation.

The tools we have

Let's start with our trusted and familiar measurement tools.  For most projects, we measure factors like cost, time to completion, the results created by implementation (dollars saved, profits created) and measure against the cost to develop, which gives us the almighty ROI.  The tools we use to measure activities and projects are tuned to the projects and activities we do regularly and that we deeply understand.  However, those measurement tools are often confounded by innovation.

Note that we anticipate being able to measure factors like time to completion.  In an innovation setting, where divergence is as important as convergence, it's difficult to say how long it will take to create a viable new idea, or when that idea will be fully vetted.  With little history, it's difficult to establish parameters about "typical" innovation projects.

Further, we've become so enamored with measuring efficiency, productivity and profitability that we neglect to measure things we can impact, like customer engagement and delight, so we tend to ignore or discount innovation that may not be able to deliver immediate profitability, even when those innovations could be valued by consumers.  We simply don't have the tools or understanding to measure or to understand how valuable those solutions are.  Like the proverbial drunk looking for his keys under the light post, we limit our investigation only to where we have the best understanding - where the light is best.


The difficulty in measuring innovation

There are several difficulties when measuring innovation.  The first is in the definition of innovation.  After all, innovation is a relatively generic umbrella term that contains a lot of different activities and outcomes.  Incremental product innovation is more definable and predictable than disruptive business model innovation, more familiar and probably easier to measure.  Given the range of activities, processes, definitions and outcomes, talking about measuring innovation is a bit difficult, especially when the range of outcomes is so broad.

Second, we lack consistent models and frameworks.  Most organizations don't have a deep history with innovation and can't compare projects to previous exercises.  There are few "knowns" and many unknowns, and many of those unknowns are unfamiliar or unknown.  Without existing rules of thumb or easy comparisons, it's hard to say if innovation is "on track" or "off track", and in the absence of innovation examples we tend to evaluate innovation in comparison with other familiar projects that are well-defined and well-understood.  In that light innovation often looks haphazard and unlikely to return any value.

Finally, most businesses are so focused on ROI - return on investment - and getting a "fast" ROI that true innovation is difficult.  Increasingly good innovation looks more like a reckless gamble.  The odds aren't great for a big return, but when the idea is right the returns are enormous.  Most would rather follow a safe, predictable model, limiting risk and uncertainty.  Therefore, they aren't really measuring innovation at all, but simply measuring predictable projects.

Measurements in Innovation Phases

Rather than a one size fits all, some measures and milestones need to apply to innovation phases.  For example:

1. Can we identify and scope an interesting unmet need or problem that customers have?  If we solve that problem do we shift significant business our way?
2. In the "discovery" phase of innovation - customer insights and trend investigation - are we learning about new needs?  Do we believe we have the "lead" in understanding those needs?
3. In the idea generation phase - are we generating interesting, valuable ideas that could disrupt a market or set a new standard for solutions or services?
4. In the validation phase - do customers indicate that our ideas solve important, relevant needs and that their satisfaction with the existing solutions is very low?

I could go on, but I'll stop there.  Note that as with many other facets of innovation, much of the measurements in the "front end" are qualitative rather than quantitative.  I can certainly measure the number of ideas generated - quantitative - but what really matters is the insight and value of ideas - qualitative.  This is another reason so many organizations struggle to measure innovation.  Not everything that counts is countable.  Sometimes a wise assessment is far more important than an accurate counting.