The importance of discovery to innovation
One question I like to ask my clients is "what new insight have you learned and how did you learn it". We can't create interesting new products and services without learning something new about customers or prospects. If you don't learn something new, all you have to offer are the same products and services to meet the same needs. Alternatively, you can decide that actually understanding needs is less important than creating and then solving a new need that you create. This is why marketing is far more important that product development. Rather than ask clients what they need and fulfill those needs, many firms prefer inside-out innovation, in which the firm creates new features and attributes and communicates the value of these new features to customers. If you don't learn something new, you have to stick with what you know, or create your own reality and hope to lead customers to that reality. So you can see why learning something new is important to innovation.
Next, if you did take the time to try to learn something new, how did you acquire the information? In our very busy world, most product development and innovation teams outsource discovery to someone else. These innovation teams work on internal issues while third parties, industry analysts and survey attendants conduct interactions with customers to discover new insights. Then, a meeting is held to discuss what was discovered and why it is important and relevant. Since the innovation team didn't participate in the discovery, it has to receive at face value the needs and wants that are reported to it. This purely passive role means that the innovation team has no investment in the needs, nor have any of them witnessed the discovery of the needs. Therefore they are at best neutral on the needs and their importance, and can find it easy to 1) reject needs that are real but don't fit their world view or 2) accept needs that don't actually reflect what the customers and prospects said.
There's another problem when a team outsources discovery: it has no emotional connection to the information presented. If a team meets a customer in his or her home or business and sees the challenges and struggles with their own eyes, then the problem takes root and has meaning for the innovation team. There's more to the idea than its existence, the team can assign importance and relevance to the problem or opportunity. When handed the description of the problem in a PowerPoint deck, it is exceptionally difficult to generate any empathy for the customer or the need.
Innovators frequently talk about "passion" for an idea. There are so many barriers that will block or delay the development of an idea that it's difficult to commercialize an idea that everyone believes in. Opportunities or problems that weren't viewed or experienced by the team directly are easy to discount. If you haven't "walked a mile" in the customer's shoes, you can't discover their problems and the importance and relevance of that problem or opportunity.
Yes, innovation is messy, time consuming and difficult. It requires skills and competencies that you don't regularly practice in your working life, but no skills or capabilities that you don't already possess or can't acquire quickly. Innovation requires your full attention and deep immersion into the lives of your customers. Discovery and validation of needs and wants is important - perhaps one of the most important acts in the process of creating a new product or service. Yet most teams spend very little time in this step, and outsource a great deal of the discovery. You'll never have much passion or investment in an idea until you've experienced the need that created it up-close and personal. No third party can adequately describe a need, and if you outsource the discovery and insight work you'll never have the depth of understanding, empathy or passion for an idea, and that just makes innovation all that much more difficult. You must do this work yourself if you hope to succeed at innovation.