In digital, there are opportunities both fast and slow

In my class today on digital transformation I was fortunate enough to have a great guest speaker, Ramesh Latan, who helped transform Bell+Howell into a digital company.  Ramesh gave a great talk and the students received it really well.

One of the key points he made was how much more efficiently (and rapidly) digital processes operate. While his key focus was the Internet of Things, he talked about speeding up a number of activities that support e-commerce:  picking, sorting, packaging, addressing and mailing products.  All of these activities were sped up by IoT and robotics, often resulting in cycle times moving from multiple minutes to a few seconds, while at the same time increasing product throughput dramatically.  It was really interesting to see the transformation.

After the class, one of the students asked a question which was interesting.  His question, somewhat paraphrased, was this:  what happens to people in the process when the process continues to speed up?  His concern was real - could people contribute to processes that are increasingly automated, increasingly data driven and increasing occurring at speeds that humans can't comprehend.  It reminded me of an old saying that a good friend used to share with me - you can think faster, but you can't fish faster.  This question gets at the heart of whether or not there will be opportunities for people as AI and robotics accelerate processes.

Why you can't fish faster

While machines may be speeding up processes that can be automated, there are plenty of activities that cannot be sped up, and will require a "human in the loop" as the saying goes, for quite some time.  For example, an e-commerce order may be received, picked, sorted, packaged and shipped in very little time - so quickly that we somewhat clumsy humans may simply get in the way of well-trained AI, robots, RPA and other activities that can be automated.  But no matter how fast the automated processes work, if a human is involved, the communication can only go so quickly.  Just because I've sped up the automated processes does not mean I can speed up the communication processes.  Humans can only gather, interpret and understand information at a specific speed, and that speed isn't necessarily increasing as we get more data and more technology.  So while the processes may get faster, the interactions and context to communicate and emote with humans isn't getting faster and demonstrates where humans play an important role.

The reference to fishing faster is an important one.  As a person who enjoys fly fishing, by law I can only have one rod in the water at a time.  It is a purposefully inefficient process.  What's more, if I am fishing for wily or easily spooked fish, I have to fish slowly and carefully, in order to present the lure to the fish at the right place and time without showing myself to the fish.  This kind of fishing cannot be sped up - it takes experience, craft and forethought, and is difficult to execute effectively.  In other words, to be more effective at catching fish when the conditions limit me to one rod, one lure, relatively difficult fishing conditions and smart fish, I can't fish faster, I have to fish smarter.  And that's an analogy to the opportunities for humans as processes speed up.

Fishing (and working) smarter

We humans will still have important roles, but the roles will shift.  While we aren't as fast as machines, we are far more flexible, more creative and less rule bound.  This means we can augment the machines, use our creativity and insight, become better interpreters or explainers of what's happening and do a better job of anticipating what will happen next.  Plus, we are better at communicating, putting things into context for our fellow humans, demonstrating understanding and empathy.  That is - for some time to come - we will work smarter than the AI or ML or robots that we work with, and that's where our opportunities lie.  In fact, there are many attributes of a process that will not speed up - which will continue to work at the pace of a human, but perhaps a human with a better understanding of the digital decision making and augmented processes.

To go back to the fishing analogy

Younger or less experienced people who fish think the activity is probabilistic - the more times I present the lure, the more chances I have for a strike.  In some wildly optimistic setting that is potentially true, but your arm will wear out long before you'll increase your catch.  It's not the number of casts, or to some degree even the placement of casts, but understanding the water, the hatch, knowing where the fish are in a particular hole or channel, understanding drift, the angle of the sun and other nuances.  More casts will simply spook the fish, and once spooked they won't bite. Fishing smarter and slower is the best recipe for success.

This is completely transferable to almost all kinds of work.  There will be components or attributes of the work that, like fishing, resist automation and place value on knowledge, flexibility, craft, insight and ingenuity.  And this is where the next working generation will thrive.

It's important that we understand what the emerging digital technologies can, and more importantly should, do.  There are many difficult, repetitive activities that will be replaced and done far more efficiently, faster and with more consistency by machines.  We need to be thinking now about the activities that don't translate to simple definition and automation, and building skills and competencies to do this work more capably.  No matter how fast or efficient some digital processes become, there will almost always be a human at the beginning and end of the process, and we will work at a much more sedate speed, but require far more expansive interaction, context and communication.

As digital transformation unfolds, it pays to be a generalist

I remember the hubbub surrounding Dan Pink's book A Whole New Mind, in which he stated that the future (this was back in the early 2000s) would belong to right brained people, because machines would ultimately automate anything that could be easily documented and that followed a standard process.  His argument was that everything that could be reduced to a defined process would be outsourced or automated, and in most cases he has been proven correct.  What remains to be seen from his prediction is whether or not the "winners" of this automation will be the people who have "right brain"skills - artistic, creative people who were most likely liberal arts majors.

What I think is another interesting and similar question is emerging in machine learning and artificial intelligence.  I think it is likely that the people whose jobs are most at risk from machine learning are the deep specialists, who have really deep but narrow knowledge.  As we train machines to interpret data and to approach an artificial intelligence, I think many of these instances will be deep learning around very specific problems.  Take, for example, breast cancer.  Machine learning and artificial intelligence may soon reach a point where detecting breast cancer from x-rays and sonograms is more consistent and less error prone than having a doctor do that work.  But unlike doctors, who are often good at many things simultaneously, the AI or ML application that's good at finding breast cancer will probably not be able to immediately also diagnose other health issues.  For a while at least AI and ML are often good, but one trick ponies.

Which raises the question - will we spend the time and effort to train the AI and ML applications in a wide array of very specific, very important and very narrow fields, where the benefits outweigh the cost of training the machine?  And, once all of these machines are trained, who oversees the transition from one model to another to do multiple diagnoses, or when will a generalist human be "good enough" because they can move between different issues more easily and perhaps more rapidly and effectively than machines?

Does it pay to be a generalist in an age of ML?

Of course this is not just an AI or ML issue.  When we think of the technologies powering digital transformation, there are a handful of technologies that are really efficient but frequently limited option solutions.  Most robots, for example, are good at one or two activities that are constantly repeated, whether that action is in 3-D space (picking and placing parts) or in automating data transcription using RPA.  IoT devices gather and transmit data effectively, but only the data that the sensors are meant to capture and transmit.

These deep but narrow competencies will eventually create highly productive and efficient but potentially fragile processes, where a small shift in focus or needs may expose the fact that these technologies, at least for the foreseeable future, aren't really good at rapidly shifting from one input, one data set or one job to another, even if the shift is somewhat inefficient.

We humans, however, have evolved to do exactly that.  For the most part we are multi-functional machines, capable of doing a wide variety of tasks without a lot of reprogramming, and we can shift from job to job, task to task relatively quickly.  It seems as though as digital transformation takes hold, the ability to adapt, to be flexible and the ability to shift quickly from one task to another will be important when working with machines and intelligences that are relatively narrow and somewhat rigid in their capabilities.

What happens to the specialist?

What happens to people who have deep, deep learning and experience in a specific field that AI or ML can rapidly learn?  At first they become the teachers of the technology, helping instruct the AI or ML on misdiagnoses, correcting errors and improving the model.  Then, once the machines become nearly as good as the humans at detecting issues, humans become the explainers, telling people how the machines made their decisions, often defending the machines.  Eventually, as machines can explain their decisions and provide sufficient evidence, humans in deep specialties may become much less valuable.  What happens to a deep but narrow specialist once explainability arrives?

Where humans will thrive

Where humans will thrive in this rapidly approaching future is in places where there is little previously documented experience, where situations and models change frequently and without a pattern, where data is messy or missing, where a fast but "good enough" answer will suffice, or where tasks are frequently changing and don't allow time for re-purposing or retooling.  These needs will still be filled by capable generalists who can apply a lot of intelligence, reasoning, creativity and dexterity to rapidly emerging challenges that haven't been seen previously or can't be adequately predicted. 

At a time when our education systems are increasingly focused on narrow fields of study, we need a more comprehensive education system that reinforces a number of good skills simultaneously and turns out people able to rapidly shift from one task or skill to another.  Instead of increasingly narrow PhD programs, what we need are robust programs that engage science, math, literature, technology, psychology and other disparate skills to prepare people for the challenges and opportunities they are likely to face as they increasingly work with intelligent machines.

There's gold in the data - but have we learned the lessons?

For some reason I can't get the song Clementine out of my head.  Perhaps it's because of the line - dwelt a miner, a 49er....  that keeps that song playing in my mind.  We are in a significant transition. One breed of miner - the hardhat wearing, coal digging, heavy equipment manager is leaving the scene, being replaced by large equipment, robots, issues with safety and the environment and of course greenhouse gas concerns.  In his or her place is emerging a new kind of miner - a data miner, a process miner, a bitcoin miner.  People who are expert at exploiting a new, but somewhat unnatural resource:  all the data that we create.

The reference to "49" in the song, by the way, is a reference to the gold rush in California.  Where there are a lot of valuable and unoccupied opportunities open for grabs, there will be a gold rush.  We are in the very beginning of one now - a gold rush to find, mine and refine data and turn the insights into value.  There are some interesting parallels between the gold rush miners in the 1800s and the people seeking to capitalize on all the data available today.

You may have heard it said data is the new oil

Let's get this out of the way - there must be at least one cliche in every blog post so there it is.  Data has a lot about it that makes it a good analogy for oil.  At the beginning of the oil boom, there were lots of deposits of oil that were easy to reach, close to the surface.  Oil as we know it today became popular as other types of fuel (whale oil in particular) were becoming more difficult to obtain and more expensive.  Plus, the oil was more plentiful and in most cases had more value.

Data is increasingly the same way.  Now there is more of it, since there are more devices and people generating more data (and more kinds of data).  Data is more valuable, because as the volume grows you can do interesting things and get interesting insights from the data.  And we are just entering an age where we have the compute power and memory access to really dig deeply into the data and find all that is interesting in there.

Of course we can mine our current databases - the CRM and ERP databases that contain so many transactions.  These should be gold mines after years of collecting data.  There are probably other significant master data bases that your company has that can be mined.

Just recently I became away of process mining - which is just data mining but based on process records.  So your order to cash process, or procure to pay process can be mined and evaluated for gaps or inefficiencies, or to discover new insights.  Even blockchain has its own miners, but they are in a different category.

Why data is unlike oil or gold

In the past, miners have been extractive, doing difficult and dangerous work to extract rapidly depleting natural resources.  These future miners are doing something quite the opposite - working in pristine conditions with the latest technologies, but more importantly working on a resource that is replenishing faster than it can be mined and understood.

The total amount of gold, coal or oil worth extracting is fixed, and the inexpensive stuff has already been extracted.  In contrast, we are only beginning to understand how to understand data, and the amount of data we create will experience exponential growth.  The problem miners will face is not where to dig, but assessing the potential value of data before digging.  If anything, data miners and data scientists risk becoming overwhelmed with data, as companies increasingly understand the potential value of the data in their organizations and make plans to capture and store more of it.

What these miners have in common

What all of these data miners have in common is that they are trying to find, collect, assess and refine the data and information in large databases within corporations.  The potential to do this well emerges from a couple of new concepts:  data scientists, who are interested in what the data contains, and new tools like predictive analytics, machine learning and artificial intelligence, as well as natural language processing.  These tools help identify interesting trends or relationships in the data in ways that humans can't do at scale.

What these miners and their companies lack

While there is great promise in data mining and process mining, many companies lack the basic infrastructure to benefit in significant ways from their data.  This is true for several reasons:

First, the depth and range of data.  While many companies have a lot of data, it often isn't reflective of a long period of history over a consistent set of customers or vendors.  Just having a lot of data does not mean much if the contents vary a lot.

Second, the quality of the data.  While some data is of relatively high quality (most likely the data in your ERP system) other data may be suspect (your CRM data is more suspect than your ERP data) or questionable (data you acquire versus data that you create).  Differences in quality can create significant errors in analysis later.

Third, the validity of your processes.  When a friend suggested I write about process mapping, I almost laughed out loud.  There's nothing wrong with the idea - in fact it's a good idea, but it assumes that 1) there are defined processes that 2) people follow to the letter and 3) that all the data within that process is captured effectively.  Since we know that SOPs are often written but rarely reviewed, many processes don't do what we think they do.  Be careful about believing that your documented processes reflect reality.

Fourth, the ability and capability of your data scientists.  I don't know about the market where you live, but if you want a job in Raleigh, all you need to do is claim to be a data scientist.  There are far more jobs than experienced people, which is another problem with the gold rush.  Lots of people who went to California had little or no experience mining for gold.  Turns out, it isn't simply lying there on the ground waiting to be picked up.  You needed experience to find it, and even then there were no guarantees.

I suspect that there will be some isolated successes in every company that has good data and good processes, and is patient with the people who are seeking value in the data.  There are a number of caveats there.  Few companies have good consistent data, and even fewer have the patience necessary to allow a lot of experiments and learning in the data.  It could be another few years before we see really interesting results from data mining from firms that are not pure data plays.

One small warning

It will be interesting to see who becomes the Levi Strauss of this particular mining rush.  Strauss became rich not from mining operations but by selling clothing and equipment to miners. There's a lesson here for current day data and process miners.  Very few of the miners in the California gold rush got wealthy.  The people who reaped most of the benefits were the people who supplied them, sold them supplies and entertainment.  The unexpected outcome of the gold rush was a growing, thriving and eventually diversifying economy, which I think might happen in this data mining rush.  I wonder how much that history will repeat itself in this rush to mine data.  It remains to be seen who benefits or ultimately profits from all the data mining that will happen.

Innovation building blocks: diverse perspectives and voices

I've been writing a series of blog posts focused on some of the most fundamental building blocks.  These fundamentals are necessary to sustain innovation in corporations, and are most often overlooked or treated as an afterthought.  Some of the building blocks I've reviewed are:
 - having a cultural bias for innovation
 - encouraging discovery and exploration
 - ensuring enough time, and the right kinds of time, for innovation

Today I am focusing on the importance of diversity for innovation success.  When I refer to diversity, I am speaking of a spectrum of concepts:  different perspectives, different experiences, different educations, internal and external perspectives, perspectives from people who are new to the problem as well as from experts, and so on.

Let's first talk about why diverse voices and perspectives are so rare in many corporate innovation teams, and then look and what a little more diversity can do.

The usual suspects

One of my favorite movies is Casablanca, and in one of the most telling scenes, the police chief acknowledges a crime has been committed and tells his men to round up the usual suspects - in other words, the same criminals they rounded up for any crime.  In many ways, innovation teams are often built this way.  Executives want the work to go well, so they round up the usual suspects to participate.  But these usual suspects aren't criminals.  No, they are the best and the brightest.

In a corporation, the "usual suspects" are typically high performing people who have demonstrated their ability to deliver high quality outcomes at low cost, typically running an existing process or product.  These folks are tightly bound to existing rules, thought processes, markets and corporate history.  This makes them the perfect suspects if we are trying to emulate an existing product or process, and perhaps the worst  participants for an interesting innovation team.

It's very often the case that these people are not very diverse.  When I say they aren't diverse, I don't mean diverse in terms of race or ethnicity, but in terms of how they think, what they know about customers or market needs, the amount of risk they are willing to bear, how creative they are when allowed to think about new ideas, their experiences and so on.  Far too often these teams are formed and have great homogeneity in the way the participants think, and that leads to limited innovation.

Let me give you an example.

Clarifier, Ideator, Developer or Implementor

I've used the Foursight personality model to help my customer teams gain some understanding of this issue. The Foursight model helps identify four personality types when it comes to developing a new product.  A clarifier seeks clarity - asks questions about strategy, purpose and scope.  An ideator loves to generate ideas but may not be all that committed to realizing an idea.  A developer likes to develop raw ideas as prototypes or pilots, but may not enjoy implementing the idea.  Implementors are the ones who see a prototype or pilot over the finish line to become a new product or service.

All of these capabilities are necessary for innovation.  If everyone is an ideator, it is unlikely the idea will progress to become a new product or service, as an example. 

In one company where I led innovation work, and where people were assigned to the innovation team, I used the Foursight model to determine the interests and proclivities of the people on the team.  Of the 8 people on the team, seven were either developer or implementors.  This was a challenge for a team that was expected to create radical new ideas!  Only one member of the team even enjoyed generating new ideas, much less thinking creatively and divergently.  I say think not to belittle the skills of the other participants, but to note how likely it is that the "best" people are often not the right people when it comes to innovation.

The Foursight analysis represents just one aspect of diversity.  It reflects the role people like to play in an innovation activity.

Another Example

In another case, I worked with a team creating new medical devices.  This meant we had to have people on the team who had 1) product development and engineering skills 2) business and marketing skills 3) medical experience and 4) legal skills.  This is also a diverse team, and without all of those skills on the team - working in some harmony - the project would fail.  Of course, each of these functions represents different skills and experiences, but they also have different language, different expectations and sometimes very different ways of working, so diversity does bring with it some challenges.

Diversity - of ability, race, ethnicity, education, age, experience and so many other factors is what creates truly interesting ideas.  So why don't companies invest more and plan more for diverse teams?

There are a couple of reasons.

Why homogeneous teams are often created

While some managers and executives will recognize that diversity is important, I'd guess that at least 75% or more of the innovation teams I've worked with were fairly homogeneous.  There are several reasons why homogeneity is so often the case.

First, the usual suspects are often the most favored and recognized people.  Many innovation teams are staffed with people known and valued by the management team.  Dissonant voices or people not well known to the management team aren't often included.  This reflects concerns about speed, comfort and awareness of the skills and attitudes of the individuals.  It's logical to want to seed the team with successful people, but often these very people limit the scope of work.

Second, some innovators or people with divergent ideas or voices are considered troublemakers or malcontents, so few managers want to appoint people to teams who may create conflict. Yet many of these people have the insights or knowledge you need.  But it requires identifying the skills and vetting the members, and perhaps taking a risk on one or two team members.  Few managers like taking risks on team members in high profile projects.

Third, diverse teams often take longer to form and norm.  By their very diversity of approach and experience, it may take longer for such a team to form and norm.  But when they do, they have the chance to perform far more effectively and creatively.  Again, an issue of speed and cost.

Finally, I think there's a real lack of understanding about the value and importance of diverse ideas and voices.  Some of that is based on how much risk and variance have been removed from business processes and management, and some is just a lack of engagement and understanding by management teams. 

Diversity - of many kinds - is a critical building block

Diverse people with widely different experiences, perspectives, education and backgrounds are more likely to spot needs, understand customers, generate newer and more valuable ideas and find the issues and challenges in an idea more quickly than more homogeneous teams. 

The problem today is that building these teams is more work, takes more time and creates more risk for managers, who prefer a predictable, quick and engaged team, even if that means the opportunities for innovation are far lower.  If your team wants more and better innovation, include more people, who have more diversity of thought, perspective, experience, education and other factors, as quickly and as often as possible.

Innovation fundamentals: using time effectively

I've been writing an occasional series on some of the fundamental building blocks for innovation success - at least those that I think are critical, intangible and often overlooked.  I've written previously about having a bias for innovation in your culture, seeking and finding important and unmet needs or opportunities, and introducing discovery and exploration in your innovation process.  These concepts all have something in common, which is the next building block I am going to discuss:  time.  Good innovation, like good cooking or the development of a human being, takes time, and time is something that is so constrained that most innovation projects simply compress time into manageable blocks, and in doing so compress and constrain good ideas.

There are at least four concepts of time I want to discuss that matter to innovation:  elapsed and committed time, development and launch time, and the opportunity window. All are important to understand and all are critical to successful innovation.

Time, in several dimensions

When I am talking about time I am actually talking about several different "kinds" of time.  There is first elapsed time - the time from the first recognition of an opportunity or need until the product comes to market.  For most companies, the preference would be that all innovation projects last no more than 88 days - a project should begin one day into the quarter and finish to deliver outcomes one day before the same quarter ends.  This is one reason why ideas like agile and rapid sprints seem so compelling.  We have an artificially imposed time constraint on innovation activities, forced upon us by the financial reporting mechanisms of the stock market.

There's another time that is also important.  That is the time that individuals and teams can commit to any specific innovation project.  Employees in most businesses are fully loaded with plenty of existing work and responsibilities, and most will prefer to focus their efforts and energy on their day jobs rather than new or unusual innovation work.  If people cannot or will not commit the necessary time to an innovation activity, then it will either fail or the scope will decrease to meet the time they can commit. This means a lot of interesting and reasonably important activities like problem and scope definition, exploration and discovery, trend spotting and customer insight work are hurried or in most cases avoided.

So, time is being compressed from beginning to the end of the project (the elapsed time is constrained) and the time committed to a project by individuals and teams is being constrained (total committed hours by team members).

Time is a valuable commodity

If time is money, or if time is a valuable commodity, and increasingly time is not invested in innovation activities in at least two dimensions, what is the logical conclusion that people assigned to innovation tasks should draw?  That time is better spent in other activities.  This is a relatively simple and obvious conclusion, and a message that far too many companies are sending to their innovation teams.

But what about lean startup, rapid sprints, agile

But aren't there entire methodologies that are meant to get to good ideas faster?  Agile, lean and rapid methodologies meant to accelerate the innovation cycle?  The answer is:  yes, properly implemented and resourced, concepts like Lean and Agile can be introduced into an innovation process, but the concepts should not simply take time out of the equation.

For example, the idea of short sprints that result in viable, testable concepts and prototypes makes sense as you build to a larger solution.  But what often happens is that the first sprint creates a solution that is "good enough" and little discovery or incubation of ideas happens after the first sprint.  Too often companies are taking the philosophy of agile and lean and using those concepts simply to remove time from an activity, without fully engaging all of the rationale of the approach.

Take all the time you need

The best advice I can give here is that teams must have the right to frame a problem correctly, explore trends and emerging needs and gather customer insights.  These activities take both 1) elapsed time and 2) project team member time and commitment  in order to be completed effectively.  This isn't news - we all know and understand this, and simply accept the inevitable when the elapsed time is shortened or the commitment levels of team members is reduced.  Likewise, as these reductions happen, the scope and potential outcomes are most likely to be reduced.

Don't get me wrong - I'm not saying that more time equals better ideas.  In fact sometimes moving quickly with a really interesting idea is the best answer, because it gives the existing culture and bureaucracy less time to fight back.  In that regard we can see that often the time that is most critical is not elapsed time but the amount of time committed by engaged employees, who are hopefully following a defined process and not spending time fighting culture and bureaucracy.

It is the case that a few really deeply engaged people, who have the permission they need to explore and discover, talk to customers and identify needs, and the skills to generate ideas and create rapid prototypes can create really interesting new solutions relatively quickly (in elapsed time).  But these teams often have people who are spending significant time on the project and who are relatively senior and experienced in their area of expertise.

Development and launch -  time innovators don't control

However, even if you can get to a really good idea quickly, there is another time that matters, and that is the time a good idea sits waiting to transition into development, and the time it takes to fully realize and launch a new idea as a new product or service.  It's often the case that the real time culprit in this regard is simply how difficult it is to get a new idea into product development and then through the product development process.  This is especially true with tangible ideas in areas like medicine, medical products, food, pharma and other areas where oversight is high and the regulatory burden is high.  Getting to a good idea can be a relatively short affair.  Getting a product to market can take years or even decades.   Again, agile and lean development and launch processes are important, but there's not nearly enough focus on reducing product development time, which brings us to the final time consideration:  the opportunity window.

Opportunity Window - the clock is ticking

One of the final aspects of time for innovation is the opportunity window.  That is, there is a sweet spot for new technologies, products or services.  Introduce a new product too early, or before customers are ready or recognize a solution, and you create an expensive beachhead that others will profitably exploit.  Introduce new products or technologies too late and you enter a red ocean, where many of the early profits have been harvested, and all that's left is to compete on price.

The "smart" play is to be the second or third player to introduce a new offering, allowing the first or second player to establish the need and solution and educate the prospect base.  Every company wants to be the smart fast follower, but most spend far too much time defining and validating a market, so they enter too late.

The opportunity window is increasingly important, because new technologies and solutions rise, gain market share and fall out of fashion very quickly.  Whereas product life cycles used to be measured in periods of decades, new products are now in and out of fashion in 18-24 months.  This means entering the opportunity window early, with the right product or service, is the only way to gain the lion's share of the profits.  Again, an issue of timing.

Who determines or controls the pace of your work?

Here's the vital question - who controls the timing and pace of your innovation work?  Corporations have people whose responsibility it is to manage people resources, funding and equipment, all of which are important inputs to any project.  But who controls and sets the time element?  Or do we simply revert to the time commitments and expectations of familiar projects, and try to reduce time commitments to innovation to get back to the day job, or to cut costs and scope?  If "time is money", then why isn't it managed as such?

Getting the time right for innovation, across these and other considerations, is critical to sustained innovation success.