The best job to have in the near future - data strategist

 As you know if you follow my blog, I generally write about the intersection of strategy and innovation.  Increasingly, however, we need to invite data into this mix.  As more and more data is generated, it creates new opportunities in the strategy realm and in the innovation realm.  This post will consider the importance of data in strategy development and execution, as well as in innovation.

Past, Present and Future

In the past, data was an afterthought, if we are being honest.  Until the era of big systems like ERP and CRM, most data was recorded manually in ledgers.  Since it was compiled manually, data basically took up space in three ring binders in shelves around the office.  Data was occasionally consulted for reporting or historical evidence, but rarely drove decisions.  Data was really more of a hassle to manage than a benefit to the business.

When larger, integrated systems like ERP entered the business world, this unlocked data from the paper ledgers and placed it in computer systems which often had reporting solutions attached.  It is interesting to note that many of the ERP systems called (and some still call) these functions ledgers.  What these electronic ledgers and the associated databases and reporting applications created was the ability to capture and report data much more easily.  However, a lot of this data was stove piped - the manufacturing floor might have its view of the data, and the marketing team it's view of the data, but the organization lacked good integrated data to report on.

So, data warehouses were created to combine and pre-process data and make it more readily available to people who wanted to combine shop floor data and marketing data into a report or KPI.  That's great, but data warehouses are really only great with structured data, and just as data warehouses became really capable, unstructured data became more important.  I've seen statistics that indicate that 80% of the data companies ingest is unstructured data, which is harder to parse and interpret.  Also seen statistics that suggest that most companies have the ability to access, interpret and report on about 15-20% of the data they possess.  A lot of the data being captured and stored is virtually useless, because it cannot be interpreted or understood by current systems and people.

It's about here that people began to realize that, like the California mountains the 1840s, there was gold in there somewhere.  Data became, and is still becoming, a source of value rather than a problem to be solved.  Now, we need to learn to access data, unlock it, and have it tell it's story to all of us.

Water, Water everywhere

As we move into the future of data as a source of value, we'll need to understand a few things:

 - Where it is coming from

 - How we can best manage it

 - Most importantly, how to extract the value it contains

First, where is it coming from?  Honestly, everywhere.  The digital transformation that everyone was talking about before COVID is arriving.  The Internet of Things that seemed like such a futuristic opportunity is happening.  Billions of devices generating data, some of it interesting and useful, some of it not so much.  Consumers are generating data on social platforms, and if businesses are smart they'll find ways to get consumers to build relationships and exchange data with products and brands.    In the near future, robotics and automation will create data about processes and products.  Finally, and we are nearing this threshold, the data will start generating data about itself.

Second, how can we manage it?  Today, most firms are fortunate if they can adequately manage and interpret 15-20% of their data, and the data volumes and varieties are exploding.  Companies need to start with a data strategy, to understand what insights they need and which data streams can provide the supporting evidence.  What companies need to do is develop a data strategy that supports and enables the business strategy, then put the requisite systems in place to capture, manage and interpret that data.

Timing is also important.  Our traditional way of interpreting data is in hindsight - reports that tell us what we did yesterday, last week or last month and compare to previous periods.  This information is helpful but does not fully illuminate future activities.  Corporations need data about the present and indications about the future as well.  We have to analyze and interpret the data in real time, but also allow the data to predict what is going to happen next.

But what's most important is to extract the value the data contains.  We need to move from being content with reporting last week's sales, or even yesterday's revenue, and move toward what the data tells us might happen and use that insight to take proactive actions.  Some companies talk about being data-driven, I like to think about being led by the data to new opportunities, new markets and new needs.  Most companies are fortunate if they can report on old data, and the data scientists they are hiring are working to create real-time interpretation and some prediction capabilities.  This is where we need to be spending our IT dollars.

What's this got to do with strategy?

So, if you are still with me at this point, you may say - this is all interesting, but what's all this got to do with strategy?  This is an interesting question.  Companies in the past wrote a three or five year strategy and, if the company was lucky, communicated it out to the leadership.  Then the business went on its merry way, mostly adhering to the strategy.  These businesses often did not have data to indicate if the market was moving in a different direction than their strategy indicated.  If there were gaps between the strategy and reality in the market, this was usually discovered two or three quarters later.

Today, we need our strategies to be more dynamic, and based on what data about markets, economies, currencies and other internal and external factors are telling us now, and signalling about the future.  We need to write strategies informed by data, and create strategies that are course corrected by data, and that are regularly testing new hypotheses about the future direction of strategy.  In other words, we need to reject the old view that the world is static and data belongs in ledgers, and adopt the thinking that the world and the markets in it are exceptionally dynamic and data should be used to tell us what's new and what's next.  Companies and executives need to be led by data, not informed or "driven" by data.

What's this got to do with innovation?

Another good question.  What impact will all this data have on innovation?  In my opinion, it may radically change what we innovate and the products and services we create.  First, more data about customers and their needs will allow companies to make better decisions about the products and services they create.  Hopefully it will not mean they abandon Voice of the Customer activities and engaging with customers and prospects to learn their needs.  Data is valuable but we cannot be overly reliant on it in all circumstances.  Further, all this data and our emerging ability to manage it will lead to new products that the data suggest or new services or solutions that are purely informational.

Whereas the light bulb has been the classic symbol for innovation in the past, a symbol for data and how it is put into use and converted into revenue may be the emerging symbol for innovation in the future.  How we develop new products and services may shift from a mostly creative, manual and emergent process to a more automated, still creative but directed process - directed by insights and data.

If all of this is true...

So, if my analysis is true, the best job to have in the future will be a position that makes sense of all the data that is generated, finds the data that really matters and converts that data into information or knowledge that speeds the business up or helps position a business for opportunities as they emerge.  In other words, some form of data scientist or data strategist.  I distinguish these terms because in my experience many people who frame themselves as data scientists are too close to the data - they see trees and not the forest.  What we need are data strategists - people who see the data and understand its implications but have the ability to pull back and see the bigger picture.  People with these skills and capabilities will be able to dictate their salaries and will be in high demand in the coming years, because they will impact the strategy of a business and its product lines or service lines, as well as direct new product development.