The digital dilemma: putting data to work

Over the history of mankind, we've constantly sought out tools and capital to make us more productive.  From the formation of basic tools to assist in farming to real cultivation and shaping of the land for greater yields, humankind learned to grow food. Further research into genetics, fertilizers and pesticides enabled us to rapidly scale food production.  From early sweatshops to almost fully automated factories, we've learned how to scale manufacturing and get far more productivity from fewer workers and more machinery and automation.

In this manner, we've learned to improve the deployment of human labor, land, tools, machinery and other capital to improve our quality of life.  Now, we must fully engage the asset that we have the most of, that is producing the least for us:  data.  It's time to put our data to work.

Does it strike you as odd that cybersecurity experts speak of needing to protect "data at rest"?  In fact it would appear that data in the cybersecurity world only has two states:  data at rest and data in motion.  Does data ever "work"?  Of course, I'm being a bit pedantic but you get the point.  For far too long we've thought about data as a by-product of our work.  We've collected data and stored it, first in data centers on site and eventually in data centers off-site, or what we now call "the cloud".  But the real question is:  what is all that data doing for us?  Is it an under-utilized asset that can be put to work for more effective gains?

The new land, or the new oil?

Lately, people are talking about data as the new oil - a cheap and easily accessible resource.  The clear difference between oil and data is that oil is a rapidly diminishing resource that gets more difficult to obtain and extract the more we use it.  The reverse is true for data.  We will never run out of data - the sources may change, but as long as the internet exists and people are engaged on the internet, we will create data.

I think the analogy is wrong.  I think we should be thinking about data as the new land.  I realize it's not a perfect analogy, because we also have a finite amount of land. But land comes in different shapes, sizes and configurations, good for different things.  If we take the US as an example, in the 1700s New York City was limited to the southern tip of Manhattan, and the rest was used to grow crops.  Long Island was basically a farming community.  Over time, as the city expanded, other uses for the land emerged - for manufacturing, for banking, for dwelling and other uses. Today, Manhattan is mostly developed, and governments have developed regulations regarding how land can be used.  

In other words, we put the land to work for us, and over subsequent generations the land creates value in different ways, either by growing crops or housing manufacturing or apartments.  Land increases in value based on its scarcity and based on what you can do with it or place on it.  To the extend that it comes full circle, and land becomes valuable when you place nothing on it.

We don't have a perfect analogy for data

Since we don't have experience with infinitely replenish-able assets, there is no good analogy for data, but the point remains.  As we've done with other assets - labor, land, equipment - we need to put the data we have and are generating to work, get it up off the couch and out of the air conditioned data centers and out working for us.  Oh, but you might say, we are doing that - the artificial intelligence guys and the machine learning folks are working on that now.

This is true, but misleading.  While AI and ML teams are working with data, they are focused on highly specific use cases of data in narrow niches, and only working with tiny subsets of data.  To the greatest extent, many of the AI and ML teams can't work with a lot of the data that exists, because the data is too...well, it's too messy, too discontinuous, exists over too short a time horizon and a lot of other reasons.  A massive amount of data we have on hand is not working for us, and we need for it to become productive or move out of the basement.

Needing a mnemonic

Now, and in the near future, we need to be asking ourselves some interesting questions as we create and store data.

• First, why are we collecting this data - what near term or longer term benefit do we hope to achieve?
• Second, what about this data makes it interesting or useful?  What "meta-data" should we be attaching to the data?
• Third, if we hope to put this data to work, what conditions must exist for the data to be useful and valuable?
• Fourth, is this data "enough" to be useful?  Does it need to be augmented with other data?  What lifespan of the data is necessary in order to gain more value?
• Fifth, how might we put this data to work, either in a statistical analysis, a predictive model, to drive or automate a process, or to provide insight into new or emerging opportunities?
• Sixth, what kinds of people do we need who can curate the data, clean and consolidate the data and eventually manipulate the data into useful information and analyze and act on the results?

Putting data to work is paramount, if our history of leveraging labor, land and equipment is any guide.  This is far too important to leave to the IT folks, or even the AI and ML folks.  It requires that everyone in an organization work to get the most out of the data you have - which I suspect is the least utilized asset in the company.  More importantly, what is your plan for the deluge of data that will be generated as we go through a digital transformation?

Farmers, Ranchers, Machinists, Data-ists?

If farmers and ranchers are people who seek to get value out of land, and machinists are people who seek to optimize machines, what do you call someone who is seeking to get the most value from data?  This is a job for data scientists, but not just them alone. Perhaps over time data scientists and others will be responsible for getting the most absolute value out of data, but until then it is the responsibility of anyone with access to the data, and therein lies the rub.

In the oil boom, basically anyone could drill a well as long as they owned the rights to drill (and often in the early days the legalities happened post facto).  Today, the real question is:  who owns the data, and who has access to the data?  In corporations, corporations own the data, and traditionally have limited access to the data.  One must proceed through the cybersecurity guys, to the IT guys and the data center guys to even get access to the data.  We've hidden, protected and partitioned the data so very few people can access it.  Further, most organizations have arcane rules and challenging tools to use to access and manipulate data.  No wonder data is at rest.  We've given it a comfy place to reside, little responsibility and limited access to people who need it. 

Doesn't "data want to be free"

Strange that the rallying cry for many at the start of the internet was that "data wants to be free", but just as we are gathering enough of it to matter, and just as we are starting to develop the tools and people to begin to make sense of it, data seems more isolated, more locked down than ever before. 

As digital transformation unfolds, and the real value in the economy is not in labor, land or equipment, the real value proposition will be in putting data to work.  Which means we need the right data, the right people, the right access to the data, and the right questions to ask.  This is a job that is far larger than the IT department and the data scientists. It is a job, and a responsibility, for everyone in the organization.

Practicing for an emerging future

I'm writing this in the stages of the corona virus outbreak in the United States where we've finally decided to take the virus and its impact seriously.  While the virus may not be as deadly as some other viruses, it is clearly contagious.  I think after weeks of ignoring it or wishing it away, or waiting for a miracle cure, we've finally decided to do what must be done - that is, more social distance to reduce the spread.  That's good news, because we've finally decided, after weeks or months of dithering, to take action.

I'm not a virologist or a doctor, more a person who is interested in the unfolding future and the opportunities and challenges that will be created as the future emerges.  New futures create new opportunities, new customer segments, new needs and wants, and also eventually introduce new business models and new entrants or substitutes.  If that sounds a bit like Porter's five forces in action, you are right.  Thinking about the future and how it will unfold is critical to the success of any business, large or small, but also any government, country, community or individual.

With the importance of thinking more carefully about the future and its implications in mind, there are a couple of points I'd like to make in this post:

1. There are no straight lines to the future
2. Shifts are caused by far more than technology
3. Demographics is destiny
4. There is a heavy hand
5. Expect the unexpected

No straight lines

We know this, but we choose to ignore it.  The future will be different in some degree than the present.  I'll address this more shortly, but as the pace of change increases, as we have more interaction and more trade, as there is more and more widely dispersed technology, the future will be different than today.  There are no straight lines, yet in most cases (businesses, governments, societies) we act and live as if the future will be a carbon copy of today.  In other words, there will always be horse drawn carriages, regardless of the advent of the car.  There will always be home delivery of milk and a newspaper.  The majority of families will consist of a traditional nuclear family, owing a home and two cars.  As we consider what certainly seemed like "givens" in the recent past, we can see how much change has happened in a relatively short period of time, to people who were fairly surprised when these changes occurred.

The sooner we decide, as governments, companies, societies, organizations and so forth that the future will be different, and that difference has consequences, the more likely we may be to attempt to understand the potential future and take action. 

Shifts are caused by more than technology

When we do look at the future, we imagine that technologies change and everything else remains the same.  When we watch movies or read books about the future, new and amazing technologies are presented, but the societal, religious, political, familial and other structures remain virtually the same as today.  Many, many shifts happen because of extraneous forces, and many of those aren't the introduction of new knowledge or technologies.  Consider how the Black Plague changed Europe, basically ending the feudal system.  As there were fewer people to work, a person's labor became valuable, and gave laborer's more control over their earning potential. 

Political shifts can have dramatic impact on the future.  What if the Wobblies had won in the US in the early 20th century?  What if Ronald Reagan had not won the presidency? Would the US look like it does today with just a few changes in political leadership?  President Obama significantly changed a significant portion of the economy with new healthcare rules.  Does anyone think that the US won't be different based on a Sanders versus Biden presidency?

Political, economic, social and environmental movements and factors have tremendous impact on the future. Often more impact than we recognize, except in hindsight.  Consider, for example, the Dust Bowl that drove many mid-westerners to the west coast.  An environmental change that largely emptied several midwestern states but fully populated and changed California.

Demographics is destiny

Most countries in the world today have a birth rate below replacement rate.  What this means is that live births are less than deaths, meaning the population will shrink unless replaced by immigration.  Many countries are rapidly aging.  Countries like Russia and Japan, where birth rates have been declining, the population aging and where immigration is discouraged will age rapidly, creating new challenges and opportunities.

The thing about demographics is that it is observable and it is closely tied to the destiny of any organization.  The Shakers, a religious movement in the 1800s, encouraged celibacy, making it difficult to sustain an organization that did not attract a lot of converts.  Their religious movement has all but died out.  Younger people will dictate the future of an organization or country to a great extent, so understanding what younger generations think, want, hope for and will act on is exceptionally important in the medium and long term.

The heavy hand

The governing factor over demographics is the power and investment of the older generations, and the amount of investment or debt left behind.  If much of the wealth and power of any organization, government or country is tied up with the elderly, then the rate and impact of many societal and demographic trends will be somewhat stymied by the lack of access to money or the inheritance of debt which limits where change may occur. 

But this isn't just a screed about old people or national debt, it's a warning that old models and old perspectives often have to be updated or even rejected when looking at future needs and opportunities.  It's critical to understand when past experience, perspectives and knowledge are helpful in determining what to do, and when they create roadblocks or barriers about how to understand the future.  Consider that Thomas Watson, the CEO of IBM thought the total market for computers in the world was about 5 machines.  Was he an idiot?  Of course not.  He just had locked into a specific way of thinking that governed how he looked at the future.

The unexpected shifts

Finally, there are unexpected shifts or impacts, not necessarily brought on by emerging technologies or demographics, or political movements.  Corona virus is an excellent example of an unexpected but powerful shift in the way we work, think and govern.  While corona virus by itself is not a trend, it is a disrupter, and could be a harbinger of other widespread illnesses or pandemics, or simply a significant one-off event.  What our thinking about the future often fails to anticipate, because the probability is so rare, is a major, unexpected outcome like corona virus.  We fail to anticipate and prepare because the impact is unpredictable and the costs to adequately prepare for something like corona virus is exceptionally high.

But globalization, trade, interconnected airlines, access to education and travel means that what were once local outbreaks, easily controlled and perhaps quarantined, now spread exceptionally quickly.  It will be interesting once the corona virus is over to look back to the Spanish Flu and the Black Plague and understand how similar or different these outbreaks were.  Given that medicine and public health have advanced tremendously over time, we are still using the same approach to slow the corona virus as we used in the Black Plague - isolation.

Prediction, Assessment, Preparation

If we are doing our jobs - in governments, companies and society - we should be more alert to the accelerating rate of change, the higher degree of impact that any trend or unexpected shift is likely to have, and be constantly thinking about the future and how it will unfold. 

We know, for example, that a major flu will emerge each year and regularly have people inoculated against the flu.  But increasingly, due to more trade and interconnectedness, we are seeing more viruses or other diseases introduced and spread.  We should predict that this is the case and predict what may happen, and try to assess what may happen and how to prevent or slow changes and do a better job preparing.  This is true of viruses or illnesses, which create future challenges, and is also true of trends that create new opportunities or new segments with new needs.

An investment in your future

It is simply unacceptable that any government, company or even individuals fails to do a good job of watching trends, making assumptions about what is going to happen in the future and failing to prepare or anticipate future change.  We have much of the data we need to make basic assumptions, and have enough experience to understand the increasing and somewhat chaotic nature of change.

Anticipating and thinking about the future, even when considering scenarios that don't pan out, give decision makers far more information about the future and potential outcomes and decisions.  We simply need to get better practicing for emerging futures.  The governments, companies and people that can constantly practice for the future will be better informed, and better able to withstand the changes that will unfold.

How supply chains and ecosystems will be shaped by the corona virus

Right now the global economy is in a bit of an uproar over corona virus and its impact on people, businesses and economies.  Starting in China, the virus has significantly slowed the production of goods from a core portion of China, which will have knock on effects to supply chains in Europe and in the US.  Travel and trade will likely slow, as governments try to contain or mitigate the spread of the virus.

We are likely to learn some new things from this pandemic, but more importantly is to see the opportunities for innovation, especially where there are gaps or weaknesses in the somewhat surprisingly fragile ecosystems, and to think about what the future looks like after a global response to the virus.

First, let's talk about fragile ecosystems.

Built for the best case

Over the last 15-20 years, as China has built a powerful manufacturing base, the global economy has increasingly become networked.  So many businesses rely on core products originally built in China that entire supply chains and shipping routes have emerged to support the flow of goods and services.  China even has a "belt and road" strategy to strengthen the supply chain and flow of goods.  However, it is apparent that in this case, and many other cases, these supply chains are somewhat fragile, and built without consideration of some important issues.  If a simple virus can shut down an entire supply chain, then the supply chain was built with the best case in mind, or has not considered the impact of factors that are adjacent to the supply chain but do have some influence.  Clearly the virus can infect workers in China, which leads to plant shut downs and less exports.  Perhaps the virus can migrate on the products from China as well - I don't know that to be true, but perhaps it could be.  Since China is often the epicenter for much of initial flu and virus origin, it stands to reason that corporations and governments that structure supply chains should consider the impact of virus origination in China and strengthen or protect the origin of the supply chain.  Or, corporations may eventually decide that housing the vast majority of manufacturing in China, close to the origin of yearly viruses, is a bad idea.

Of course this isn't really a new insight - placing all your manufacturing eggs in one basket is rarely a good idea.  Diversifying the development and manufacturing of a product could mean more costs, but could also mean that production continues when one plant suffers from an earthquake, or a typhoon, or other natural causes. 

I would think that many companies will be reconsidering their over-reliance on China as a manufacturing location, to seek to diversify manufacturing to more locations. The costs will increase, but the ability to continue production, and to shift some production away from China, could be a positive move.

This isn't the first time

What many people fail to recall is that we've been through something very much like this before.  In the years leading up to World War I, the economy was globally connected, with a lot of trading and economic development around the world.  After the war and partially due to the Spanish Flu, which was far more deadly than the corona virus, economies increasingly became nationalistic and raised tariffs.  As a short history lesson, more people died from the Spanish flu in 1918-1919 than died in the war.  While corona is serious, we've seen far more deadly pandemics.

So the second question becomes:  do we repeat history by closing our borders and raising barriers and tariffs as the virus continues its spread, and where does that lead?  In the 1920s and 1930s it led to a global depression, and eventually a world war.  I'm not quite so pessimistic about the future - I think we all stand far too much to lose in an economic downturn.  However, we should learn from the past.  Santayana said that those who fail to understand history are doomed to repeat it.

The good news out of this is we can anticipate what the outcome looks like, and we know what the likely outcomes of some of the options are, and can make better choices.

What comes next

Perhaps the most important question of all is:  what comes next?  The corona virus will end, hopefully in just a few months, and the nations and people will breathe a collective sigh of relief.  Then we'll have to make some decisions.  Do the day to day operations of all the supply chains go back to where they were, with little or no change?  Do human interactions continue as before, or do we put more plans and programs in place to prepare for more epidemics?

So, for example, do companies adopt far more telecommuting and working from home?  Do universities accelerate distance learning for more classes, to keep people from congregating?  Is there enough internet bandwidth to support far more people working from home?  Will companies continue to place all their manufacturing assets in China, or will they look to distribute the manufacturing to other locations, or perhaps return them to home shores and run them with robots?

Emerson said that "a mind, once stretched by a new idea, never returns to its original dimensions".  It could also be true that a country or government or people, once confronted with a crisis, won't accept the solutions from the past.  Perhaps the biggest impacts from the corona virus will be the way we live and work in the future.  Hopefully we won't simply isolate ourselves from each other, but will find new ways to work together and congregate regularly but safely.

This is an inflection point - a time when the way we live and work could be changed by how we react and respond to the threats of the virus, which seems to have an inordinate economic impact.  We innovators ought to be looking at future trends and outcomes to decide what attributes or factors of our economy are likely to change, and what that will mean for business and society.  We have the learnings from the past, and plenty of signals and signposts along the way.  As our minds are stretched and challenged by this virus, how much will our future dimensions be changed?

Overlooked in the digital transformation: Ethical, Societal, Legal, Regulatory issues

A friend and I were talking recently about the dramatic change that digital transformation will create.  The potential impact on our lives is tremendous, from autonomous vehicles that whisk us to and from work, to drones that deliver goods directly to our homes or other locations, to the insights generated from data harvested by IoT devices. We can expect significant upgrades in customer experience and business models from the use of data massaged by artificial intelligence and machine learning. 

Of course, on the flip side, there are concerns. Governments and private companies are gathering more data about individuals than ever before.  What will they do with that data?  Who decides who owns your image, and what your rights are to privacy?  What will Apple, Google, Facebook and others do with all the data they collect about your activities and selections online? 

We've reached a point where the technology and what it can do has easily surpassed the consideration of its consequences and the impact on our ethical and regulatory frameworks.  This isn't the first time this has happened, however.  Recall just a few years ago that a scientist manipulated genes in a fetus, causing an uproar in the scientific community.  The fact that we can do something because of advances in science does not mean that everyone is prepared for, or comfortable with, the ability to do it.

In the past, non-technical factors - ethical, moral, legal, societal, regulatory - slowed the advancement of science.  If we look back only a few centuries, we can see that ethical, religious and moral issues resisted science and what it could do or tell us. From Galileo confronting the church on the geocentric model of the universe to doctors desecrating graves to get access to human cadavers for dissection, science has often been held back by societal, regulatory or ethical concerns. 
 

Today, the opposite is true

Today, science and technology move so quickly that ethical, moral, societal, legal and regulatory frameworks struggle to understand what is happening, much less keep up with the changes.  Airbnb can enter a city and dramatically change the housing landscape before the city government determines its opinion on short-term leases.  Facial recognition programs tied to cameras in public places can capture millions of images and those images can be used to determine the attitude and potential behavior of citizens.  The speed of implementation has shifted - science and technology move faster than people, societies and beliefs.  The impacts that new science and technology have on how we live, the rules we agree to, the expectations we have about security and privacy are not aligned with all the new technology.

Confounding the new technology will be the fact that society in general, our politicians and the laws and regulations they create and other administrative rules and burdens won't change quickly, and will delay the promise of many of the emerging technologies.

Pushing more than one rock uphill

The use of autonomous vehicles and drones creates an excellent example.  The adoption rate of these devices is NOT based on technology.  Autonomous vehicles are at least as safe as human drivers are today, but that does not mean we'll see rapid adoption.  The barriers that exist are important and diverse.  They include:

 - Multiple jurisdictions.  Just because California likes and supports AVs does not mean that Arizona or Nevada will.  This could even be true between local jurisdictions or counties.  Who will risk buying a car that can be used in only certain locations?
 - Insurance.  Until the insurance industry can determine how to price the risk of autonomous vehicles, and more importantly, who bears the risk, it will be challenging to get a lot of autonomous vehicles on the road.  Moreover, who owns the risk?  Does the passenger in an AV bear the liability for an accident, or the company that is technically in control of the vehicle?
 - Blended traffic.  The perfect world for AVs is when every car is an AV, because they will be more consistent and predictable.  As long as AVs and humans are sharing the road, the level of danger and unpredictability goes up dramatically, meaning that more accidents are likely, which will probably be blamed on the AVs.
 - Standards.  While it is good to have competition in AV technology, we will probably need a unified set of standards so that cars and the devices that control them all work with and on a set of agreed standards. 

Thus, the full scale implementation of a technology that is already reasonably mature will not depend on the technology, but on the legal, jurisdictional, administrative and societal acceptance.  Who is doing the work to prepare the population, revise the laws, change expectations?

Why drones are an even more interesting challenge

Keeping the challenges of the autonomous vehicle in mind, let's make the problem slightly more difficult, and 3 dimensional, by considering the challenge of building a drone business.  We add the complexity of the AV, with the added issue of flying a large object overhead, where risks are greatly increased, and where there are even more regulatory bodies involved (FAA).  If you are trying to build a business in this sector, you are facing a problem that even Sisyphus would find difficult - pushing several different rocks uphill at the same time.  Two points are critical from this sentence:  Several rocks and simultaneous advancement.

Several rocks:  to win in this space you have to 1) demonstrate the technology works, and provides benefits over existing solutions first,   2) convince local, state and federal authorities that the benefits are worth the risks, and to change laws and regulations, 3) convince people within the industry that the new solution is worth adopting, and keep your end customer or consumer from turning against the technology, and 4) demonstrate to the consumer that the value of the new technology outweighs the cost.

Simultaneous advancement:  To win in these very complex technologies, you'll need to do all of these things relatively simultaneously. This is the definition of a "wicked" problem - one that has many participants and constituents. You don't want to gin up too much excitement in the consumer space and be unable to demonstrate the technology works or adds value.  You don't want to over invest in a technology only to find that regulators are unwilling to change the laws to accommodate your new technology.

Many public implementations of digital technologies - especially those that interact with the public like robots, AVs, and drones - need to consider the ethical, moral, societal and regulatory challenges.  While Asimov may have created the 3 laws of robotics, his stories don't consider how the population reacted to the advancement of robots, how they were compensated for the loss of their jobs.  While AVs, drones and robots are risky because they could interact with humans, other digital technologies like AI, facial recognition, natural language processing and IoT are also interesting and potentially problematic, because they could lead to a loss of privacy and security.

How do we prepare the population for the advent of digital technologies?

What we need is more thinking and more investment in the secondary and tertiary impacts of digital transformation - what does it mean that governments have more of our data and images?  How should they use them?  What could it mean that robots and other digital transformation eliminate jobs?  What risks are we willing to accept to live with and among AVs and drones, and what are unacceptable risks?  How do we condition people to the fact that technology will become more prevalent and more overt in their lives?

As we've seen, in the not so distant past, societal norms, religious authorities and governments had significant control over the pace and impact of new scientific advancement.  Queen Elizabeth (the first one) once rejected a patent for an automated loom because she was concerned that her population would lose jobs.  Today, the reverse is true - we base our hopes, companies and futures on rapidly emerging science and technologies, often with little understanding of how much change these technologies unleash, how unprepared the population is for the actual impacts, and how existing laws and regulations may limit the value or use of technology.

There's an opportunity in here somewhere for someone to do some serious thinking, and bringing together different constituencies to help provide a pathway for more information, more analysis of the impact of digital transformation, more sense of the changes necessary in legal and regulatory frameworks and more understanding of the risks to income, privacy and security.  Who is doing this work?

The best way to address this is to bring people from different disciplines together in one team or organization - people who are interested in the technology of course, and what it can do, as well as people from the political realm (who can change laws or create new ones), funding mechanisms, education entities (because we need to educate both young people and older people on the possibilities and impacts of new technologies), people who focus on privacy and ethics, sociologists and of course we'll need experts in the law.  It would work best if we could create integrated information about these topics, with these diverse perspectives, because the technologies will have impacts that cross all of these functions - and probably more.