The Most Powerful Realization in Predictive Analytics
It was true five years ago when I wrote this and much more applicable today. Most data science can be improved with a few simple insights.
Vin Vashishta | Originally Published: June 16th, 2015
People see the world as they are, not as it is. That is one of my favorite lines about human nature. It is a statement that explains the divide between visionaries and the realization of their vision. It is why brilliant products fail while being called, “ahead of their time.” It is why so many startups cannot seem to get past 10 customers. When it comes to innovation, it is the barrier to entry.
For predictive analytics, that statement is a call to action. The truth behind predictive analytics is the best insights in the world fail when they do not move people to action. The ROI of a predictive model is 0 if no one acts on it. That puts predictive analytics in an exceedingly difficult position. It is not enough to be right. That is just step 1. There is a lot more work to getting value from predictive analytics that has nothing to do with the initial model.
The challenges for analytics adoption start with how people make decisions. If we were more like computers and made completely rational decisions, analytics would be a seamless fit for our process. We are people. We are creative. We can make leaps of intuition because we do not think or decide rationally. It is the irrationality of our decision process that leads to so many advances but also leads to a lot of biases. Two in particular create the biggest obstacles to integrating analytics into decision making.
People suffer from overconfidence. This is a bias that makes us believe our decision-making abilities are much more accurate than they really are. We have developed several decision-making heuristics over the course of our lives. Especially for successful people, our faith in those heuristics is extremely high. That is where the tendency to go with our gut comes from.
People also suffer from confirmation bias. Not only are we confident in our decisions, we are also confident in our beliefs. Once those beliefs are formed, people are more likely to gravitate towards evidence which supports their beliefs while dismissing evidence that contradicts them.
The problem for predictive analytics is clear. Decision makers are most likely to act on analytics that agree with their beliefs. When analytics are crowbarred into an organization, there is rejection, anger, bargaining, surrender and finally some measure of acceptance. Unfortunately, what is accepted is what aligns with the existing belief structure. Contradictory analytics are spun or ignored. That means the actual ROI of analytics is 0 because there is no change in outcomes.
The most powerful realization in predictive analytics is that data must work within the existing decision-making framework not against it. Our decision-making processes are deeply hardwired. Data will not change the way our brains work overnight. It is going to take a long time to undo all the biases we have built to support decision making under uncertainty. That means analytics must work with our biases, not against them.
Said another way, analytics must support irrational decision-making processes. For us to profit from data driven insights, those insights need to be presented to us in a way that seamlessly integrates with our current decision-making processes. That is step 2 for predictive analytics and it is as big a challenge as getting the initial models to work.
Is any insight no matter how well visualized, presented, and supported going to change decades of individual conditioning? Of course not. Changing behaviors we’ve built over a lifetime does not happen in the course of a meeting. It is a much longer process.
Some people in any given organization will convert quickly and become analytics believers. However significantly more people will stay entrenched in their process, especially those people who have found success with that process…like CEOs, CMOs, CFOs, etc. The most influential people in any given room are the ones who put the most faith in their ability to make good decisions and have reaped the benefits of those decisions. The only tools they will use and benefit from are the one that work with them rather than the tools that try to change them. There is just too much inertia, not to mention success and reinforcement, for that approach to work in the short term.
Telling people what they should do or trying to change their decision processes is the shortest path to analytics failure. The right approach is to reveal uncertainty. I take an “as is”, “to be” approach to this. I show leaders the blind spots in their decision-making process; places where incomplete information is making it difficult to choose the best course of action. Then we review how having more information, eliminating the blind spots, would alter their decision process.
In some decisions we find that analytics would cause a change, in others no change or even paralysis by analysis. With this approach it becomes obvious to everyone where predictive analytics will generate the highest returns as well as where it will cause more harm than good. It is critical to understand decision making within an organization before starting predictive initiatives.
That sounds simple but the decision-making process within an organization is anything but simple. At a high level, the concepts of decision enabling analytics seem obvious and straight forward. That is true when we are talking about an individual. As complicated as people are our individual biases are self-evident and easy to build analytics for. The complexity increases rapidly when decisions are made collaboratively.
The information that will work for an individual making a decision alone won’t work, even for the same individual, making a decision within a team. Our decision processes change in a team environment. The interactions of different perspectives, objectives, areas of focus, skills and biases create a complicated decision system. That is why step 2 is just as challenging as building the initial predictive models; it is another complex systems model in and of itself.
Insights drive better outcomes when we are ready to see them for what they are, not what we are. That is why step 2 is just as important as step 1. When I build analytics presentations, I will sometimes have the same information on 3 slides in different formats. After the 3rd slide someone will ask, “Why did you just show us that 3 times?” I ask the group, “Which slide did you have your ‘Ah Ha!’ moment on? Raise your hand for slide 1. Slide 2. Slide 3.” That is when it starts to sink in.
Analytics need to speak the language of the target audience but that is not the only consideration. Next to personalized presentation, timing is key as well. It is not enough to know what data a team needs to make a good decision. Analytics solutions need to present data at the appropriate point in the decision process. Everything up front, the firehose, is overwhelming and leads to over analysis. Too little information stretched out over too long makes people feel manipulated by the system; like they are playing some kind of game. Timing the right information delivery cadence makes an analytics solution significantly more effective in improving outcomes.
Predictive analytics is about more than being right. It is about more than delivering insights. For a predictive solution to be effective it needs to work with people. Crowbarring a predictive methodology in place does not work. It is disruptive and there is too much inertia to overcome in the short term. Predictive analytics must seamlessly integrate into the organizational process to be adopted and to drive better outcomes. Otherwise it is just another tool with amazing potential that will not deliver.