I've written a lot over the past couple of years on the science of business and evidence-based management. In my "Science of Business Manifesto" blogs a few months back, I used EBM as a synonym for the SOB, drawing on the connotations of both to the conduct of business by the rigorous formulation, measurement, testing and evaluation of alternative courses of action. But a recent interview/article in the MIT Sloan Management Review, “Matchmaking With Math: How Analytics Beats Intuition to Win Customers,” is making me question whether there might be a nuanced difference between the two that's important for BI.
The science of business framework is the similar to the one that's driven scientific discovery for centuries:
- Observe a phenomenon or group of phenomena;
- Formulate a hypothesis to explain the phenomena. The hypothesis takes the form of a causal relationship or a mathematical function (the more of X, the more of Y);
- Use the hypothesis to predict the results of new observations; and
- Perform tests (experiments) with the predictions.
From this perspective, scientific discovery seems more a planned, top-down search for truth driven from guiding theory and hypotheses. Statistical analysis serves the discovery efforts by testing, confirming and adapting hypotheses.
In the “Matchmaking with Math” article, Assurant Solutions' Cameron Hurst articulates a different role for analytics in support of his credit and debt insurance company's call center operations. Repeatedly shown a $10.95 line item on their credit card bill for insurance to protect them if they're unable to pay, customers generally opt out over time – to the tune of a retention rate of just 16 percent. Even an operationally optimized call center replete with skills-based routing, customized desktops with screen pops, and high-end voice recording and quality assurance tools can't elevate its performance over industry norms.
According to Hurst: “we operated under the fallacy – and I believe it’s fallacious reasoning – that if we improve the operational experience to the nth degree, squeeze every operational improvement we can out of the business, our customers will reflect these improvements by their satisfaction, and that satisfaction will be reflected in retention”
So instead of staying with the conventional call center “theory” and “hypotheses” that failed 84 percent of the time, Assurant embarked on a clean-slate, assume nothing, hypothesis-unencumbered, data-driven-only approach to optimize how it interacts with its call center customers.
“Success and failure are very easy things to establish in our business. You either retained a customer calling in to cancel or you didn’t. If you retained them, you did it by either a cross-sell, up-sell or down-sell … So this is what they started asking: What was true when we retained a customer? What was true when we lost a customer? What was false when we retained a customer? And what was false when we lost a customer? For example, we learned that certain CSRs generally performed better with customers in higher premium categories while others did not. These are a few of the discoveries we made, but there were more. Putting these many independent variables together into scoring models gave us the basis for our affinity-based routing.”
In other words, throw hypotheses away and let the data alone suggest relationships from the bottom up. And Assurant did precisely that, introducing a host of math and analytics types with no background with call centers to analyze their operations. They didn't want the intelligence to be contaminated by generally accepted wisdom.
“The approach they took was to break down our customers into very discrete groups … It wasn’t on an aggregate macro but on an individual basis, every single interaction that we recorded over the last four or five years. Looking at all of these interactions let the team see patterns that establish that this CSR tends to do well, historically and evidentially, with customers in these specific sets of clusters …What they also discovered was that the results were completely different from the existing paradigms in the contact center.” In other words, conventional CS hypotheses were off the mark.
The analytics encouraged Assurant to reject many sacred CS tenets. One such belief was the importance of matching customers waiting in the queue with CSRs who had expertise in their particular product lines. “That’s what everyone does in the call center world …What the evidence showed us is that the carbon-based intelligence tends to judge incorrectly. The silicon never does. If the model is set up properly and it has the ability to detect performance through whatever way you tell it to detect performance – by noting cross-sell, down-sell, up-sell, whatever – it will always measure a CSR’s performance correctly and in an unbiased way.” Hurst certainly doesn't hedge his position on experts versus analytics!
Having “big data” covering the gamut of customer background and experience is, of course, central to the “silicon” approach. “We go down to a deep level of granularity. Not body type and hair color like online sites might ask, but we do know that, for instance, certain CSRs perform well with customers that have $80 premium fees, but they don’t do so well with customers that have $10 premium fees. We don’t necessarily know the reason why. Nor do we need to … And therein lies the difference. In our system there isn’t a lot of science behind why these differences exist” Guess where Assurant's puts its dollars on the explanation vs. prediction tug of war in science?
What Assurant has learned from its bottom-up, data-driven approach is that current best practice – the hypotheses and theories generally accepted – is often flat wrong. “The conventional wisdom in the contact center is 80/20 – 80% of calls answered in 20 seconds or less. That’s a promise that most businesses make, because they believe that drives satisfaction …What we learned is that satisfaction has almost nothing to do with that …We’ve done tests that push all the way out to 60/60 – 60% of calls answered in 60 seconds or less.”
And the financial benefits to Assurant for establishing a new, evidence-based “best practice?” “We’ve seen our retention rates, our actual save rates, go as high as 30% to 33%. But that’s not the end of the story. For us, we’re more focused on saved fee rate. Save rate is if two people call in, save one, lose one, that’s 50%. But if two people call in and one is worth $80 to you and the other is worth $20, you save the $80 one, you’ve got an 80% saved fee rate, because you saved $80 out of a total $100 eligible … So while our save rates went into the 33-ish range, even as high as 35%, our saved fee rates went into the 47% to 49% ranges. We’ve seen days where we’ve been in the 58% range. Effectively that means that 58 cents of every dollar that was at risk has been saved. Those are very substantial numbers for us in our business.”
In summarizing Assurant's success, Hurst is adamant that it was their willingness to abandon the currently accepted, state-of-the-art CS practices for a dispassionate, evidence-only based approach that turned the tide. “Another objection is the perception that this is just a skills-based routing solution and that we already have skills-based routing. That’s an interesting one to overcome because, first off, this use of analytics is not skills-based routing. It’s evidence-based or success-based routing. We don’t really care about a CSR’s skills as defined by a skills-based routing system, and in fact we tell you that the skills that you assign a CSR are practically irrelevant …We can show you, based on your data, that you are not fully optimized and that you are relatively randomized in your routing – because effectively that’s the premise statement here. We are taking randomness and chaos and making order out of it. “
So maybe there is a distinction between the top-down, hypotheses-driven science of business and the bottom-up, data-only-driven evidence-based management that's in fact important for BI. And maybe the differences between traditional, hypotheses-testing, parametric statistical methods and dispassionate, prediction-focused machine learning simply parallel this.
What do readers think?