I’m reading this fabulous book right now that is perfect for anyone making important decisions—The Book of Why.
The author, Judea Pearl, subtitled the work as “The New Science of Cause and Effect,” and the “new” part is what caught my eye.
In the introduction, Pearl shares how the distinction between causation and correlation—and the science underlying this difference—is relatively new in the human situation. And in my experience in business, the shades of gray between these categories are still lost on many people. Recognize any of these?
- “I can think of at least three customers that we lost because of this feature.”
- “Many of our best people worked at [company X] before…”
- “The top performing IPOs all have great cultures.”
And of course, the supposedly causal relationship that’s so accepted it’s almost never discussed:
- “Our sales target is going up, so obviously we need to hire more reps!”
This dialog reminds me of all of the spurious correlations we find in life—like:
- The divorce rate in Maine is highly connected to the per capita consumption of margarine.
- Total revenue generated by arcades links up directly with the number of Computer Science doctorates awarded in the US.
And my favorite:
- The number of pool drownings strongly relates to the number of films Nicholas Cage appeared in.
The most terrifying jack-o-lantern.
I think this question of causation or correlation is super relevant as the industry analyzes the impact of Customer Success.
As I write that statement, I realize I’ve been asked that question as much as almost any other.
- “How do we quantify the impact of Customer Success?”
- “How do we justify the need for Customer Success?”
- “What’s the ROI of Customer Success?”
What the questioner (sometimes your CFO, sometimes your CEO) is asking is, “Why should I invest in Customer Success?” And more subtly, “How do you know there is a causal relationship?”
This dialog often comes up when CS teams present correlations:
- Time-based Correlation: “We invested in Customer Success this year—look how much our retention rose!”
- Account-based Correlation: “We focused Customer Success on these accounts—look at the saves we made!”
- Leading Indicator Correlation: “Check out how much adoption has gone up!”
Now, if your CFO is like many with whom I’ve worked (I’m not going to pick on my own though—hi, Igor!), they might lean to the skeptical side:
- “Sure retention went up, but how do we know your team is the CAUSE?”
- “Sure those accounts did well, but were they going to do well anyway?”
- “Sure adoption went up, but wouldn’t that happen naturally?”
In The Book of Why, the author, Pearl, talks about the near-impossibility of the “counterfactual”—namely, analyzing what would have happened in the alternate decision path. What would have happened if you hadn’t invested in CS? If you hadn’t focused on those accounts or if you hadn’t driven adoption?
Now, Pearl would argue that sophisticated CS teams (with a lot of volume) might design a controlled experiment to truly figure out the causality. As in the drug industry, CS teams could run an A/B test. They could perfectly separate out a set of accounts upon which to test an intervention (e.g., having a CSM) versus a control group where no CSM is assigned. They would need to be careful to make sure there is no “bias” in the group design. For example, you couldn’t have the test group be the customers who wanted to engage with a CSM (since those might be more likely to stay anyways).
And the good news is several companies have done this. In fact, athenahealth presented at our Pulse World Tour Boston event on experimental testing in their client base.
But the reality is that most companies don’t have the volume, consistency, or processes to perform a real-life A/B experiment. And don’t forget about the “harm” issue. Just like it’s sometimes unethical to put a patient into a control group if the drug is life-saving, it’s often challenging to imagine ignoring a certain set of clients.
So what do we do? Throw up our arms and say, “CS has no provable ROI?”
No! I think we look to our colleagues in other functions to realize how they analyze their own impact. From my vantage point, there are four models:
1. CSM Is Like Support: CSM as a Cost of Doing Business
For some companies, CSM has become part of the product and part of the value proposition. The customer needs the CSM to get value and to work with the vendor. Over time, this cost looks a lot like a Customer Support cost. It’s often categorized as a part of “Cost of Goods Sold.”
Given this, CSM as Support lowers Gross Margins. Hence, the pressure will be about automating and reducing costs to the barest minimum that still works.
2. CSM Is Like Services: CSM as a Product
Many companies are trying to break the logjam of budgeting by self-funding CS out of their own pockets. Specifically, they are creating premium “Customer Success Plans” which often including premium support, training, professional services “points,” and advanced Customer Success. So in a way, Customer Success becomes another “product” (though a service product).
The great thing is that this approach is easier to justify at the CFO level. Indeed, Salesforce pioneered this model. At Pulse events, Splunk and PTC have presented on similar concepts.
One challenge to wrestle with is, “What about customers who don’t sign up for the plans?” The answer from some of these companies is that they use profits from the plans to fund a “for free” Customer Success service at scale.
3. CSM Is Like Pre-Sales: CSM as a Sales Enabler
Some companies have evolved their thinking to look at CSM as a cost of the sale (renewal). As such, they may have a renewal rep and a CSM working on a given client, and they look at the combined cost as a cost of the overall transaction.
These models look a great deal like the justification of a Sales Engineer or “Pre-Sales” resource to partner with an Account Executive for a new sale.
Indeed, just as companies establish target ratios of “SE:AE”, they may have a ratio of “CSM:Renewal Rep.”
In this approach, the CSM makes the overall renewal and/or expansion more effective, just like a great SE helps an AE close.
4. CSM Is Like Marketing: CSM as a Sales Input
The concept of a “lead” was brilliant. Marketers created a well-defined (though often debated!) output metric that could be quantified. Sophisticated companies measure a “cost per lead” and a lead-to-sale conversion rate.
I’ve seen some more sophisticated teams look at their CSM team this way. Allison Pickens, our COO, articulated the concept of the three leading indicators for CS: Customer Success Qualified Save (CSQS) for clients that the CSM team helped bring back to health, Customer Success Qualified Lead (CSQL) for upsell generated by the CSMs, and Customer Success Qualified Advocate (CSQA) for references and case studies generated by the CS team.
If you have the appropriate formulas of conversion from each of these to “lagging indicators” (e.g., new business, renewal an expansion), you could come up with a funding model for CS.
5. CSM Is Like Sales: CSM as Obvious
Or…
…we can get to the point where CSM is so fundamental that we don’t even question it.
Isn’t that how sales works? Most companies have a “capacity model” spreadsheet. You input in the number of Account Executives to hire, along with assumptions around attainment, ramp time, quota, and other factors. And the model spits out a bookings number. Sales reps create sales—it’s like magic!
I think many of us have been in situations where it wasn’t so easy. Sales reps can’t succeed without leads—or happy clients—or products/services. Sales reps often drive deals and make them happen—but rarely on their own. Sometimes they get lucky and get a “bluebird.”
So maybe one day, far into the future, your CFO will ask, “How do we know those Sales reps caused the expansion versus just being there to facilitate? How do you know it wasn’t the CSM that made it happen?”
Maybe I’m dreaming? Or maybe I’ve seen the future!
