As the name suggests, Customer Health Scores are one of the most powerful metrics a Customer Success Manager has in their tool kit to measure the overall picture of an account’s health. However, simply building a customer health scoring model doesn’t guarantee its effectiveness—far from it. 

If you’re looking for a guide to building an effective customer health score model, our post on creating a customer health score with four metrics is a good place to start. That’s not this post. In this post, we’ll be covering how you can see whether or not your customer health scores (and scoring model) are accomplishing what they’re meant to accomplish—giving your Customer Success team, and business, an accurate pulse on the ‘health’ of your book of business. Before we get into that, we’ll do a quick refresher course of what customer health scores are, factors you should consider including in your customer health scores, and why you should even bother evaluating the effectiveness of your customer health scoring model. Once we cover the basics, we’ll share our five step guide to measuring the effectiveness of customer health scores. 

Refresher Course: What are Customer Health Scores and Why are They Important?

To put it simply, a Customer Health Score is an index made up of several “vital” key performance indicators (KPIs). It gives you an overall idea of how healthy a customer’s relationship is with your product. It will also help your Customer Success team predict future customer behaviors like renewing or canceling subscriptions. 

Customer health scores are most often used by Customer Success teams to determine if a customer or account is "healthy" or "at-risk." While many models exist, in order to actually facilitate proactive and reliable health scoring, you need a model where customers are scored uniquely based on contextual factors. 

Collection and analysis of customer data around everything from product usage to NPS to CSM anecdotes, and everything in between, have become a requirement for modern Customer Health Score models. Analyzing customer data can help SaaS companies formulate strategies that increase net retention, improve customer satisfaction, and minimize customer churn. 

Access to actionable data also allows Customer Success teams to proactively engage with customers and formulate solutions to problems before they become disruptive. Equally important, teams can pinpoint which accounts are high-value and worth a greater level of care versus those that drain company resources.

Do Your Customer Health Scores Cover All Necessary Factors? 

We recommend having multiple health scores for your customers e.g. one for "Product usage" and another for "NPS". You can then deduce an overall health score by calculating the weighted average of each health score. However, doing this manually can get tedious and leave room for human error. Fortunately, with the right technology in place, you can automatically calculate an overall health score using weights, equations, and health conditions you define. As for the factors you should incorporate into your customer health score model, here’s a high-level overview:

The Basics

• Frequency: How much time users spend within a product and how many times they log in. 
• Breadth: How many users within an account are accessing and interacting with a product. 
• Depth: How many of the product's key features are being utilized.

Factors that will take your Customer Health Score to the next level

• Customer lifecycle stage
• Change over time i.e. trends and historical data
• Qualitative data e.g. CSM pulse
• Weight each health score holds in the overall health score
• Customer segment or cohort e.g. trials, MRR range, etc.  

Why Bother Evaluating the Effectiveness of Customer Health Scores? 

The importance of evaluating the effectiveness of customer health scores comes down to two main reasons; 1) Ignorance is not bliss; 2) Customer Health Scores can easily become outdated as your product and customer base evolves. 

Ignorance is Not Bliss When it Comes to Inaccurate Customer Health Scores

If your Customer Health Scores are not accurately depicting the status of an account’s health, especially key customers, you’re setting your team and customers, and ultimately your business, up for failure. So, after building out the baseline model of your Customer Health Score, the key to success is to continuously fine-tune your health scores as you discover more about your customers and their behaviors in and outside of your product by gathering and analyzing more customer data. The TL;DR? The more you let customer data (both qualitative and quantitative) influence adjustments to your customer health score, the better your health score will be at predicting churn.

Outdated Customer Health Scores

For young companies, or any company building out a customer health score for the first time, it is unlikely—understatement—that you create a perfect customer health score model on the first try. The good news is that it’s in a *good* customer health score’s nature to be constantly evolving. That’s especially true for young companies that are changing and evolving at the speed of light. 

Five Steps for Measuring Customer Health Score Effectiveness: A Data Analysis Exercise 

In short the purpose of the following data analysis exercise is to objectively identify trends in your customer data, and ensure that your health scores are accurately indicating churn-risk. Note that if your Customer Success team is not currently leveraging healthscores, you can also use this exercise to help develop your customer health score model. At the very least, this exercise should help you answer the following two questions—well, four questions, if we’re getting technical:

Question 1: Did the customers who you expected to churn actually churn? If so, why?

Question 2: Did the customers who you did NOT expect to churn, churn? If so, why?

One last thing before we dive into the exercise, it should go without saying that if your team is not using your customer health scores correctly/efficiently, then your analysis will be skewed. Do with that what you will. 

Step One: Track Your Churn

It should come as no surprise that churn is the primary performance metric for Customer Success teams at SaaS businesses. There’s a handful of formulas you can use to calculate churn, but you can start by tracking a few basic data points i.e. cancellation, reason for cancellation, and win-back probability. If you already have a formula for calculating churn in place, some additional metrics you may want to consider are customer health score at time of churn, lifetime expansion, product fit, and number of renewals. Curious about what constitutes a ‘good’ churn rate, look no further. Looking for ways to improve your customer churn predictions? We’ve got you covered. 

Step Two: Define Your ‘Churned Customer’ Segment 

Your ‘churned customer’ segment is the cohort of customers you’ll analyze to help you determine customer churn reasons at a high-level. To define this segment, start by identifying customers who have churned within a timeframe that makes sense for your business model. Whether it’s the last three months, six months, or the last year—the key is to have enough customer data to analyze and get conclusive results. Analyzing this segment should help you answer two foundational questions about your customer churn; 1) what is your churn rate over time? 2) What are the high-level categories for churn at your organization? 

Step Three: Add Relevant Data Fields

Once you’ve identified the ‘churned customer’ segment you’ll be analyzing, the next step is to layer in data points that could potentially help you identify trends in the data—such as sales rep, CSM, LTV, etc. 

Step Four: Review and Analyze Customer Health Score Data

Until this point, the previous steps have been focused on compiling and priming your data for the actual analysis. However, in step four, you will actually be taking a look at your health scores and analyzing your customer data. 

Review Customer Health Score Data

When reviewing your health scores, compare current health score data to your ‘churned customers’ segment. You should be looking for things like: missing data points that should be included in your health score based on factors that have caused customers to churn; whether or not your health scores are too specific or too broad, and how they can be tweaked in order to make them more accurate. If, after reviewing your customer data, there are no missing data points and you determine they’re neither too specific or broad, you need to expand your dataset. The last thing you need to do in this step is to gather this information into a segment, containing all the relevant data points that will contribute to your analysis. 

Analyze Customer Health Score Data

Now that you’ve confirmed that you have all the data points you need, you’re ready to analyze. Woo! You can start with these questions, and adjust according to your business needs. 

1. Who are the ‘low risk’ (according to their health score) customers that churned?
2. Who are the tenured customers that are expected to renew?
3. Are there clusters of churn in specific areas i.e. where product usage is low, where there is a low ratio of activated licenses.
4. Who are the CSMs with the highest renewal rates?
5. Who are the sales people that have the highest churn rates?
6. Who are the team members that are excelling at renewing and expanding within their accounts?
7. Is there misalignment between sales and CS on which customers are a good fit?
8. For customers who renew, are they using your product at the minimum level?
9. Do customers renew in their first year, but not in the years following?
10. Percentage of first-year customers who churned?
11. Are all of your health score factors still relevant?
12. Are the initial data points you used accurately depicting customer scenarios?

In addition to asking these questions, you should also solicit your team’s feedback on accounts whose health score’s depicted them as ‘healthy’, but did not renew, in order to identify factors that may have been missed in the initial development of your customer health score. 

Step Five: Report, Iterate, Repeat

Report

Once you’ve completed your analysis, it’s time to share your results with all necessary stakeholders, including your Customer Success team. 

Iterate

With your analysis in hand, make the necessary adjustments to your health score model—add in newly discovered factors, remove factors that are no longer relevant, and/or adjust weights.

Repeat

As with most companies, but B2B SaaS products especially, things are constantly changing—in fact, change is the only constant. Features get added and removed, lifecycle stages fluctuate, positioning changes. So, as your product changes, and how your customer interacts with your product changes, your customer health score model should also change. 

As for how frequently you should be adjusting your health score, a good rule of thumb is quarterly for very young SaaS companies and at least yearly for more mature businesses. With this additional investment of time, you’ll have developed health scores that are like crystal balls—continually predicting what your customers will do next, so your Customer Success team can take action before it happens.

Build Remarkably Accurate Customer Health Scores

Vitally’s Customer Success platform empowers Customer Success teams of all stages and sizes to build world-class customer health score models with our proprietary technology. 

From different health score equations with custom metrics and weights for every single one of your customer segments to tracking multiple metrics—even the unconventional ones—at your accounts, Vitally’s platform doesn’t make your team compromise between power and ease of use.

See Vitally’s health scoring capabilities in action by scheduling a FREE personalized demo today.