← Back to Blog
customer successretentionchurnproduct analytics

Deleting survivorship bias from your product

By Jay Nathan, Jonathan Huhn|
Diagram of a WWII bomber with red bullet holes across the wings and fuselage, with the undamaged areas highlighted in green

Beyond The Click · Vol. 43 · October 1, 2026

“Ride your winners, cut your losers” is a popular trope in business and investing, but it hasn’t served the product org well.

It’s fun to celebrate our most successful customers. But many of the ones we lost along the way could have also become winners, and ignoring the lessons about why they didn’t is just leaving future revenue on the table.

Welcome to Beyond The Click by Balboa. Today we’re sharing how we think about bringing lessons from churned customers into your product success lifecycle, including:

  1. The planes that made it home ✈️
  2. Look for important, boring metrics 😴
  3. Start with your last ten losses 🔟

Let’s dive in.

The planes that made it home ✈️

The best example of survivorship bias comes from WWII. Early on, the US military was losing way too many aircraft in battle. When technicians analyzed the damage on planes that survived, they noticed certain regions tended to be more shot up than others, and they began to reinforce the armor on the high-damage spots.

But a statistician named Abraham Wald convinced his colleagues to do the opposite, aka reinforce the areas with no damage. He argued that the undamaged spots must be the most critical, that the returning planes survived precisely because they weren’t hit there, and that if you could analyze the planes that were shot down, you’d find most of them hit in these critical regions.

Survivorship bias, aka learning only from the planes that made it home, nearly sent the armor to the wrong places.

Diagram illustrating survivorship bias with bullet damage on returning WWII aircraft

Look for important, boring metrics 😴

Some metrics are great at measuring the success of “survivor” customers, for example:

  • Net revenue expansion
  • MAU growth
  • NPS score

But the metrics that explain your losses often lie deeper in the product. For example, a legal ops software company we work with discovered that churned accounts often shared a few traits:

  1. Small named-user counts
  2. Low adoption of key features (e.g. <10% adoption of a bulk editing tool)
  3. Just one admin user

These might not be headliners for the case study, but they were clear indicators of low customer value and likelihood to churn.

Start with your last ten losses 🔟

Whenever someone asks us how to begin adding churned customer analysis to their health scoring, we suggest these starter steps:

  1. List your last ten churned customers within a segment of interest (e.g. North America, 50-1,000 employees, <$5M revenue - you have to compare apples to apples).
  2. Identify one commonality between at least eight of the ten (we try to guard against overzealous pattern-seeking).
  3. Verify that pattern, first with your own account owners, and then with the customers themselves (if they’re willing).
  4. If it checks out, then incorporate it as a detractor variable in your health score.
  5. At the end of next quarter, list your newly churned customers and calculate if there was a positive, strong correlation between the new variable and churn. If yes, keep it.

Even with the help of AI, there will always be limited armor to go around. But it’s the customers we lost who best tell us where to put it.