Churn & demand prediction

What the model actually learns

Prediction sounds like the most advanced item on this page. It is the most ordinary, and understanding why makes it much easier to judge whether it will work for you.

A churn model looks at every customer you have ever had and asks what the ones who left had in common in the months before they left. Not opinions — measurable things. How long since the last order, and is that gap longer than their own normal. Whether order size is trending down. Whether they have narrowed from six product lines to two. How often they contact support, and whether that changed. Whether invoices started being paid later.

None of those signals means much alone. A customer who has not ordered in six weeks might be perfectly happy if they only ever order quarterly. The model's job is to weigh them together, relative to each customer's own pattern, and produce one number: how much this account looks like the ones that left.

Demand forecasting works the same way in reverse. It learns that a line sells more in the weeks before Raya, that a particular SKU has been trending up for nine months, and that December is always slow — then combines those into an estimate for next month.

What matters commercially is that these are estimates with error bars, not prophecies. A good churn model does not tell you a customer is leaving. It tells you this account looks more like the leavers than eighty percent of your book, which is enough reason to make a phone call.

Why simple models usually win here

There is an assumption that a better model means a more sophisticated one. On SME-sized data the opposite is generally true, and it is worth knowing so you can recognise when you are being oversold.

Deep learning needs a great deal of data to beat simpler methods. It shines on millions of examples. If you have two thousand customers and three years of orders, gradient boosting or even a well-specified regression will typically match or beat it, train in seconds instead of hours, cost nothing to run, and — critically — be able to tell you why it produced a given score.

That last point is the one that decides whether the project succeeds. A score with no explanation is something your salesperson will ignore by week three. A score that says "flagged because order frequency has halved and the last two invoices were late" is something they can act on, because it tells them what to say when they call.

So if someone proposes a neural network for a thousand customers, ask what it buys you over a simpler model. Sometimes there is a good answer. Usually there is not.

How to tell whether it is actually working

A model that is never checked quietly stops being right, and the failure is invisible because it still produces confident numbers.

The honest way to evaluate a churn model is to train it on older data and test it on newer data it has never seen — never on the data it learned from, which flatters it enormously. We report both what it catches and what it misses, because a model that flags everyone catches every leaver and is useless.

For SME churn, identifying sixty to seventy percent of leavers early enough to act is a genuinely good result. If someone promises ninety-five percent, they either have extraordinary data or they are describing a test they overfitted. Be suspicious of very high numbers.

After launch, the check is simple and you can run it yourself: of the accounts flagged high risk three months ago, how many actually stopped ordering, and how many of the ones nobody flagged left anyway? Customer behaviour shifts, so plan on retraining roughly twice a year.