Forecast Error Myth #4: Most Forecast Error Measurement Supports Identifying How to Improve Forecast Accuracy

Executive Summary

• The standard forecast error measurements are universally accepted.
• What is rare is to ask the question of whether they direct companies to improve forecast accuracy.

Video Introduction: Forecast Error Myth #4: Most Forecast Error Measurement Supports Identifying How to Improve Forecast Accuracy

Text Introduction (Skip if You Watched the Video)

Forecast error calculation comes typically from accepting one of the standard forecast error calculation methods. Often, companies think they are finished with all of the decisions as soon as they select a method. And the forecasting literature, software vendors, and consulting firms do little to disabuse them of this belief. The truth is that there are many more decisions to be made — and even the first decision of the forecast error calculation method is typically not helpful in providing them with a forecast error that points them to forecast improvement. You will learn why forecast error measurement is instrumental in improving forecast accuracy.

Our References for This Article

If you want to see our references for this article and related Brightwork articles, see this link.

Myth #4: Forecast Error Measurement Supporting Forecast Accuracy Improvement

It is often overlooked that the main point of forecast error measurement is forecast accuracy improvement. However, in virtually all cases I have seen, forecast error is only reported as a single point error.

There is a second problem where the context of the forecast error is not presented.

For instance, a specific number does not mean all that much without context.

1. Low volume items will only very rarely have a low forecast error.
2. High volume items will only rarely have a high forecast error (unless promotions or other factors damage the demand history). However, context-free forecast error reporting is the norm.

For All of the Myths, See the Following Table

What Is The Purpose of a Forecast Error Measurement?

Forecast error measurement should point to where to make changes to improve forecast accuracy. If the forecast error does not serve this purpose, then it just becomes an inert number.

Reporting Out Forecast Error from the Demand Planning Department

The forecast error measurement in forecasting applications is primarily for the user or planner. As we cover in the article Forecast Error Myth #1: One Can Rely on The Forecast Error Measurement in Forecasting Applications, this forecast error is only calculated for the product location combination.

Companies will then create a custom report that pushes the forecast error, or planners will export the error to Excel, perform the calculation in a spreadsheet, and provide it to their Director of Forecasting/Demand Planning, who then provides it to the broader company. The forecast error report is often presented with great anticipation as if this is the “horse’s mouth” on forecast error for the period.

It isn’t. It does not direct the company were to make forecast accuracy improvements.

A Better Approach

Observing ineffective forecast error measurements at so many companies, we developed the Brightwork Explorer to, in part, have a purpose-built application that can measure any forecast. The application has a very straightforward file format where your company’s data can be entered, and the forecast error calculation is exceptionally straightforward. Any forecast can be measured against the baseline statistical forecast — and then the product location combinations can be sorted to show which product locations lost or gain forecast accuracy from other forecasts.

This is the fastest and most accurate way of measuring multiple forecasts that we have seen.

Why Do the Standard Forecast Error Calculations Make Forecast Improvement So Complicated and Difficult?

It is important to understand forecasting error, but the problem is that the standard forecast error calculation methods do not provide this good understanding. In part, they don't let tell companies that forecast how to make improvements. If the standard forecast measurement calculations did, it would be far more straightforward and companies would have a far easier time performing forecast error measurement calculation.

What the Forecast Error Calculation and System Should Be Able to Do

One would be able to for example:

1. Measure forecast error
2. Compare forecast error (For all the forecasts at the company)
3. To sort the product location combinations based on which product locations lost or gained forecast accuracy from other forecasts. 
4. To be able to measure any forecast against the baseline statistical forecast.
5. To weigh the forecast error (so progress for the overall product database can be tracked)

Getting to a Better Forecast Error Measurement Capability

A primary reason these things can not be accomplished with the standard forecast error measurements is that they are unnecessarily complicated, and forecasting applications that companies buy are focused on generating forecasts, not on measuring forecast error outside of one product location combination at a time. After observing ineffective and non-comparative forecast error measurements at so many companies, we developed, in part, a purpose-built forecast error application called the Brightwork Explorer to meet these requirements.

Few companies will ever use our Brightwork Explorer or have us use it for them. However, the lessons from the approach followed in requirements development for forecast error measurement are important for anyone who wants to improve forecast accuracy.