How to Understand the Forecastable Non Forecastable Formula

Executive Summary

  • There is a misimpression that all every product location combination must have a forecast. This is incorrect.
  • One way to test this is to use a forecastability formula and another way is to use the Brightwork Explorer to calculate the financial benefit of forecasting.

Introduction

I have analyzed a good deal of product databases over the years, and many of the products that I have analyzed from different companies are clearly unforecastable. There is a simple reason for this. Many products that are difficult to forecast have no discernible pattern in their demand history, and without a discernible pattern, no mathematical algorithm can create a good forecast. This is not generally understood. Part of the reason that too much effort is spent on very hard-to-forecast products is due to a misimpression about when statistical forecasting can add value, and when it can’t. This is well said by Michael Gilliland in The Lean Approach to Forecasting:

“The best a forecaster ever can do is discover the underlying structure or rule guiding the behavior that is being forecast, fi nding a model that accurately represents the underlying behavior—and then hoping that the underlying behavior doesn’t change. Unfortunately, there is an element of randomness that surrounds virtually all behavior, and the degree of randomness will limit the accuracy you can achieve.”

The Question to Ask Regarding Forecastability

One of the first questions to ask is whether there is value to actively generating a forecast. In some cases, the answer is no. However, instead of recognizing that a product is not forecastable and adjusting to this reality, more sophisticated mathematics are often employed in a vain attempt to improve the forecast. Clients I have worked for in the past have adopted this philosophy, as have the majority of consultants and vendors I have worked with, and this philosophy is also reflected in forecasting academic papers I have read. Since so many well-educated people agree on this thinking, it must be correct—right? Well, actually they don’t all agree.

A number of academics have written on the concept of unforecastable products, but for some reason, their research does not seem to get sampled and disseminated. However, the scholarly literature is not objectively sampled. In fact, most consultants in don’t read it at all. Proven approaches like turning off forecasting for unforecastable products leads to short, and insufficiently lucrative consulting engagements.

Success Through Using More Sophisticated Forecasting Models/Mathematics for Poor Demand Histories?

However, there is little evidence that sophisticated mathematics can improve the forecast of difficult-to-forecast products, and this is a problem. Some studies do not show improvement from more advanced methods. But first, the improvement is never very large, and secondly, other studies come by later to contradict the original studies.

In addition, complex methods should have to exceed a higher bar. Academics can apply complex methods in a laboratory environment over a few products far more easily than can be done by industry. This fact, along with the point that sophisticated methods are much more expensive for industry to implement than simple methods, is rarely mentioned.

This point is made very well by J. Scott Armstrong:

“Use simple methods unless a strong case can be made for complexity. One of the most enduring and useful conclusions from research on forecasting is that simple methods are generally as accurate as complex methods. Evidence relevant to the issue of simplicity comes from studies of judgment (Armstrong 1985), extrapolation (Armstrong 1984, Makridakis et al. 1982, and Schnaars 1984), and econometric methods (Allen and Fildes 2001). Simplicity also aids decision makers’ understanding and implementation, reduces the likelihood of mistakes, and is less expensive.”

The Inconvenient Truth About Statistical Forecasting

For statistical forecasting, the only products that can be forecasted are those that have a discernible pattern to their demand history, and not all products have this pattern. Forecastability can usually be determined—or at least indicated—without any math by simply observing a line graph of a product’s three-year demand history. If there is no discernible pattern, it is unlikely that the product is forecastable with mathematical methods. (Products that are using just the last few periods to create a forecast are the exception to this rule.) An algorithm that can appear to be predictive can be built for unforecastable products, but more often than not this is an illusion created by the forecaster who over-fitted the forecast.

As is pointed out by Michael Gilliland, just because a model can be built to match the past, does not mean it should be used to perform forecasting:

“The statistical approach is based on the assumption that there is a structure or pattern in the behavior we are trying to forecast. As human beings, we are very good at finding structure and pattern in the world around us—even when none exists. Clouds look like poodles, a burnt cheese sandwich reveals the Virgin Mary, and an ant’s innocent meandering in the sand caricatures of Winston Churchill. We readily come up with lucid explanations of the ups and downs of the stock market and of demand for our products and services. Unfortunately, the patterns we see may not be real, and even if they are, we have no assurance they will continue into the future.”

Stable Product Forecasting

Products that have a very stable history exist at the other end of the continuum of forecast difficulty. Typically, it is very easy to forecast for products with a stable demand history; however, if this is the case, actively forecasting the product does not add very much value to supply planning (the ultimate consumer of the demand plan) because a product with stable demand history does not need to be forecasted. Products with stable demand can be managed effectively and efficiently with reorder point logic, where orders are based upon a reorder point or a reorder period. Very stable and very unstable products converge in their forecasting approach, as is evidenced by the fact that a many-period moving average is equally useful for products with both a stable demand history and products with an unstable demand history.

When both stable and unstable demand history products are run through a best-fit forecasting procedure, the normal result is that both will be fitted with a stable or level forecast. Companies have a very strong tendency to actively forecast all items in their product database without first asking the following question:

“What is the value added by forecasting for the different product categories?”

The Rule of Thumb

The rule of thumb is simple:

“A forecast adds value to the supply planning process when the demand planning system is creating a forecast for a product for which there is a discernible pattern for demand and if the forecast is not simply a constant or relatively constant value.”

Creating forecasts for the entire product database for S&OP forecasting or for other purposes may be important and necessary. However, the forecasting process that results in a demand plan being sent to supply planning can be segregated based on the rule of the value added to supply planning.

What is Being Tested?

The concept behind this test is that for product location combinations with sufficiently intermittent demand, it is not useful or value-add for companies to be forecasting them. For a title to be forecastable, there must be some discernible pattern.

Without a pattern, the best that any forecasting engine can do is create a level forecast which only averages the previous demand volumes. The issue of the diminishing marginal utility to forecasts was brought up by George Plossl in his book Production and Inventory Control over twenty-five years ago.

“The present rational approach recognizes that forecasts will always be subject to error and that, while there are tools available to improve the art of forecasting, the amount of money and effort put into applying such tools rapidly reaches a point of diminishing return. Beyond this point, it is far more profitable to develop flexibility to cope with forecast inaccuracy instead of trying to improve forecasts. The best solution is to develop a formal forecasting program and a system that detects and measures forecast errors and then to react quickly to correct for such errors. This solution is covered in this chapter in detail.” – George Plossl

Forecasts Being Generated at a Sufficient Level of Accuracy

At the point where forecasts cannot be generated with a sufficient accuracy level, it no longer makes sense to forecast the item, and instead, it is recommended that the item is removed from the forecasting system and be managed purely with consumption logic in the ERP system. It is not always the case that these items should move towards consumptive logic, but it is often the case. This is an extremely rarely made recommendation, so I have this method thoroughly explained here.

When to Switch to Reorder Point Planning

Reorder point planning is activated by populating product location fields in either SAP ERP’s Material Master, in SAP SNP’s Product Location Master, or in the material object in any supply planning engine. Sometimes people will say something like “reorder point with forecasts,” however a reorder point is a method used without projections.

The APO or ERP Formula

The determination of whether a product will be exclusively kept in APO, or kept both in APO and in SAP ERP can be determined by a formula which will run off of several key measurements that can be obtained from the statistical export file from a forecasting environment.

The Forecastable or Unforecastable Formula

The formula I have developed is listed below. It certainly is not be used in a pure form for every company and should be adjusted per company. However, one can start by using it, and tune it through showing the results with a group of people with the domain expertise to provide input as to whether the results of the adjusted formula “seems” correct.

=IF(R-Square>0.7,”Forecastable”,IF(AND(R-Square>.15,Standard Deviation/Series Mean<3,Series Mean>3, MAD>Series Mean*0.333333),”Forecastable”,”Unforecastable”))

The Flow of Forecasts

The entire flow of this approach can be described in the graphic below. Then the formula above, or some modification of the formula can be run against each item to determine if the material will be placed in the forecasting engine, or instead only reside in ERP or the supply planning engine and be entirely controlled by the consumptive logic settings.

How Companies and Implementers Ignore the Research

System implementers are not adjusting their approach to be considerate of the research in this area. The results for many decades now is consistent in showing that more complex forecasting techniques do not improve forecastability of difficult to forecast items and certainly do not pay off given their implementation investment, and long-term maintenance costs. A perfect example of this is the Crostons method, a method that Wayne Fu of Servigistics and myself discuss in this are an article as a highly complex formula which is very popular in software, but which has extremely limited areas where it can defeat many periods moving average.

Reorder Points Don’t Provide Visibility?

Comments that reorder points “don’t provide visibility” miss the point that the supply planning system can only provide visibility which is commensurate with the demand plan, which is the quality of the demand plan.

Even the most sophisticated cost optimizer solution for supply planning will not provide visibility if the forecast is of poor quality. On the other hand, products that have very low forecast error and are primarily a level forecast also do not benefit from forecasting effort beyond simple automation. This is because a level forecast is emulated with a reorder point. That is they provide the same result to supply planning.

  • Many companies are spending time forecasting items that are not “forecastable.”
  • Much of the history of computerized demand planning has been spent in attempting to apply very complex methods under the assumption that there is always a method of developing a good forecast, as long as enough effort is placed into forecasts.
  • This approach has had decades to prove itself out and has failed to improve forecast accuracy both at any company I have consulting with and in meta-analysis in the academic literature.
  • What is not recognized is that not every product in the database is forecastable.

The Benefits of Applying No Statistical Forecast

There are many circumstances where creating a statistical forecast is not helpful. This is because there is no pattern that any statistical forecast can use to create a reasonable projection. In most cases, the standard approach is to go ahead and create the forecast in any case and populate the planner user interface with values — values that the planner will typically ignore.

In this article, I am going to question this orthodoxy regarding the forecast. The reason for this is I have come to the conclusion that items that cannot have a good statistical forecast generated for them do not help planners. Secondly, it miscommunicates to the planner. That is — the implication is that the statistical forecasting system is adding value to the process — when in fact isn’t. One of the most significant forecasting issues in supply chain departments is the use of planners for non-value added activities. Creating a forecast tends to both underuse statistical forecasting and make their planners perform too many adjustments and overrides.

Communicating the Limits of a Statistical Forecast

By creating no forecast, this approach is honest with the planners and tells them

“The statistical forecasting methods has nothing to offer on this particular item.” 

This helps in two ways:

  1. It clearly communicates that the planner must control the forecast for this item.
  2. It reinforces the concept that different things need to be forecasted with various approaches. A few examples of these different approaches are listed below:
  3. 100% statistically forecasted items
  4. Statistically forecasted + manually adjusted items
  5. 100% manually adjusted items

Other authors are quite clear that different items require different approaches – and my experience shows that this is always true after I analyze the company’s product database. While this is a constant, very few companies incorporate this knowledge into how they forecast. Yes, the term forecastability has become standard at this point, how many companies use forecastability? The answer is very few. Segmentation of the product database is one of the most powerful ways to manage the forecasting process, allowing companies to have a much better chance of improving forecast accuracy.

Conclusion

Creating no forecast is a significant step in improving the communication of what can be statistically predicted and what cannot to planners. There are other measures as well — for instance creating a report which provides detailed information about every forecasted item — such as what forecasting method it is assigned, whether the forecast should be manually adjusted and other necessary codings.

Search Our Other Forecasting Content

Brightwork Forecast Explorer for Monetized Error Calculation

Improving Your Forecast Error Management

How Functional is the forecast error measurement in your company? Does it help you focus on what products to improve the forecast? What if the forecast accuracy can be improved, by the product is an inexpensive item? We take a new approach in forecast error management. The Brightwork Explorer calculates no MAPE, but instead a monetized forecast error improvement from one forecast to another. We calculate that value for every product location combination and they can be any two forecasts you feed the system:

  • The first forecast may be the constant or the naive forecast.
  • The first forecast can be statistical forecast and the second the statistical + judgment forecast.

It’s up to you.

The Brightwork Forecast Explorer is free to use in the beginning. See by clicking the image below:

References

“Production and Inventory Control: Techniques and Principles 2nd Edition,” George Plossl, Prentice Hall, 1985

Forecasting Software Book

FORECASTING

Supply Chain Forecasting Software

Providing A Better Understanding of Forecasting Software

This book explains the critical aspects of supply chain forecasting. The book is designed to allow the reader to get more out of their current forecasting system, as well as explain some of the best functionality in forecasting, which may not be resident in the reader’s current system, but how they can be accessed at low-cost.

The book breaks down what is often taught as a complex subject into simple terms and provides information that can be immediately put to use by practitioners. One of the only books to have a variety of supply chain forecasting vendors showcased.

Getting the Leading Edge

The book also provides the reader with a look into the forefront of forecasting. Several concepts that are covered, while currently available in forecasting software, have yet to be widely implemented or even written about. The book moves smoothly between ideas to screen shots and descriptions of how the filters are configured and used. This provides the reader with some of the most intriguing areas of functionality within a variety of applications.

Chapters

  • Chapter 1: Introduction
  • Chapter 2: Where Forecasting Fits Within the Supply Chain Planning Footprint
  • Chapter 3: Statistical Forecasting Explained
  • Chapter 4: Why Attributes-based Forecasting is the Future of Statistical Forecasting
  • Chapter 5: The Statistical Forecasting Data Layer
  • Chapter 6: Removing Demand History and Outliers
  • Chapter 7: Consensus-based Forecasting Explained
  • Chapter 8: Collaborative Forecasting Explained
  • Chapter 9: Bias Removal
  • Chapter 10: Effective Forecast Error Management
  • Chapter 11: Lifecycle Planning
  • Chapter 12: Forecastable Versus Unforecastable Products
  • Chapter 13: Why Companies Select the Wrong Forecasting Software
  • Chapter 14: Conclusion
  • Appendix A:
  • Appendix B: Forecast Locking
  • Appendix C: The Lewandowski Algorithm.

Software Ratings: Demand Planning

Software Ratings

Brightwork Research & Analysis offers the following free demand planning software analysis and ratings. See by clicking the image below:

software_ratings