How do Complex Forecasting Methods Stack Up?

Executive Summary

  • History shows some interesting insights as to complex methods.
  • There are frequent misunderstandings regarding forecastability which leads to the misuse of naive methods.

Introduction to Complex Forecasting Methods

Selecting the right forecasting methods is a critical component of forecasting. You will learn why complex forecasting methods tend to be overestimated and how to select the right level of complexity.

Misunderstanding Forecastability

Individuals and companies and have a great tendency to overestimate forecastability. For difficult to forecast products there is an interpretation that they can be overcome with more complicated methods. In finance very, complex models are trusted which are often not much better than a guess. For instance, difficult to forecast products have weak demand patterns, and because of this, any statistical method employed has difficulty in predicting the future.

Underuse of Naive Methods

Naive forecasting methods, which should be used for many products, end up not being used because testing is often not performed which compares the naive forecast to the more sophisticated method.

How the Incentives Work to Promote More Complex Forecasting Methods

Software vendors know that companies are looking for this magic bullet and this is why a large amount of marketing effort goes into including more advanced methods in the application. Similarly, academics have incentives to publish on sophisticated techniques because more complex methods are more publication worthy and considered more prestigious than less complex methods.

As an example, it would be tough to get a paper published regarding the benefits of using a simple moving average because academic journals require that researchers submit “original work.”

The Effect of Computerization

There is little doubt; forecasting can be logically broken into pre-computerization and post-computerization. Automation was introduced at an elite level, so in the 1950s researchers users computers to calculate mathematical methods applied to supply chain problems. Gradually computers enabled everything from warehouse workers to supply chain planners to do their job more efficiently.

George Plossl on Historical Inventory Planning

George Plossl had the following to say about historical methods for inventory planning:

“Planning was manual, slow and crude. Large clerical groups calculated gross planning requirements for major components of their products and time-phased (albeit very roughly) these and their procurement. Revising such plans was even more tedious and was rarely done. The capability of massive data storage and manipulation required for sound inventory management planning simply did not exist at that time. Because of this constraint, stock replenishment (OP/EOQ) methods predominated prior to the 1970s. Inventory control was attempted utilizing paper records and electoromachanical desk calculators to apply essentially simple mathematical formulas for order-quantity and safety stock calculations.”

Computers and the Methods Applied

As computerized forecasting methods gave way to more advanced statistical approaches that were encapsulated in software, there became a bias within software vendors to overestimate and over-sell the ability of their algorithms to forecast difficult to forecast products. For whatever reason, rather than introducing complex algorithms that were viewed as magic bullets by executives making the purchasing decisions, but often with little real-world experience performing forecasting or working as demand planners themselves.

How Was This Done?

This was often done while shorting the effort placed into developing either better demand planning interfaces or better data layers. This turned out to be a poor allocation of development resource. This is because of more advanced statistical models most often failed to outperform simpler methods. The infatuation with increasingly esoteric forecasting algorithms prevented executives from understanding that there was an upward boundary regarding how much a forecast for a product with little discernable pattern to its demand history could be improved. As pointed out by Michael Gilliland a very reliable source of forecasting:

“Forecast accuracy is determined more by the forecastability of the demand than by the sophistication of a statistical technique. Statistical techniques work well as long as the historical patterns are properly captured in the forecasting model, there is not a lot of randomness in the patterns and these patterns continue to be the same in the future. Unfortunately, these conditions don’t always apply.”

Michael Gilliland describes the efficiency of statistical forecasting:

“A major benefit of the statistical approach is that it can be a very efficient way to generate forecasts. Once the system is implemented and the models are built, it can run on autopilot with little or not management intervention—which is about as efficient as an approach can get. In addition to being efficient, the statistical approach can also be effective in situations when demand follows consistent and detectable patterns, and continues to behave obediently into the future.”

Using complex methods as being normally more beneficial has been extended into machine learning and AI, where again it is assumed more complex methods are more effective. This leads to the topic of forecasting myths.

Major Myths of Demand Planning

In comparing consensus-based demand planning to statistical demand planning within a historical context, I came across several myths that are believed by many but hamper effective decision-making on supply chain forecasting. I have listed them below:

  1. Simply using statistical demand planning software would significantly increase forecast accuracy.
  2. More complex statistical demand planning methods would improve the forecast over simpler methods.
  3. Consensus forecasting is where the majority of the opportunities lie to improve the forecast (versus statistical methods).
  4. Sales forecasting improves forecast accuracy because sales is “closest to the customer,” aka the more people that are involved in the forecasting process the better the forecast will be.

The reason for the first and second myths was first given life was solely based on the incentives of software companies and consultants to sell.

Who Promoted These Myths?

These actors were able to convince industry to invest in complicated software by the promise of better forecasts, but without ever actually proving that these more complex solutions would improve forecasts. The third myth is related to the first two myths and can be seen as primarily a reaction to it.

Generally, it would be very difficult to find a consulting company that would provide the real story on these myths. Consulting companies have become so software-centric that they are normally partners with a software vendor, and simply repeat what the software vendor says. And most consulting companies are aligned with the largest software vendors, which normally have acquired their applications or built their applications without having many people that understand forecasting. Furthermore, marketing rules supreme at these vendors, so the people most experienced in forecasting don’t set the messaging.

Why These Myths Exist

These are myths because there was never any academic or other evidence to prove these ideas, yet they each in a different time gained widespread acceptance. One myth can lead to another myth, the essential element being that by believing something which is false, to begin with, it makes it easier to draw flawed conclusions a second time around.

Conclusion

Complex forecasting methods are often assumed to be more accurate than less complex simply because of the “wow” factor. Furthermore, more complex methods are less well understood by a larger percentage of the buyers of forecasting systems, and directors of forecasting, giving them what is often an unearned prestige.

Search Our Other Forecast Improvement Content

Brightwork Forecast Explorer for Forecastability

Improving Your Understanding of Your Forecasting Database

Companies required an understanding of the forecastability of their datasets. Brightwork Research & Analysis offers the following software which provides the measure of forecastability.

It is free to use until it receives “serious use,” and is always free for students and academics. See by clicking the image below:

 

References

Forecasting Software Book

FORECASTING

Supply Chain Forecasting Software

Providing A Better Understanding of Forecasting Software

This book explains the critical aspects of supply chain forecasting. The book is designed to allow the reader to get more out of their current forecasting system, as well as explain some of the best functionality in forecasting, which may not be resident in the reader’s current system, but how they can be accessed at low-cost.

The book breaks down what is often taught as a complex subject into simple terms and provides information that can be immediately put to use by practitioners. One of the only books to have a variety of supply chain forecasting vendors showcased.

Getting the Leading Edge

The book also provides the reader with a look into the forefront of forecasting. Several concepts that are covered, while currently available in forecasting software, have yet to be widely implemented or even written about. The book moves smoothly between ideas to screen shots and descriptions of how the filters are configured and used. This provides the reader with some of the most intriguing areas of functionality within a variety of applications.

Chapters

  • Chapter 1: Introduction
  • Chapter 2: Where Forecasting Fits Within the Supply Chain Planning Footprint
  • Chapter 3: Statistical Forecasting Explained
  • Chapter 4: Why Attributes-based Forecasting is the Future of Statistical Forecasting
  • Chapter 5: The Statistical Forecasting Data Layer
  • Chapter 6: Removing Demand History and Outliers
  • Chapter 7: Consensus-based Forecasting Explained
  • Chapter 8: Collaborative Forecasting Explained
  • Chapter 9: Bias Removal
  • Chapter 10: Effective Forecast Error Management
  • Chapter 11: Lifecycle Planning
  • Chapter 12: Forecastable Versus Unforecastable Products
  • Chapter 13: Why Companies Select the Wrong Forecasting Software
  • Chapter 14: Conclusion
  • Appendix A:
  • Appendix B: Forecast Locking
  • Appendix C: The Lewandowski Algorithm.