How to Best Understand Demand Sensing and Demand Shaping

Executive Summary

• Demand sensing and demand shaping are two methods that are often promoted to improve forecasting. However, one of these methods is barely ever applied.
• The other is not a way of improving the forecast but is presented as one by vendors.

Introduction

Demand planning always has new terms in supply chain forecasting that are necessary to keep up with. Two of the present ones are demand shaping and demand sensing. I decided to include both in an article because, for some time, I would get the two confused for one another, and it is essential not to do as they are unrelated, except for being two terms that are currently popular.

Our References

See our references for this article and related articles at this link.

Demand Shaping

Demand shaping is the process of creating incentives with customers that smooth demand or eliminating pre-existing incentives such as promotions or end-of-quarter pushes, which distort the demand history, making forecasting more challenging to perform.

Demand Sensing

Demand sensing is the use of a procedure to analyze the demand history to gain new insight into how to develop a better forecast and make changes to the forecast in the short term.

This entry into Wikipedia on the topic is patently ridiculous.

“The typical performance of demand sensing systems reduces near-term forecast error by 30% or more compared to traditional time-series forecasting techniques. The jump in forecast accuracy helps companies manage the effects of market volatility and gain the benefits of a demand-driven supply chain, including more efficient operations, increased service levels, and a range of financial benefits including higher revenue, better profit margins, less inventory, better perfect order performance and a shorter cash-to-cash cycle time. Gartner, Inc. insight on demand sensing can be found in its report,” – “Supply Chain Strategy for Manufacturing Leaders: The Handbook for Becoming Demand Driven.”

This entry was made by a software vendor selling demand sensing software. No forecasting methodology produces a reduction in forecast error by 30%. If the forecast error were 30%, demand sensing improved the forecast by 30 % x 30 % or nine percentage points. I would very much like to see the hard data on those numbers. Secondly, what is near-term forecasting? Forecasting is produced long-term, and the sales orders exceed or are short of the forecast. I am not sure that “near-term” forecasting is a legitimate term. If the lead time is longer than the duration of the short-term forecast to sale, how to adjustments to the forecast help?

And this gets to the next topic.

Who Provides Information on Demand Sensing?

Supply chain software vendors currently and primarily control the information on the definition of demand sensing. Demand sensing did not come out of the academic community, so there have been few unbiased descriptions.

Curiously, I found this article plagiarized as I updated it as I found this quote from Logility.

That is unbiased in that the definition is not promoted by someone trying to sell software.

This is why I have developed the definition below:

“Demand sensing is the adjustment of forecasting inside of the lead time of the product, and therefore when the supply plan cannot respond. If our lead time is 2 weeks, then demand sensing means changing the forecast less than 14 days out. Demand sensing is the adjustment of forecasting inside of the lead time of the particular product, and therefore when the supply plan cannot respond.”

Because demand sensing changes the forecast within the lead time, demand sensing cannot be considered a forecasting approach.

And once again, Logility agrees.

Well well.

To understand this, it is essential to comprehend that while broadly speaking, a forecast is a prediction of a future event, in practical terms, a forecast is a prediction of a future event that is given with sufficient advanced notification to be worthwhile. For instance, a forecast could be improved for a football game by waiting until 1/2 the game is over. However, it is too late to place a bet on the match when half the game is over.

What is and What is Not a Forecast

Therefore, the forecast is not particularly useful because it occurs within the lead time of when some benefit from being received from it. The forecast for the game could be further improved by waiting until the minute before the match ends, but again, it is hard to see how anyone would accept this as a forecast. One could not want to compare the prediction accuracy of a person who forecasts games while in progress versus those who forecast the game before the game begins. This, of course, brings up the topic of demand sensing and forecast accuracy “improvement.” Demand sensing does not help supply planning and does not affect inventory management. Furthermore, it reduces the data quality of forecasting history.

I have a better definition of demand shaping.

“Demand sensing is the adjustment of forecasting inside of the lead time of the product, and therefore when the supply plan cannot respond.”

Therefore, demand sensing is not a forecasting approach. It is a method of falsely improving forecast accuracy by changing forecasts in a way that can never translate into improving supply chain performance.
Why Demand Sensing is Not a Forecasting Approach

Demand sensing is not a supply chain forecasting approach because it changes the forecast within lead times, and therefore, when supply planning cannot respond. 

This is for a variety of reasons:

• Skills
• Software selection (many have selected the wrong software)
• Their management does not understand forecasting
• Many companies do not hire people with forecasting knowledge or education in forecasting, often choosing to hire people out of finance instead.
• Marketing has increased the product database with challenges to forecast product location combinations.
• etc..

The list of essential things that most companies cannot do in their supply chain forecasting systems is often fantastic.

A typical list would be:

1. Can’t-do attribute-based forecasting.
2. They don’t know or track their forecasting bias.
3. They don’t know what part of their forecasting process contributes or detracts from forecast accuracy.
4. They cannot manage their forecast parameters correctly.
5. Measure forecast error in the wrong location.
6. They have weak lifecycle functionality in their present application.
7. Can’t perform software selection because their executives do not understand forecasting themselves and because they rely on consulting companies that choose their software-based on billing hour maximization.

The list would go on, but supply chain forecasting is terrible at even the biggest companies in the US.

How Common is FEF – Forecasting Error Falsification?

What I call FEF is a common issue in forecasting departments as they attempt to look good to the rest of the organization. The fact that vendors can’t agree as to what it is is a problem. The way I have seen demand planning departments use DS is forecast error falsification.

FEF is very popular. When companies report forecast errors in the supply chain at aggregated levels in group meetings, that is a form of FEF. Many demand planning departments want to spend more time engaging in FEF than improving the forecast. You may not believe this, but I witnessed a presentation of overall monthly UNITS forecasted versus UNITS sold as a measure of forecast error. Everyone around the table said, “Gee, that looks pretty good.” And it is good — if you sell a  product location combination– but somewhat illogical if you don’t. And demand sensing provided them with the opportunity.

I have companies that can’t master statistical forecasting, can’t maintain promotion history adjustments in sales history, perform model reassignment, and refuse to measure marketing and sales inputs to the final forecast. Still, they want to engage in demand sensing. This is a real problem for making progress because we keep hopping around to whatever is hot.

The most common type of FEF measures forecast error at too high a level of aggregation and at a level that means nothing for supply planning. For forecast error to say something to the supply plan, it must be at the product location combination, which can drive safety stock.

Supply planning also fakes its numbers, which is typically some service level. Pretending “99%” supply chain service level is natural and quite common, and studies show that companies achieve around seven percentage points lower than the state that they do.

For instance, many companies maintain their supply chain service level only by deleting orders they cannot meet. Demand sensing is brazen because it pretends it is an actual forecasting approach. Companies across the country don’t know how to forecast yet have accuracy targets they must meet. This is where demand sensing can come to the rescue.

Improve the Forecast or Fake the Forecast Accuracy with Demand Sensing?

Demand sensing is a very convenient tool, but it could be used for changing the forecast at the last minute, far after it has any relevance. This is similar to changing your “forecast” for the winner of a football game after three-fourths of the game is over. It no longer meets the definition of a prediction.

This brings up the critical topic of when forecasting changes can be made. As explained in the graphic above, changes are no longer part of forecasting/demand planning after a certain point, and they do not improve the forecast for apparent reasons. 

The demand planning department will use this term to, in effect, fake out other departments that rely on the forecast into telling them that they are using a legitimate technique to improve “forecast accuracy.” In the short term, this may work in departments that don’t understand forecasting. In the long-term, it won’t work.

Demand sensing is not a part of forecasting because it occurs after the forecasting period has passed.

Searching for Logical Consistency

The more I read on-demand sensing, the less it makes any sense. Let us take a few examples from the Internet because there are significant issues with how demand sensing is explained by some of the top websites in Google.

But first, let us review a definition by Stefan De Kok.

Demand forecasting = given all known information, what do we expect will happen. Demand shaping = what we do to change how demand will occur Demand sensing = capturing demand data further downstream to reduce information latency.

Demand sensing is NOT forecasting. Also agree you should not measure accuracy of sensed demand the same way you measure it for forecasted demand. IF you do, indeed 30% error reduction is a common low achievement. 50% is usually possible. The business value is there and that is what will drive its continued use and expansion.

What is amusing is analyzing Gartner, where they talk about demand sensing.

Demand sensing is a forecasting method that leverages new mathematical techniques and near real-time information to create an accurate forecast of demand, based on the current realities of the supply chain. Gartner, Inc. insight on demand sensing can be found in its report, “Supply Chain Strategy for Manufacturing Leaders: The Handbook for Becoming Demand Driven.” ^[1]

So, this directly contradicts Stefan De Kok’s statement about whether demand sensing is part of forecasting. However, Gartner typically knows very little about forecasting, but what they say is curious because other people pay attention to what Gartner says.

Let us review another entity’s page on DS.

Traditionally, forecasting models were based on time series techniques that create a forecast based on prior sales history. However, where past sales are sufficient for predicting future forecast levels in mid- and long-term planning horizons, they have shown to be less accurate for short-term planning. The promise of Demand Sensing is that we can do better. In plain English, the premise is this: if the past is the best predictor of the future, it follows that the very recent past is the best predictor of the very near future. – Halo

I have been working in forecasting for several decades, and I don’t know what short-term planning is. Planning is, by its nature, not short-term. That tends to be called executing. Let use review more..

Near-future can mean hours or days, depending upon how dynamic your supply chain is. Demand Sensing grew out of the broader trend toward collaborative supply chains, where insular thinking (“What can we make, transport and deliver?”) has given way to a customer-centric view of the market: what doesthe customerwant and when? Customers need their supply chains to simultaneously deliverhigh service levels and optimized stock levels. – Halo

This quote seems inaccurate and does not match my experience consulting companies in forecasting in any way.

More Collaborative Supply Chains?

First, supply chains are not becoming more collaborative, and companies can’t even integrate their forecasts within their own four walls. I cover this topic in the book Sales and Statistical Forecasting Combined, the problems, and opportunities to improve this issue, but it is a severe problem.

Secondly, as firms in the US at least have become more focused on short term financial results, this has caused a reduction in collaboration between firms. It is not demonstrated that DS grew out of collaboration. The first proponent of DS was Terra Technologies (now owned by E2Open), and Terra Technologies made several claims I consider to be false. That is my view that DS grew out of the need for Terra Technologies to differentiate itself from other forecasting vendors.

Secondly, let us review the beginning of the quote.

Near-future can mean hours or days, depending upon how dynamic your supply chain is.

If the discussion hours or days, unless one is selling something with an hour or daily lead time, there is no reason for the topic of forecasting to concern itself with these short-term gyrations.

Halo continues.

The goal of Demand Sensing is to help planners make short-term decisions based on what just happened hours or days ago, not what happened last year.

No, that is incorrect.

Planners don’t focus on short term decisions. Short term issues are the job of inventory fulfillment specialists. This is problematic because it promotes demand for planning departments to use their personnel ineffectively.

Halo continues..

The goal of Demand Sensing is to help planners make short-term decisions based on what just happened hours or days ago, not what happened last year.

This sentence is ridiculous.

Again, planners don’t make short-term decisions.

Planning or Fulfillment

This is mixing up planners with fulfillment. Furthermore, the statement that DS moves companies away from what happened years ago is incorrect. A statistical forecast typically has a monthly planning bucket and uses monthly data. It will use the previous months. It may also use last year, but that is only because last year is, in any case, more relevant than last month or the past few months. Seasonal patterns are a perfect example of this.

This is applying an increase for May (the green line) based on August and the previous May. Using historical data is a proven way to produce a forecast. There is no basis for criticizing it simply because the data is not the most current. Forecasting is not about merely using “the most current data.” 

Halo continues..

Planners learned long ago that it’s difficult and time-consuming to force real-time data into existing time-series statistical models. It’s not surprising then that Demand Sensing is fundamentally different in that it often incorporates a broader range of demand signals (including current data from the supply chain) and different mathematical techniques to create a more accurate forecast that responds to real-world events.

Did they? When did this happen? First, when did planners realize that they needed to use real-time data? Secondly, why are different mathematical techniques necessary — and what are these techniques? DS was to incorporate more recent data, but now we are also changing the mathematical methods used.

Halo continues..

Bottom line: a plan that relies on a weekly or monthly forecast to create a daily forecast from a manually entered day-weight calendar will consistently be less accurate than a daily forecast built using Machine Learning principles derived from daily POS data.

Now, of course…..ML is brought into the conversation. Where Halo is getting the evidence for these statements is quite a mystery. Halo has proposed that all weekly and monthly forecasts are obsolete, and all forecasting should be a combination of ML with POS and DS.

Halo continues..

Still, it’s natural to wonder what would happen if we just threw more horsepower at the traditional approach. The answer is a bit wonkish.

It turns out that time-series forecast accuracy eventually hits a ceiling. A ceiling governed not by computer processing power, but rather fundamental limitations imposed by information theory and the fact that historical data is a flawed predictor of supply chain conditions at any specific moment.

There is a ceiling regarding statistical time series forecasting, but it has nothing to do with information theory. It has to do with the ability of companies to focus on mastering time series forecasting. Now, is historical data a flawed predictor? Amazing. These are just amazingly unsupported contentions on the part of Halo.

Halo explains how information theory proves time series forecasting is less effective than DS.

Information theory dictates that increasing a model’s sophistication to pursue a ‘perfect fit’ reaches a point where the outcome is decreased forecast accuracy. Industry figures show that despite highly tuned models, forecast error remains a challenge. Even high-volume consumer packaged products with well-understood seasonality established over decades continue to experience high near-term forecast error rates.

It’s challenging to determine precisely what Halo refers to here, but my guess is overfitting. Yes, overfit models will reduce forecast accuracy, so you don’t try to get a perfect fit, which is why best-fit forecasting procedures are generally overrated. However, overfitting has nothing to do with whether time series forecasting is “kaput.” And the comment that high-volume consumer packaged goods with well-understood seasonality have high forecast error is false. I know, as I have created many forecasts for these products in the food and beverage industry. However, the term “near-term” forecasting is their trap door — but near-term, forecasting is not forecasting.

Halo’s article on DS is filled with false information about time series/statistical forecasting. The best-case scenario is that Halo has no idea about time series/statistical forecasting. The worst-case scenario is they are deliberately misleading readers.

So let us switch to E2Open. E2Open is, of course, the company that purchased Terra Technologies.

Myth #5: “A daily forecast will send shock waves through our supply systems.”

We hear this one a lot. To be clear, just because you now have daily forecasts, it does not mean you need to re-plan every day! Most companies keep the same supply planning cadence, but now each time they go to make a supply decision, they start with the best possible information. Think of demand sensing as providing a new level of granularity and accuracy to your forecasts, and this extra fidelity allowing you to make better quality decisions.

It should not be presumed that the most recent information is the best possible information. Any statement about forecasting must be demonstrated as accurate. This is the problem with E2Open’s statement. It merely asserts that the most recent information is “the best possible.”

As for granularity, it is not in itself an unquestioned virtue. Notice this research using real client data that demonstrated a less granular forecast planning bucket beat a more granular planning bucket in the article Test Results Quarterly Versus Weekly Buckets — also — this research showed quarterly forecast planning buckets beating weekly planning buckets. Test Results Quarterly Versus Weekly Buckets. The assertion was that weekly forecast buckets would be the most accurate, because as I recollect..

“You can use more recent information and pick up on patterns.”

When I proved this hypothesis incorrect, the promoters of the hypothesis did not say

“I was wrong”

..instead, they just became very quiet.

E2Open goes on to say..

For companies that are on a journey towards continuous planning – where planning and execution begin to converge – daily forecasts are an important enabler.

This is absolutely what you want to happen. Planning and execution are two different skills and don’t have much to do with each other.

E2Open continues..

Myth #6: “30% of my business is driven by promotions so I can’t use demand sensing.”

It’s not uncommon for promotions to drive 30% of a manufacturer’s business. Companies spend a lot of time, energy and money running events so it is important to properly support them. The problem with forecasting promotions is that it’s hard to predict their performance with traditional demand planning. Why? Traditional planning systems rely on history to predict future performance, whereas promotions are, by nature, designed to ensure that history does not repeat itself. What can help are algorithms that automatically sense how promotions are performing in real-time. This way event items that outperform expectations can be quickly replenished to ensure on-shelf availability and maximize the return on promotional investments. On the flipside, it also prevents you from getting stuck with too much inventory from underperforming promotions – this is especially important for event-specific items that are hard to repurpose or are perishable.

This paragraph is correct; it is nonsense. I cover in the book Promotions Forecasting: Techniques of Forecast Adjustments in Software. Traditional forecasting software does not usually use history to account for the effect of promotions. The promotional impact typically comes from marketing (if it happens), and there is an uplift in the forecast as per the marketing request. The statement about “sense(ing) how promotions are performing in real-time, is not helpful because companies cannot respond to real-time signals. Again, there are lead times involved. The promotion must be forecasted. This paragraph is promising things that defy how supply chains function.

Stefan De Kok makes a related but more reasonable claim.

As an example, say you are a manufacturer supplying many retailers. For some unpredicted reason one retailer is suddenly seeing a big increase in sales for some item. If that retailer shares that info with you, not by a single phone call, email, EDI or order, but as part of a steady stream of sales data, it allows you to sense demand. In this case, you could determine that the retailer will place an order to replenish a week earlier than usual and for twice the normal quantity. You start moving that item to the local DC, and when the order does come in, as you knew it would, you can satisfy it. Happy customer, happy boss.

This seems reasonable. It is, again, not forecasting, and there is nothing objectionable in this description.

Problematic Quotes

The quotes I have taken from multiple sources on DS are questionable because they provide false information about numerous aspects of time series/statistical forecasting. This seems to be an attempt to get companies to see issues with time series/statistical forecasting as not due to their maintenance and mastery of these systems. Instead, it is to the entire approach of time series forecasting as being flawed. This is an odd position to take, given the enormous amount of academic research supporting statistical forecasting.

Firstly, many of the things that are often touted as part of demand sensing should be performed by the regular demand planning system. Secondly, making short-term changes to the forecast introduces significant noise. Forecasts need to be set and left unchanged. Sales orders may be higher or lower than the forecast, but the forecast is a value to be frozen.

Why Forecasts are Created in the First Place

The entire purpose of setting up an estimate is that there is a lead time; if there were no lead time, there would be no need to predict. Therefore the concept of demand sensing is on fragile ground. At this point, it seems to be simply a popular term used to sell software rather than any substantial addition to demand planning. The article definition I have on-demand sensing explains the fact that no academic literature supports demand sensing. When I searched for an academic search engine, I could only find this article, An Efficient Interactive Model for On-Demand Sensing-As-A-Services of Sensor-Cloud. However, upon review, it had nothing to do with demand sensing.

Using Either Demand Sensing or Demand Shaping

Demand shaping is a valuable function. However, remarkably few companies perform demand shaping. The vast majority of enterprises achieve the opposite or demand distortion. This is because the supply chain department does not control the conditions and terms or pricing their product offers customers. This is determined by sales or marketing, which generally could not care less about how difficult it is for operations to fulfill the demand or what the cost of sales ends up being. Therefore, hearing so much about demand shaping is strange when demand distortion rules the show.

So, while it’s certainly a helpful concept, there is very little chance of this occurring, so it is necessarily a waste of time to continue to discuss. The topic should be passed to sales and marketing, who will promptly put the idea in the trash bin.

Conclusion

Neither of these will amount too much in the longer term.

• One is a nice idea, but companies are positively opposed to its implementation. They actively distort demand, thinking this maximizes profits and helps them meet quarterly numbers in a pinch.
• The other has little foundation in any logic and seems to distract companies from improving their forecasting capability by adding a post-forecasting processor that jiggers the forecast around.

DS leaves out the overhead in constantly adjusting a forecast that only makes a difference under a situation of expediting.

DS sources have a problem making numerous false statements about time series/statistical forecasting. If DS proponents are going to make their case, they have to do so without providing inaccurate information about time series/statistical forecasting.

Secondly, DS has already been used for FEF and is a prime candidate for use for future FEF. This means forecasts, under the rubric of DS, will be included in the forecast history — that was entered inside lead time. This is the equivalent to me changing the record of my selection of the winning team in a game in the after the halftime or the third quarter. And then, without acknowledging when I made this prediction. It creates a scenario where a timestamp must be placed on forecasts. That row in the forecast history distinguishes between the time series-based forecast and the DS forecast.

It should also be noted that many of these companies use demand sensing to justify adjusting the forecast before its occurrence. This allows them to change their forecast value and measure from the changed forecast value….thereby improving their forecast accuracy. Just because a company is large, like P&G or Nike, does not mean it understands or performs good forecasting practices. This article describes how executives at Nike don’t seem to understand elementary forecasting terminology, as is covered in the article How to Understand Incorrect Forecasting Terminology by Nike.

Forecasting should not be thought of as simply within the forecasting “box.” If a short-term forecast is changed, and the supply plan cannot respond, nothing of value was performed (except to game the forecasting accuracy measurements) that the company is keeping.