The Rise and Decline of Supply Chain Cost Optimization

Executive Summary

• Optimization became a prominent component in supply chain planning software. However, optimization struggles with unintelligibility, and there are major challenges in getting cost optimization to work in SAP.
• We cover implicit versus explicit costs, how cost optimization compares to inventory optimization, and the current state of optimization.

Introduction

Optimization has proven much more tricky to implement than initially thought. Optimization, where the company must agree internally to set costs, is far harder to implement than where the optimization is mostly a black box. When one hears the term “optimization” in supply chain planning, in the vast majority of circumstances, this refers to cost optimization, where the objective function is to minimize costs. This is slowly changing as more various optimizers have been brought out, but we will be discussing the history of cost optimizers.

Many supply chain gurus have recommended supply chain optimization; however, what has been the output?

How Optimization Drove the Supply Chain Planning Software Industry

In its early years, APO was sold to perform optimization in the supply chain planning market. This is primarily because it was an industry-wide practice to sell advanced planning software in this way. APO or Advanced Planner and Optimizer – had the term directly in its name.

The Dual Meanings of Optimization

The term optimization has two meanings as used.

• One is more of a business usage, which means to produce the best outcomes.
• In operations research, from where supply chain optimization originates, it has a more specific meaning.

Its more technical definition is unknown for those who did not do work in operations research or advanced mathematics.

For this reason, we wanted to display Wikipedia’s definition.

Technical Definition of Optimization

“In mathematics, linear programming (LP) is a technique for optimization of a linear objective function… Linear programming is a considerable field of optimization for several reasons. Many practical problems in operations research can be expressed as linear programming problems. Certain special cases of linear programming, such as network flow problems and multi commodity problems. Although the modern management issues are ever-changing, most companies would like to maximize profits or minimize costs with limited resources. Therefore, many issues can boil down to linear programming problems.“ – Wikipedia

Linear vs. Discrete Optimization

Optimization works best in situations that are entirely “linear, “that is, inputs can be increased or decreased continuously. An example of a direct input would be an order quantity. Perfect linear optimization would mean that any order number from zero to infinity could be placed and fulfilled.

In reality, supply chains are not entirely linear problems. For instance, the lot size is a discrete value that limits the flexibility of the order quantity. One item may be ordered in units of 50. If 135 units are desired, and the current inventory is less than 35, 150 must be ordered to meet this demand. APO has some techniques, such as lot size, then alter the problem being solved from entirely linear to discrete or known as a step function.

This is crucial to making the resulting recommendation realistic.

Optimization by Industry

Different industries have different propensities to implement specific solutions over others. This is highlighted by the quote below:

“Clients in companies which have a strong history in mathematical optimization…..

1. Refineries
2. Metal Industry
3. Chemical Industry

…do not easily trust black box optimization models. They expect a high approximation quality to their “real world.” SAP as a company does not primarily position SAP APO as a tool for mathematical optimization and does not provide extensive in-depth training regarding modeling and optimization. Conceptually optimization in SAP APO has been designed in such a way that the users build the model via the master data and can choose from pre-set solution techniques, but do not see the mathematical model equations. For many users this is what they probably prefer. Some real world planning problems, however, require a more in-depth treatment of the mathematics and or modeling tricks with the available objects in SAP APO. SAP’s in-house consultant’s will help explain the underlying mathematical assumptions, but consultants with deep optimization and business software knowledge are often scarce.” – Real Optimization with SAP APO

The authors then point out that some sectors are not as willing to go with enterprise optimization “black boxes.”

“In most cases it is not easy to replace an existing optimization application based on a tailored mathematical model by an APS based on predefined models. Of course this depends on the complexity of the planning problem, the planning philosophies and the solution techniques.” – Real Optimization with SAP APO

Cost-Based Supply Chain Optimization Struggles with Unintelligibility

A common complaint with cost optimization is that a small change can make the results difficult to explain. Enterprise vendors have taken optimizer engines such as ILOG’s (now IBM’s) CPLEX and encapsulated them in C and C++ to hide or protect their intellectual property, which is the mathematics of the optimizer. This is the opposite approach when engines like ILOG or MatLab are purchased and used in their native state, where all the math and arrays are readily observable. This is covered in this post. This also imbues the optimizers with a certain “mystique,” which the vendor is quick to play up.

Most of these vendors have mined published works, with the slight alteration in the mathematics through client interaction, which tends to be lost.

Unintelligibility Issues

To not be inconsistent with other posts, I should say that clients like the mystique of the mathematics of cost optimization for supply chain planning applications. However, they dislike the unintelligibility of the results.

And in many cases, cost optimization results can be challenging to explain, mainly before the cost optimizer is “dialed in.” This is described in the quotation below:

“One reason why the optimizer has not been chosen in some cases of SAP APO implementations at semiconductor manufacturers may be because the results of a purely rules based algorithm like CTM are easier to explain to planners and business people who are used to priority and strictly rules based planning. This match with the way that users are used to work and think is a very strong asset difficult to overcome with an algorithm that is seen as a black box and hence is not trusted.

Only sufficient knowledge of mathematical optimization can create trust in the results, which requires thorough understanding of optimization based planning and education of the planners. This, again underlines the need for optimization experts closely involved in any optimization implementation” – Real Optimization with SAP APO

The second portion of this quote gets into the topic of socialization of the solution, which has been a serious shortcoming on many cost optimization for supply chain planning projects, covered in detail in this post.

The Decline of Cost-Based Supply Chain Optimization

However, while supply chain optimization drove development in SCM at one time, it no longer does. The evidence for this is that optimization is an option in three old modules (SNP, PP/DS, and TP/VS). But it is not an option in any of the newer modules (EWM, SNC, EM, SPP, and F&R).

Furthermore, the core optimization functionality in SCM has been stabilized for some time. A related story to this is that optimization did not meet its initially envisioned potential. This is indeed not due to the stagnation of technology like the following quote and can attest:

“There is a continuous trend in mathematics that new development become standard technologies after some time. In the 1950’s special solution techniques were developed to solve different equations numerically. Nowadays these are standard. Until 1980, solving MILP (Mixed Integer Linear Programming) problems was a challenge. Today many difficult MILP problems are solved in short time with standard commercial solvers such as CPLEX or Xpress-MP.” – Real Optimization with SAP APO

Of course, the more powerful hardware is brought out, which can run optimization faster than the year before.

The last data point is in 1986 because to include further data points makes the earlier data points impossible to differentiate on a graphic this size.

What is Running the Cost Optimization Solution?

There is a story at the client of an advanced planning vendor that the optimizer engine running in the background was not operational for some days, and no one noticed. This was because the planning was primarily being performed by heuristics that had been custom coded with scripts, and the solution was not using the optimizer at all. Whether the client knew the optimizer was not being used is unknown. This is more common than a reading of press releases and industry periodicals in the area would think.

While optimization is what sold a lot of supply chain software, it was not necessarily what the customers of these solutions used to go live. As is evident from our example above, this experience is more extensive than merely SAP APO. In working on many cost optimization accounts over the years and in conferring with other individuals who have worked in the area for some time, I have concluded how costs are set in both supply planning and production planning implementations.

The graphic above describes major elements of cost optimization that apply to supply planning.

Implicit Versus Explicit Costs

Cost optimizers use both explicit and implicit costs to generate the plan. During the software selection stage, this tends to be covered from a surface level. But it is probably the most critical element to investigate thoroughly before engaging in cost optimization implementations. Before we go into the cost development methods, it is essential to review the distinction between explicit costs versus implicit costs.

Explicit Costs

Explicit costs are the company’s costs; these would include transportation costs and storage costs. They are paid out by the company but must be aggregated because it’s not feasible to enter every single shipment’s costs into the model. Estimating these costs is tricky, and surprisingly, there is no group inside a company that can help with this cost estimation.

Implicit Costs

These are costs that the company does not pay out but are inherent in the business’s activities. Penalty costs are an example of an implicit cost. When a company fails to meet demand, there is no accounting entry for this. The customer goes away, and no sale is made on that item. That customer may purchase another item from the company’s offering or move on to a different company’s item. In the latter case, the first company has lost the profit and goodwill of that potential sales had they been in stock.

For a cost optimizer to appreciate that not stocking out is vital to the company, and for an appreciation of how important it is to meet this demand, a cost optimizer requires a cost associated with an unmet need, which is then traded off against the costs of ordering, keeping items in stock, transporting items, etc. Implicit costs like this are more difficult to estimate than explicit costs because they can’t be found in any invoice or real cost the company pays but comes down to the company’s goals and objectives.

Actual Versus Relative Actual Costs

There are several ways to set cost, both explicit and implicit costs, at a cost optimizer. The primary ways I have seen are listed below:

1. Actual Cost Development: Use actual costs for the explicit costs, and then perform full cost-based accounting to develop the implicit costs.
2. Relative Actual Cost Development: Start with close to real costs for the explicit costs (transportation and storage, etc..) and then derive the implicit costs, such as penalty costs, about these explicit costs.
3. “Crazy Costs” or Interactive Cost Development: Here, costs are determined based on repetitive testing and the desired model output. The costs do not relate to real costs but are learned through trial and error, which provide what the planners want to see.

What Academics Would Say on Cost Development

The underlying assumption of cost development that underpins cost optimization is that the costs reflect the actual costs the company incurs. If one were to think about the objective function of cost optimization, that is, to minimize costs; the concept is that the optimizer is, in effect, reducing the real costs that the company is incurring through its various operational behaviors. This was the original intent of the software and is the assumption under which the optimizer is sold to the company and explain to executives.

However, the way that the costs are placed into the cost optimizer is developed is not what the cost optimizer is accomplishing.

The Reality of Cost Optimization

What would come as a surprise to academics, and what has surprised me, is that the only way I have ever seen costs developed on cost optimization projects is the third method, the Crazy Cost, or Interactive Cost Development Method.  Interestingly,  in different conversations with the various implementers of cost optimization systems, the responses I get back are.

“Actual costs were not known,”

and…

“We tried actual costs, and they did not work, so we went to this.”

This is not one observation, but many observations, not just my observation, but have also been verified with some other consultants who have also seen multiple cost optimization projects. WhaThis tells meat the Crazy Cost Method is the dominant method of developing costs on cost optimization projects. Usingtual costs or even actual relative costs is the exception, not the rule.

The Implications

This means that the cost optimization that is occurring is not based on the costs that various things cost the company. The costs are rigged to give the desired output. So if a very high penalty cost is required to get enough stock held by the system, it is increased until the desired effect is obtained.

Secondly, if actual costs cannot be estimated and, if estimated, cannot be used. It calls into question whether the objectives of companies that implement cost optimization projects consider cost minimization as the overriding objective of their supply chain organizations. Cost minimization is the objective function of the cost optimizer, and, therefore, it should be the company’s objective to implement this technology. If this is not the actual goal, then cost optimization or cost minimization is not the correct supply planning method for that company to use. This addresses the core of how various techniques are selected for implementation.

Many people think that cost optimization is advanced, and it is. However, it is also definite and does only one thing, and it minimizes costs. The match between the biases of the technology and the business’s goals is the most crucial factor in method selection, not whether one method is more advanced than another.

Implications When Comparing Cost Optimization to Inventory Optimization and Multi-Echelon Planning (MIEO)

The implications for inventory optimization bear discussion. Inventory optimization and multi-echelon planning use goals that are more easily quantifiable by supply planning organizations. These goals are service levels and inventory quantities. This means that inventory optimization should not face the same issues that have faced cost optimization on variable estimation. Companies have specific service levels and inventory goals that they can place into the system. However, while MEIO can undoubtedly be thought to have an advantage over cost optimization in determining their input controls, once service levels are broken down into different product groups, it turns out companies run into complications developing these service levels.

Conclusion

The way costs are developed during implementation is not how cost optimizers were initially developed to be used. This is not to say that this method is wrong. However, when costs are developed in this way, when any input is developed differently from how the software has assumed it would be developed, it should be understood and recognized regarding the plan and cost optimization implications. Currently, I don’t think these effects are widely understood.

Why Not Use Cost-Based Optimization?

If optimization did not “change the world “of the supply chain, the question naturally becomes, “Why not? “There are probably some reasons; however, in our view, one of the most prominent came down to implementation and maintenance difficulty combined with company knowledge. Implementing and maintaining optimization methods requires lots of effort and long-term investment. Secondly, optimization requires an enormous amount of discipline and expertise on the part of the implementing company.

Many companies want the benefit of advanced planning but are not culturally, financially, or prepared from a skills perspective to make the sacrifices required to obtain their desired outcomes. More often than not, companies want simple solutions that deliver value over a short time horizon. That is not what optimization provides. On the other hand, software deserves some of the blame as much advanced planning software has been more complicated than necessary.

Optimization is always the most complex and challenging of implementation but also brings significant benefits if implemented correctly. However, for most, that mountain is too high to climb. I sometimes get an email from technical people who like optimization and wonder why I am “negative.” This has nothing to do with whether I like optimization or not. I am reflecting on the experience of companies, most of which are unsuited to use it. Much money has been spent on optimization that could have been used to solve the same problem more directly and more sustainably.

In addition to lower investment, the returns from optimization have not been encouraging. Thus, it’s incumbent upon me as a consultant to bring these failures to light for clients and gauge their ability to implement optimization. This is very infrequently done.

References

“Real Optimization with SAP APO,” Josef Kallrath, Thomas I. Maindl, Springer Press, 2006