- Optimization became a prominent component in supply chain planning software. However, optimization struggles with unintelligibility and there are major challenges in getting cost optimization to work in SAP.
- We cover implicit versus explicit costs, how cost optimization compares to inventory optimization as well as the current state of optimization.
Optimization has proven much more tricky to implement than initially thought. Optimization, where the company must agree internally to set costs, is far harder to implement than where the optimization is mostly a black box.
When one hears the term “optimization” in supply chain planning, in the vast majority of circumstances this refers to cost optimization, where the objective function is to minimize costs. This is slowly changing as more various optimizers have been brought out, but we will be discussing the history of cost optimizers.
Many a supply chain guru have recommended supply chain optimization, however, what has been the output?
How Optimization Drove the Supply Chain Planning Software Industry
In its early years, APO was sold on its ability to perform optimization in the supply chain planning market. This is primarily because it was an industry-wide practice to sell advanced planning software in this way. In fact, APO or Advanced Planner and Optimizer – had the term directly in its name.
The Dual Meanings of Optimization
The term optimization has two meanings as used.
- One is more of a business usage, which means to produce the best outcomes.
- In the area of operations research, from where supply chain optimization originates, it has a more specific meaning.
For those that did not do work in operations research or advanced mathematics, its more technical definition is unknown.
For this reason, we wanted to display Wikipedia’s definition.
Technical Definition of Optimization
“In mathematics, linear programming (LP) is a technique for optimization of a linear objective function… Linear programming is a considerable field of optimization for several reasons. Many practical problems in operations research can be expressed as linear programming problems. Certain special cases of linear programming, such as network flow problems and multi commodity problems. Although the modern management issues are ever-changing, most companies would like to maximize profits or minimize costs with limited resources. Therefore, many issues can boil down to linear programming problems.“ – Wikipedia
Linear vs. Discrete Optimization
Optimization works best in situations that are perfectly “linear,“ that is inputs can be increased or decreased continuously. An example of a direct input would be an order quantity. Perfect linear optimization would mean that any order number from zero to infinity could be placed and fulfilled.
In reality, supply chains are not entirely linear problems. For instance, the lot size is a discrete value which limits the flexibility of the order quantity. One item may be ordered in units of 50. If 135 units are desired, and the current inventory is less than 35, then 150 must be ordered to meet this demand. APO has some techniques, such as lot size, then alter the problem being solved from entirely linear, to discrete, or what is known as a step function.
This is crucial to making the resulting recommendation realistic.
Optimization by Industry
Different industries have different propensities to implement certain solutions over others. This is highlighted by the quote below:
“Clients in companies which have a strong history in mathematical optimization…..
- Metal Industry
- Chemical Industry
…do not easily trust black box optimization models. They expect a high approximation quality to their “real world.” SAP as a company does not primarily position SAP APO as a tool for mathematical optimization and does not provide extensive in-depth training regarding modeling and optimization. Conceptually optimization in SAP APO has been designed in such a way that the users build the model via the master data and can choose from pre-set solution techniques, but do not see the mathematical model equations. For many users this is what they probably prefer. Some real world planning problems, however, require a more in-depth treatment of the mathematics and or modeling tricks with the available objects in SAP APO. SAP’s in-house consultant’s will help explain the underlying mathematical assumptions, but consultants with deep optimization and business software knowledge are often scarce.” – Real Optimization with SAP APO
The authors then bring up the point that some industries are not as willing to go with enterprise optimization “black boxes.”
“In most cases it is not easy to replace an existing optimization application based on a tailored mathematical model by an APS based on predefined models. Of course this depends on the complexity of the planning problem, the planning philosophies and the solution techniques.” – Real Optimization with SAP APO
Cost Based Supply Chain Optimization Struggles with Unintelligibility
A common complaint with cost optimization is that a small change can make the results difficult to explain. Enterprise vendors have taken optimizer engines such as ILOG’s (now IBM’s) CPLEX and encapsulated them in C and C++ in order or to hide or protect their intellectual property which is the mathematics of the optimizer. This is the opposite approach when engines like ILOG or MatLab are purchased and used in their native state where all the math and arrays are readily observable. This is covered in this post. This also imbues the optimizers with a certain “mystique” which vendor is quick to play up.
The fact that most of these vendors have mined published works, with the slight alteration in the mathematics through client interaction tends to be lost.
So as not to be inconsistent with other posts, I should say that clients like the mystique of the mathematics of cost optimization for supply chain planning applications. However, they dislike the unintelligibility of the results.
And in many cases, cost optimization results can be difficult to explain, mainly before the cost optimizer is “dialed in.” This is described in the quotation below:
“One reason why the optimizer has not been chosen in some cases of SAP APO implementations at semiconductor manufacturers may be because the results of a purely rules based algorithm like CTM are easier to explain to planners and business people who are used to priority and strictly rules based planning. This match with the way that users are used to work and think is a very strong asset difficult to overcome with an algorithm that is seen as a black box and hence is not trusted.
Only sufficient knowledge of mathematical optimization can create trust in the results, which requires thorough understanding of optimization based planning and education of the planners. This, again underlines the need for optimization experts closely involved in any optimization implementation” – Real Optimization with SAP APO
The second portion of this quote gets into the topic of socialization of the solution, which has been a serious shortcoming on many cost optimization for supply chain planning projects, which covered in detail in this post.
The Decline of Cost Based Supply Chain Optimization
However, while supply chain optimization drove development in SCM at one time, it no longer does. The evidence for this is that optimization is an option in three of the old modules (SNP, PP/DS, and TP/VS). But is not an option in any of the newer modules (EWM, SNC, EM, SPP, and F&R).
Furthermore, the core optimization functionality in SCM has been stabilized for some time. A related story to this is that optimization did not meet its initially envisioned potential. This is indeed not due to the stagnation of technology like the following quote and can attest:
“There is a continuous trend in mathematics that new development become standard technologies after some time. In the 1950’s special solution techniques were developed to solve different equations numerically. Nowadays these are standard. Until 1980, solving MILP (Mixed Integer Linear Programming) problems was a challenge. Today many difficult MILP problems are solved in short time with standard commercial solvers such as CPLEX or Xpress-MP.” – Real Optimization with SAP APO
Of course, the more powerful hardware is brought out which can run optimization faster than the year before.
The last data point is in 1986 because to include further data points makes the earlier data points impossible to differentiate on a graphic this size.
What is Running the Cost Optimization Solution?
There is a story at the client of an advanced planning vendor that the optimizer engine that was running in the background was not operational for some days, and no one noticed. This was because the planning was primarily being performed by heuristics that had been custom coded with scripts and the solution was not using the optimizer at all. Whether the client knew the optimizer was not being used is unknown. This is more common than a reading of press releases and industry periodicals in the area would think.
While optimization is what sold a lot of supply chain software, but it was not necessarily what the customers of these solutions used to go live. As is evident from our example above, this experience is more extensive than merely SAP APO. In working on many cost optimization accounts over the years and in conferring with other individuals who have worked in the area for some time, I have come to a conclusion regarding how costs are set in both supply planning and production planning implementations.
The graphic above describes major elements of cost optimization that applies to supply planning.
Implicit Versus Explicit Costs
Cost optimizers use both explicit and implicit costs to generate the plan. During the software selection stage, this tends to be covered from a surface level, but it is probably the most critical element to investigate thoroughly before engaging in cost optimization implementations. Before we go into the cost development methods, it is essential to review the distinction between explicit costs versus implicit costs.
Explicit costs are the costs that the company incurs; these would include things like transportation costs and storage costs. They are paid out by the company but must be aggregated because it’s not feasible to enter the costs for every single shipment into the model. Estimating these costs is tricky, and surprisingly, there is not a group inside a company that can help with this cost estimation.
These are costs that the company does not pay out but are inherent in the activities of the business. Penalty costs are an example of an implicit cost. When a company fails to meet demand, there is no accounting entry for this. The customer goes away, and no sale is made on that item. That customer may purchase another item from the company’s offering or may move on to a different company’s item, and in the latter case, the first company has lost the profit and goodwill of that potential sales had they been in stock.
For a cost optimizer to appreciate that not stocking out is important to the company, and for an appreciation of how important it is to meet this demand, a cost optimizer requires a cost associated with an unmet demand, which is then traded off against the costs of ordering, keeping items in stock, transporting items, etc.. Implicit costs like this are more difficult to estimate than explicit costs because they can’t be found in any invoice or real cost the company pays but comes down to the goals and objectives of the company.
Actual Versus Relative Actual Costs
There are several ways to set cost, both explicit and implicit costs, in a cost optimizer. The primary ways I have seen are listed below:
- Actual Cost Development: Use actual costs for the explicit costs, and then perform full cost-based accounting to develop the implicit costs.
- Relative Actual Cost Development: Start off with close to real costs for the explicit costs (transportation and storage, etc..) and then derive the implicit costs, such as penalty costs about these explicit costs.
- “Crazy Costs” or Interactive Cost Development: Here, costs are simply determined based upon repetitive testing and the desired model output. The costs do not relate to any real costs but are simply determined through trial and error which provide what the planners want to see.
What Academics Would Say on Cost Development
The underlying assumption of cost development that underpins cost optimization is that the costs are some reflection of the actual costs the company incurs. If one were to think about the objective function of cost optimization, that is to minimize costs; the concept is that the optimizer is in effect reducing the real costs that the company is incurring through its various operational behaviors. This was the original intent of the software and is the assumption under which the optimizer is sold to the company and to explained to executives.
However, the way that the costs that are placed into the cost optimizer are developed, this is not at all what the cost optimizer is in fact accomplishing.
The Reality of Cost Optimization
What would come as a surprise to academics, and what has surprised me, is that the only way I have ever seen costs developed on cost optimization projects is the third method, the Crazy Cost, or Interactive Cost Development Method. Interestingly, in different conversations with the various implementers of cost optimization systems, the responses I get back is.
“Actual costs were not known,”
“We tried actual costs, and they did not work, so we went to this.”
This is not one observation, but many observations, and not just my observation, but has been verified with some other consultants who have also seen multiple cost optimization projects. What this tells me is that the Crazy Cost Method is the dominant method of developing costs on cost optimization projects and that using actual costs, or even actual relative costs are the exception, not the rule.
What this means that the cost optimization that is occurring is not based upon the costs that various things cost the company. The costs are simply rigged to give the desired output. So if a very high penalty cost is required to get enough stock held by the system, then it is simply increased until the desired effect is obtained.
Secondly, if actual costs cannot be estimated and if estimated cannot be used, then it calls into question whether the objectives of companies that implement cost optimization projects consider cost minimization as the overriding objective of their supply chain organizations. Cost minimization is the objective function of the cost optimizer and therefore should be the objective of the company implementing this technology. If this is not the actual goal, then cost optimization or cost minimization is not the correct supply planning method for that company to use. This addresses the core of how various methods are selected for implementation.
Many people think that cost optimization is advanced, and it is. However, it is also very specific and does only one thing, minimize costs. The match between the biases of the technology and the goals of the business is the most important factor in method selection, not whether one method is more advanced than another.
Implications When Comparing Cost Optimization to Inventory Optimization and Multi-Echelon Planning (MIEO)
The implications for inventory optimization bear discussion. Inventory optimization and multi-echelon planning use goals that are more easily quantifiable by supply planning organizations. These goals are services levels and inventory quantities. This means that inventory optimization should not face the same issues that have faced cost optimization on variable estimation. Companies have specific service level and inventory goals that they can place into the system. However, while MEIO can certainly be thought to have an advantage over cost optimization in the determination of their input controls, once service levels are broken down into different product grouping it turns out companies run into complications developing these service levels as well.
The way costs are developed during implementation is not the way that cost optimizers were originally developed to be used. This is not to say that this method is wrong. However, when costs are developed in this way, in fact when any input is developed differently from how the software has assumed it would be developed, it should be understood and recognized regarding the implications for the plan and cost optimization. Currently, I don’t think these effects are widely understood.
Why Not Use Cost-Based Optimization?
If optimization did not “change the world“ of the supply chain, the question naturally becomes “Why not?“ There are probably some reasons, however, in our view, one of the most prominent came down to implementation and maintenance difficulty combined with company knowledge. Implementing and maintaining optimization methods requires lots of effort and long-term investment. Secondly, optimization requires an enormous amount of discipline and expertise on the part of the implementing company.
Many companies want the benefit of advanced planning, but are not culturally, financially or prepared from a skills perspective to make the sacrifices required to obtain the outcomes they desire. More often than not, companies want simple solutions that deliver value over a short time horizon. That is not what optimization provides. On the other hand, software deserves some of the blame as much advanced planning software has been designed to be more complicated than is necessary.
Optimization is always the most complex and challenging of implementation but also brings significant benefits if implemented correctly. However, for most that mountain is too high to climb. I sometimes get an email from technical people who like optimization and wonder why I am “negative” on it. This has nothing to do with whether I like optimization or not. I am reflecting on the experience of companies, most of which are unsuited to use it. A lot of money has been spent or wasted on optimization that could have been used to solve the same problem more directly and in a more sustainable manner.
In addition to lower investment, the returns from optimization have not been encouraging, and thus it’s incumbent upon me as a consultant to bring these failures to light for clients and to gauge the customer’s ability to implement optimization. This is very infrequently done.
“Real Optimization with SAP APO,” Josef Kallrath, Thomas I. Maindl, Springer Press, 2006
I cover supply chain cost optimization in the following book.
Supply Planning Book
Showing the Pathway for Improvement
Supply planning software, and by extension supply planning itself, could be used much more efficiently than it currently is. Why aren’t things better?
Providing an Overall Understanding of Supply Planning in Software
Unlike most books about software, this book showcases more than one vendor. Focusing an entire book on a single software application is beneficial for those that want to use the application in question solely. However, this book is designed for people that want to understand supply planning in systems.
- What methods fall into APS?
- How do the different methods work and how do they differ in how they generate output?
- What is the sequence of supply planning runs?
These types of questions are answered for readers in this book.
This book explains the primary methods that are used for supply planning, the supply planning parameters that control the planning output as well as how they relate to one another.
Who is This Book For?
- Chapter 1: Introduction
- Chapter 2: Where Supply Planning Fits Within the Supply Chain Planning Footprint
- Chapter 3: MRP Explained
- Chapter 4: DRP Explained
- Chapter 5: APS Supply Planning Methods
- Chapter 6: APS for Deployment
- Chapter 7: Constraint-based Planning
- Chapter 8: Reorder Point Planning
- Chapter 9: Planning Parameters
- Chapter 10: How MRP, DRP, and APS Relate to One Another
- Chapter 11: Supply Planning Visibility and Master Data Management
- Chapter 12: Understanding the Difference Between Production Versus Simulation