Who Performs Honest IT Risk Management and Duration Estimation?

Executive Summary 

• Many factors conspire against realistic and accurate estimates of IT project duration estimates.
• We cover who has input to the estimate and how their bias impacts the estimate.

Introduction

For software buyers, they face a significant amount of information that causes underestimation of implementation duration. This article will discuss who does implementation duration estimation and what it means for project planning.

Duration Estimates from Software Vendors

Software vendors attempt to reduce the appearance of risk on the application. Vendors could be an excellent source of information about how long a software project takes. However, because they are selling software, their quotations are often unreliable. Working on the consulting side and providing many sales estimates over many years, I was always pressured by sales to reduce the duration. With confidence, I can say that normally sales want the project to use as oversimplified assumptions as possible and assume that there will be no problems on the project. To salespeople, problems don’t happen; everything runs smoothly. Sometimes estimates are developed by presales rather than consulting, and in those cases, the estimate’s reliability is even lower. Presales reports to sales and pre-sales resources that don’t give short-duration estimations won’t work in presales for very long. The threats against those who would provide accurate project estimations are a constant factor of working in most software vendors.

Those that give a realistic plan, no matter how well-founded and how much experience the person making the plan has, will need to prepare for negative reinforcement. This opens up an opportunity for less principled resources, which will give the prospect and sales what they want, which is an unrealistically fast implementation plan. This is why it should be noted. Accuracy is not the design goal of the vast majority of IT project plans.

This is why to reduce the risk of the project, the information provided by the vendor must be filtered, and things like estimates of how long the project will take must be made not by taking some of the information provided by the vendor and using benchmarks for projects of the same type.

The Best Case Scenario

If we think of all of the participants, there is a person from consulting from the vendor involved in the best-case scenario. But this person is often senior themselves. They have to combat their own sales team’s expectations, the customer, their own consulting management (that is often receiving pressure from sales). Therefore, the number of entities that favor an unrealistically short estimate outnumber the number of entities that favor a realistic estimate.

Bias on the Part of Software Buyers

Software buyers want fast implementation timelines. If they plan to use the new system to replace an ailing system, the fast implementation motivation is even faster. Therefore, while there is plenty of blame for software vendors and their consulting partners, project duration estimates are made by the management of the software vendors and consulting partner to the management of the software buyer. That is, very few of the people involved in the estimation end up doing the hands-on work.

Implementation Duration Estimation and Project Quality

If the project duration is not correctly estimated from the beginning. If it is set as too short compared to what is needed, the risk increases because companies have a natural incentive to meet their deadlines that they set for themselves. And if the time estimate is too short for the functionality that needs to be implemented, they will often roll out the application before it is ready and properly tested.

This can lead to a vicious cycle where the company keeps trying to meet successive deadlines, such as rolling the functionality to new regions, or performing new phases of the project, or bringing new functionality within the application online – and there is no time to go back and fix the functionality that was not configured properly in the first place. This problem impacts not only software implementations but development projects as well.

The Example of the Fusion Debacle

One of the most poorly managed software development projects in history was/is the Oracle Fusion project. This was an attempt to modernize and integrate many of Oracle’s acquisitions. This is a project that never seems to end but also never seems to make much progress. It was announced too early and has been in firefighting mode since its origin in 2005.

Breaking the Project into Shorter Go Lives/Agile

One adjustment that can be performed to increase the duration’s estimation is to break the project into more go-lives, bringing on the application’s functionality in successive waves. This uses the observations from the early go-lives to adjust the implementation duration of the successive implementations. That is, the scope of the first implementation is small. But it is something that can be brought up relatively quickly. The company should not adhere strictly to the duration estimate on this first implementation because the intent is both to bring up functionality and check the discrepancy between how accurate the estimate was compared to how long the implementation actually took.

Agile Versus Faux Agile

This project approach overlaps with the Agile method. Most people that discuss Agile tend to cherry-pick how it works. Under Agile, there is no duration per se. That is, the scope is added or removed based upon the timeline. True Agile essentially questions the ability to predict both scope and duration. Therefore, It is curious to analyze Agile and see how radical a concept is. After analyzing it in detail, it is clear that the vast majority of companies have no hope of implementing true Agile. True Agile contradicts the desire for control and the illusion of precision of project estimation prevalent in corporations. Agile is currently used as a type of virtue signaling and is often used as an excuse to engage in low-quality rework type development. For example, when Oracle failed to meet nearly all of their deadlines with Fusion, they moved to “Agile.”

Anyone who has worked in systems can easily imagine how to take a small portion of the overall functionality planned to be live eventually and only implement one portion of it. However, there is also the adjustment of implementing a more simple functionality method within the application rather than more complex functionality. For instance, in my supply chain planning area, there is functionality that is either simple or complex, which is designed to solve the same problem. A simple approach is called a heuristic, while the more complex approach using an optimizer because optimizers are much more complicated to implement and troubleshoot. If a company decides to implement the optimizer, it will always take longer.

The Problem With Putting Complex Functionality Earlier in the Project

What companies traditionally do is implement a more complex technique without first trying the more simple technique. And what often happens is they become overwhelmed with the complexity of the solution. This is a high-risk approach to system implementation, and interestingly, most companies have no idea that this is, in fact, a high-risk approach. They think they are doing the right thing by starting with the most complex method – that hypothetically will deliver the greatest business value. Typically, the software vendor will have spent a good deal of time differentiating itself because of its advanced functionality in some areas. Therefore, when the implementation project planning begins, the natural starting point is implementing this complex functionality. This is a faulty assumption. The problem with putting the more complex and advanced functionality in the first implementation/go-live is that the company is not experienced enough to create an accurate implementation duration estimate and will have a natural tendency to underestimate the duration. As with any topic, the less one knows, the more likely it is the estimate will leave out important details and result in underestimation. Choosing a simple method as the first implementation also allows the company to begin receiving value from the application sooner. The users use the application sooner, become familiar with it sooner, and a better-trained group of users can better support the future rollouts.

However, in most cases, the very opposite of this approach is employed. The implementing company will most often base its implementation duration estimate on biased estimates from the vendor or the consulting company. To dig a bit deeper on this topic, which provides the estimate within these companies very much changes the estimate’s accuracy.

Who Makes the Initial Estimate?

When I worked for a software vendor out of the consulting arm, I routinely had to lengthen the estimate that had already been provided by the software vendor’s sales arm, which had already made unrealistic estimates of how long implementations would take. This made me quite unpopular. However, none of the salespeople had implemented any kind, much less the one they just sold and most often pulled the estimates out of thin air. As they would not personally be performing the implementation and wanted to make the implementation appealing, they had a strong bias to quote the shortest implementation possible. That way, they got the most money (through the software sale) and left the consulting team with the least amount of money to actually implement the software.

After the software would be sold, I would come in and say that the estimate needed to be lengthened, which would create friction between the salesperson and myself and would usually get escalated to the director.

Getting the Estimate from the Right People in the Software Vendor

Any duration estimate from an individual who works against a sales quota and does not implement for a living is not reliable. It should go without saying that if one has no experience doing the work, they should not provide duration estimates. For instance, I am often asked to estimate the duration of development work. However, I am not a development manager and have never even worked as a developer. Because of this, I am not qualified to provide an estimate of any accuracy concerning software development. And if I do, I put the developer who does have to do the work in the undesirable position of going back and changing the estimate.

As my previous example shows, a buyer will often receive contradictory estimates depending on who they speak to within the software vendor. As a consulting manager, I was continually asked/pressured by sales to bring down estimates to not “kill the deal.” Sufficed to say, this is not a scientific environment within which to make an accurate estimate. Providing this background is why I say that the company must make its estimates without relying upon biased information from the consulting company’s software vendor. Furthermore, while a software vendor may have more knowledge about their application, the buyer has more information about their environment. Many software vendors will say, and I tend to agree with this, that the main factor in the implementation duration is the buyer, not the software vendor or the software.

• Buyers consistently underestimate their ability to load the new project work on top of their employees’ current workload. Buyers normally expect this extra work to be done for free.
• Buyers consistently overestimate the skill levels of their internal resources. Buyers prefer to “stretch” employees’ skills as it is preferable to the alternative, which is to hire more expensive employees. This translates to the “smart guy syndrome,” where a young, smart resource is overloaded with wor. This extends to loading some of the most complex work onto interns who are not in a position to say “no.”
• Buyers underestimate the amount of time their resources require for training.
• Buyers consistently overestimate their ability to incorporate complex functionality into their companies. Currently, ML/AI is all the rate. However, very few companies that go into ML/AI will mast ML/AI. Most of the projects will end up as failures.

For these reasons, I recommend creating an estimate based on a test case. And the best test case is the first implementation, which should be a simplified implementation using the simpler methods available within the application and a restricted scope (geography, user base, etc.). Once some basic functionality is working, and a first go-live is completed, the company is better positioned to estimate the follow-on implementations. This approach also forced the consultants to produce something in the short term, which can shortcut the common occurrence that the consultants become wrapped up in lengthy conversations with client resources discussing the advanced areas of functionality with client resources with no experience ever using the application.

Minimizing Risk Through Software Selection

Risk factors change based upon the software category but also within the software category by the vendor. Some vendors have far more difficulty implementing products than other vendors. This can be easily determined by simply attempting to configure the application oneself.

If we compare two applications with which the author has first-hand experience – SAP PP/DS and PlanetTogether Galaxy APS, Galaxy APS is far easier to configure, easier to change the master data, easier access more advanced functions, and has easier to setup simulation capabilities.  The two applications not only differ in their speed of implementation (Galaxy APS can be implemented in some weeks, rather than a year for PP/DS), but they also have very different risk levels. Because of the design problems with PP/DS, the application is simply a high-risk implementation.

All enterprise software categories are littered with similar examples. Although most companies do, it is a losing strategy to select software by a consulting company’s recommendation and based on short demos. Consulting companies are paid based on how long the project lasts. Companies can easily improve their IT implementations by simply choosing software, which is easier to implement.

Conclusion

There is ample evidence that duration estimates are both difficult and are developed under highly biased circumstances and impacted by both supply and demand-side bias to shorten the estimate to an unrealistic length. IT media entities often reported that a particular percentage of IT projects exceed their estimate and, therefore, their budgets, but what is left out of this observation is how biased and often lacking in information the initial estimates for IT projects are. In most cases, the estimates are not provided from how long it will take but rather from how appealing the duration is to the software buyers and the degree to which it helps salespeople fill their quota. Software duration estimation suffers from too little input from those that have to do the actual work and too much input from management that nearly always has blind spots that they do not recognize as blind spots.

Once the duration is set, the project plan takes on an undeserved reverence. When the initial unrealistic project plan cannot be met, it is most common to question the inability to meet the plan rather than questioning the plan itself.

Buyers that want realistic plans must not only fight against biased and overly management-centric project estimates from vendors and consulting companies but biased and over management-centric influences within their own companies.