- Consulting companies, vendors, IT analysts all seek to corrupt the process.
- Find out how to manager the process the best way.
This is the most conventional chapter in the book. While there are few books on software selection, there are some articles that focus on how to do the basic parts of software selection such as asking for responses from the vendors, and creating a vendor selection matrix. The reason I call these tasks “basic” is that literally anyone can do them, and for the most part they are not what differentiates successful software selections from the unsuccessful. As I stated in the book’s introduction, the most important part of a software selection is not the elementary blocking and tackling, but the evaluation of the information that is brought into the software selection process. The almost total ignorance on this issue was a main motivation for writing this type of book in the first place. However, this book would be incomplete without a nod to these other undertakings, and in fact I have found that there are some ways to improve even these basic activities.
The Different Roles of the Software Selection Team
Michael Burns, in his article Software Selection Done Right, describes the software selection team in the following way: “At our first meeting, we also explain the roles and responsibilities for the project. We typically identify roles for the sponsor, steering committee, project manager, project coordinator, business-process owners, subject-matter experts and technical leads. The idea is to determine which tasks fall within the purview of each role and to name the people responsible. It is essential that the right people be assigned to the project. For example, the project manager must be very organized and subject-matter experts must be highly knowledgeable about their business processes. Also, they must have enough time for the project.” While I love the idea, I have never seen a software selection team so dutifully configured.
In fact, I cannot even recall very much input being solicited from the business-process owners, subject matter experts or the technical leads. Most executives seem to think they have all the requisite knowledge to make the decision as long as they have seen a demo and read the Gartner Magic Quadrant Report on the software to be selected. I don’t see how multiple areas of expertise could be leveraged (as proposed by a number of authors) into advice that is followed because I observe clients being blind-sided continually by requirements that cannot be met by the software they selected. Furthermore few people within the buyer have enough time to dedicate to the software selection. However, this intelligent orientation regarding team development and providing employees with sufficient time to participate and contribute to the software selection is shown in the quotation below from the book Modern ERP:
“ERP system implementations require a significant human resource cost. Smart companies dedicate their most valuable and knowledgeable employees to the ERP project for a significant period of time. Each team member will need to educate other members on their respective functional area. Commitment to the team will undoubtedly conflict with the employees’ normal job functions…. Creating the optimum team often requires backfilling key personnel to allow those chosen on the team to be fully dedicated to the implementation.”
Again, this also sounds great. But it just is not done in practice, or at least not frequently. Instead the participants are expected to fit the software selection in with their regular duties, but they are given very little incentive to make software selection a priority. Those on software selection teams are generally juggling their software selection duties on top of their regular work. This is one basic reason that software selections are performed so poorly. When I arrive at a client, I can find no well-reasoned explanation that has been documented as to why software was selected. If I am lucky I will be able to find a spreadsheet/matrix, which describes the different criteria that were selected. Companies simply do not make software selection a priority. Therefore, while Michael Burns is describing the right approach, I question how much time I should spend explaining the different roles if companies are unwilling to incorporate the input from such a large cross-section of the company. I found this comment from Michael Burns amusing:
“First, many of them know the business really well and can add a lot of value and input. Second, they are more likely to buy into the selection decision.”`
This is absolutely true, but it is also generally true that input from these individuals is not valued. Does this attitude lead to bad decision-making? Yes, it does.
However, I don’t think this will change by me writing a book on the topic.
We have entered a new era of elitism in the US where the executives know everything, add most of the value in the company, and the rest of the company is basically expected to “go with the flow.” How often do we hear about the insights and experience of the top managers of the company, that must receive top compensation or else they will fly away to another company. Compare this to how often we hear about the insight of technologists or applications users and how important they are to the company’s success. It’s a question worth considering, because if we have a flawed model as to where the knowledge lies that is required to make good software selection decisions, poor decisions will continue to be made. On the other hand, I only have consulted with large companies, and large companies tend to be quite hierarchical and therefore exclusionary. Also, as an implementer, I have not seen as many software selections as others. I know the job is not being done well, because I deal with the fall out. The individuals who see the most software selections are those who work in a sales or marketing capacity for a software vendor. For this reason I decided to ask a few colleagues their opinions of how well most software selections are performed. These individuals work in senior leadership roles in marketing in some vendors.
I received some interesting responses.
“I have seen so many variations, with perhaps these common themes:
Processes that are too ‘top down’ and IT-driven just as you describe, especially in ERP and commodity buys.
Others where the ‘users rule,’ also not so good because they tend to block innovative thinking.
The best processes tend to be driven by line-of-business middle managers, who often are low enough to understand the subtleties of what they need, but high enough to be able to think outside the box. So for instance, in our area, a supply chain director. I would also say that procurement (i.e., software selection) processes have actually gotten marginally better in terms of ‘due diligence’ since the ‘go-go’ days you and I remember. I can remember C-Level deals that weren’t vetted at all. In supply chain, that rarely happens, and in fact the RFI process can become rather too long and laborious, with perhaps adding little value for the extensive level of effort required.”
The Role of the Coordinator
In a software selection, the decision makers will be a company’s executive leadership, typically a combination of business and IT executives. Finance will be involved when it comes to discussions about budgets and money, but unless the application is finance-related (that is, software for their department), they are typically not that involved in the software selection process. However, while executives are the primary decision makers, they do not necessarily have to be the ones managing the process.
One weakness of many software selection initiatives is that the executives often try to manage all of the logistics of the process, when in fact it makes much more sense to assign the task to a manager-level individual in the company. As long as this person is not overwhelmed with other work (something to be considered when making the assignment), he or she provides a significant benefit in managing logistics and communication, in maintaining the folder of vendor documentation and the requirements list, and in tabulating the scores for the vendors. They may also have domain expertise if they are chosen from the area where the software is to be implemented.
These are the major roles of the software selection coordinator:
- Manage the Communication with all of the vendors: This includes everything from the initial introduction, to providing feedback on what the buyer would like to see demonstrated and how they would like the demonstration to be presented.
- Timing administration: Setting up the meetings and managing the calendar and logistics for web meetings and for vendor visits to the buyer’s site.
- Maintain the network folders for the implementation: These folders include correspondence from the vendors, their marketing literature, screen shots, IT analyst reports; anything that is pertinent to the software selection should be maintained in such a way that all the decision makers can easily find it and review it at their convenience.
- Development of the Vendor Grading Document: This means continually updating the document with contextual information as well as maintaining the scores.
- Serving as a mediator for all meetings: This ranges from managing introductions to walking the executives through the vendor grading document. Having a single person do this increases the likelihood that the things that need to get done will get done in a consistent manner. The executives themselves decide how much of a vote they would like the coordinator to have in the final software selection. As the coordinator should be from the area where the software is to be implemented, he or she should be able to offer good, detailed advice to the executives. Beyond domain expertise, the coordinator should have the following qualities:
- They should have the respect of the executives: Part of the coordinator’s role is to serve as a moderator, and this means managing the interaction of the executives. This is related particularly to the next topic.
- They should mediate selection bias: Software selection is frequently victim to selection bias. This means that each executive decision maker will seek to find the best software for the group or department that he or she represents, rather than the software that offers the best overall compromise for the company. This issue is most prevalent on ERP projects because ERP applications are such a diverse combination of functionalities. Applications that are more narrow in focus are less affected by this bias, but the problem arises again when buyers purchase software suites from one vendor rather than software from separate vendors for each category.
- They must be organized and good at documentation: This will be necessary to perform the coordination and maintain the documentation.
- They must be good communicators: The coordinator will need to communicate with many individuals and so must be able to do so in an efficient manner.
Implementing the Scientific Method in the Software Selection
I explained that the advice provided would focus on making the software selection process more scientific. Many people confuse science with sophisticated technology or scientific instruments. However, to be scientific is simply to follow principles related to how testing is performed. Following the scientific method does not require any scientific knowledge or even fancy equipment. This is explained in the quotation below from Wikipedia on the scientific method:
“Scientific inquiry is generally intended to be as objective as possible in order to reduce biased interpretations of results. Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, giving them the opportunity to verify results by attempting to reproduce them. This practice, called full disclosure, also allows statistical measures of the reliability of these data to be established (when data is sampled or compared to chance).”
The analogy between software selection and science is, of course, not perfect. A company does not actually publish the results of its software selection documentation for other companies to evaluate and reproduce.
Unlike public entities, private companies don’t have any interest in helping other people understand what they are doing or in advancing knowledge. Instead their objective is to “maximize shareholder value,” which essentially means to maximize the share price. In fact, even public companies do not publish the software selection results. However, portions of the software selection can be approached scientifically and the following sections will explain how.
The Importance of Documentation in the Software Selection Process
Gregor Mendel is an excellent example of the importance of documenting observations. He advanced the understanding of genetics through the rather low-tech approach of measuring the characteristics of successive generations of peas, which he grew in the garden of his monastery.
“By simply counting peas and keeping meticulous notes, Mendel established the principles of inheritance, coined the terms dominant and recessive, and was the first to use statistical methods to analyze and predict hereditary information. For eight years, Mendel cultivated thousands of pea plants and used a paintbrush to painstakingly transfer pollen from one plant to another to make his crosses (all the while still attending to his duties as a monk and a teacher).” — Wikipedia
His research paper is still available, at a link that is in the reference section of this book. His notes are quite meticulous, and this was a researcher who really knew how to document his results. With Mendel’s disciplined and scientific mind, he would have been excellent at software selection, or any area, which required a detailed and analytical mind. Science works because it follows a set of rules that allows for objective comparison of the variables to be observed. Documentation is a big part of the scientific approach because our memories are far from perfect, and recording allows for comparisons across observations that are separated in time. For example, not all the vendors that a buyer is looking at will present their demos on the same day. Information from vendors and from other sources will similarly be time lagged. Secondly, information will come to different people.
Consistently documenting these disparate data points allows them to be analyzed in a controlled manner. It also allows others to analyze them in a comprehensive fashion both prior to the software selection decision as well as after the fact. There is a list of what is called cognitive biases (such as confirmation bias, which is the predisposition to search for data points that support pre-existing beliefs) and many of these cognitive biases are at least partially addressed through thorough documentation.
Tabulating the Scores for Each Vendor
Some software selections use a formalized scoring spreadsheet and some do not. Some companies simply write a short report for why the software was selected, but many do not. However, for the following reasons I would very much recommend putting together some type of scoring sheet:
- Software selection means comparing a large number of criteria. It is difficult to really see how the vendor products compare across so many criteria without some type of graphical representation.
- The scoring document clearly communicates to everyone what criteria are being used for the selection and how the software under consideration is weighted. I am surprised to see so many software selection spreadsheets that do not have the ability to weight the criteria. The executives do not have to agree on the weighting. Each can provide a weight, which can be adjusted based upon how much of a vote they have. However, the company needs to come to an agreement on the weight per criteria.
- At the final selection meeting, having a document helps ground the discussion and allows everyone to use a single frame of reference for the decision. Many of the executives will have missed some of the data that has been documented in the scoring spreadsheet. Reviewing the scoring document will help bring all the executives, as well as other software selection team members, to the same understanding.
- A software selection is an important decision for the company. For such a decision it’s important that the outcome is based upon as much objectivity as possible. I find that matrix helps increase objectivity.
- The scoring document becomes a long-term reference for the company. It is also useful for the stage that follows software selection: negotiation (covered in the following paragraphs).
- The software selection matrix is easy to do. I have placed a sample software selection matrix below.
A software selection matrix combines the scores for each vendor for a variety of criteria along with the criteria weight. This software selection matrix is simply one example, and of course it can be extended to have any number of criteria. The individuals who are part of the selection must agree as to the weight per criteria. As soon as that is done, the vendors are rate, and oftentimes sorted on the basis of their combined score. I have used a derivation of this matrix and I have always found it useful. Occasionally individuals who are pulling for a particular vendor will criticize the scoring document by saying something like,
“I already know what the scoring will be, so I don’t have to look at it.”
Typically these statements are an attempt to undermine a rational approach to the software selection and to minimize the inputs of others so that their preferred vendor can win. Emotions can run high during a software selection, which is why it’s important to gain agreement on both the criteria and the weight assigned to each criterion (which can be voted on, resulting in a blended weight) before the software is scored. The criteria and weights should be reviewed several times and then set in stone. If the weights are reviewed or requests to change the weights are received after the software vendor gets its rating, then what is most likely happening is that the data is being changed to fit a desired outcome. The methodology of the scoring for the software selection should be straightforward. The winning vendor should be the one with the highest combined weight output of all the criteria. Anything can be included as criteria—price, the perceived ease of working with the vendor—the criteria are subjective. They are whatever the software selection team chooses to value. There are competing ideas regarding the inclusion of vendors that the buyer cannot afford. On one hand, the buyer is wasting the vendor’s time and their own time. On the other hand the company learns more about what is available in the marketplace as well as understanding what they are trading off to get a vendor they can afford. There is no perfect answer to this question.
Making the Decision
There has been a good deal of research into group decision-making. One of the findings is that the more vocal members of a group easily influence other group members, even if the more vocal members do not necessarily have more domain expertise than the less vocal members. What is desirable is that each individual makes their own decision such that the decision is reached independently and is not unduly influenced by the personality of one or a few people. To accomplish this outcome, the RAND Corporation developed the Delphi Method, named after the city of Delphi in Greece, the location of the Oracles who were consulted by the Ancient Greeks and Romans, among many others. RAND’s research into the Delphi Method goes back to 1943 when they performed a number of studies on this topic. The original intent of the research was to obtain better group judgment, by performing research, for example, into how the effect of strong personality types on groups can be mitigated (the research isolated the participants from one another).
The Delphi Method eliminates the group interaction as explained by RAND:
“RAND developed the Delphi method in the 1950s, originally to forecast the impact of technology on warfare. The method entails a group of experts who anonymously reply to questionnaires and subsequently receive feedback in the form of a statistical representation of the ‘group response,’ after which the process repeats itself. The goal is to reduce the range of responses and arrive at something closer to expert consensus. The Delphi Method has been widely adopted and is still in use today.” — RAND
The Delphi Method is often thought to relate directly to forecasting only, and in fact is an approach to consensus-based forecasting. However, it applies to any decision-making situation. The research into consensus decision-making, much of which was rolled into the Delphi Method, is useful for software selection, as is shown in the following quote about the Delphi Method (the emphasis is mine):
“The Delphi method (/’dεlfai/ del-fy) is a structured communication technique, originally developed as a systematic, interactive forecasting method which relies on a panel of experts. The experts answer questionnaires in two or more rounds. After each round, a facilitator provides an anonymous summary of the experts’ forecasts from the previous round as well as the reasons they provided for their judgments. Thus, experts are encouraged to revise their earlier answers in light of the replies of other members of their panel. It is believed that during this process the range of the answers will decrease and the group will converge towards the ‘correct’ answer. Finally, the process is stopped after a predefined stop criterion (e.g., number of rounds, achievement of consensus, stability of results) and the mean or median scores of the final rounds determine the results. Delphi is based on the principle that forecasts (or decisions) from a structured group of individuals are more accurate than those from unstructured groups.” —Wikipedia
Interestingly, I have never heard of an approach to making the software selection process more scientific and less susceptible to the influence of one or a few individuals. I also very much doubt that executives at a company would agree to something like the Delphi Method, as getting together with other executives is the normal way that decisions are made in most companies. On the other hand, people that reviewed this book prior to its publication have told me that they did something similar in their software selection, even though they may not have related it to the Delphi Method. One quotation I have included below:
“I facilitated two software selections. As I recall our methodology was to form a team comprised of middle management from the user department and IT, and we reported to upper management. We ‘workshopped’ the vendor questionnaire. I facilitated the workshops in a participatory manner that honoured all input without (too much) judgment and arrived at a consensus on the questionnaire. The questionnaire was based on the current business process (with improvements that could be taken advantage of if new software was acquired) and on user requirements in addition to IT’s requirements. So, in this case executives did agree to both the consensus process (probably because they didn’t care how it got done, just that it got done) and to the decision that was made via the process. We continued the consensus all the way through to selection without much dispute, and the same team, more or less, worked on the implementation. Because there was firm agreement on the criteria and on the software selected, we knew exactly what we were getting into and the implementation was very smooth.”
However, as I have explained with numerous examples, the current approach to software selection results in poor outcomes. The current design means that vendor sales and marketing count for more than the actual software. Companies that follow the realistic approach as outlined in this book can greatly improve the outcomes of the software they select. Moving from Selection to Vendor Negotiations As I do not have experience in negotiating with software vendors, I am not qualified to advise on the topic of vendor negotiations. Furthermore, a different group than the team that performed the software selection typically handles this part of the process. But the software selection is an input to the negotiation process, and the thoroughness of the software selection process can put those that negotiate with the vendors in the best possible position. Here is how:
- Often the group that performs the software negotiation, which may be from the procurement or contracts department, will not be familiar with the software and will not have participated in the software selection process. Therefore, it’s important to have an easy-to-understand document that explains the conclusions of the software selection.
- The vendor negotiation group should keep in contact with the software selection coordinator. There are all types of reasons for this, but one is so that the negotiation group does not lose the context of the software selection. Secondly, there were statements made by the vendor during the software selection, and those statements should not “go away” when the contract is about to be signed. A frequent technique is to sell one thing, but then have the contract say something that is more modest. Some very experienced consultants recommend writing the important promises from the sales process into the contract. By keeping a good communication level between the software selection coordinator and the vendor negotiation group, the buyer can stay out of these traps. Finally, the coordinator, and possibly other members of the software selection team need to review the software license and consulting agreement before it is signed. This will be a tedious process because much of the documentation will be in legalese. Rather than having each person try to get through the contracts themselves, it can make more sense to go through the documents as a group, and to have the company’s internal council there to translate what things mean in plain English.
- The software selection process will result in a list of vendors, which can be sorted from the first choice to the remaining choices. As with any purchase decision, the first choice may lose out to the second or third choice when negotiations determine that an agreeable arrangement cannot be arrived at with the first choice. A thoroughly-filled out software selection scoring matrix explains not only the sequence of the choices, but also the degree of preference for one vendor over another. This gives very important information to the individual/group negotiating with the software vendor. A small differential between the top-rated vendors means that the negotiating group can look for opportunities for a lower price from another vendor to tip the balance in their favor. For instance, in the sample-scoring matrix shown previously, there is a very small difference between Vendor 1 and Vendor 2, and only a one-point difference between the estimated total cost of ownership (TCO). If Vendor 2, the top choice, were to take a hard position in the negotiation, Vendor 1 would seem to be an available option for this buyer.
- The software selection team should be careful to communicate the TCO aspect of the selection (which is a criterion in my software selection matrix), because without this context the negotiating team may be blind to these costs and make their negotiating decision based upon the initial purchase price only. This will require the software selection team to have made certain assumptions regarding the total cost of ownership. It’s not an easy task, as TCO is tricky and has many factors, which must be added to the calculation. I believe it is more realistic, given the resources that companies seem willing to allocate to software selection projects, to simply rate a vendor’s predicted TCO on a scale from one to ten. The TCO value can be either added to the scoring matrix or assigned a weight. However, a thorough software selection would result in an estimate. There is also a question of whether to exclude those vendors entirely when it is suspected that their price will be too high. However, vendors are cagey regarding the price of their software and it can depend upon many factors, including which other products produced by the vendor the buyer currently owns. Furthermore, the “real price” can drop different degrees per vendor per situation, depending upon how the negotiation proceeds. This, of course, greatly increases the complexity of the selection process and reduces the transparency and efficiency of the enterprise software market.
Most publications on software selection cover the tactical elements of software selection far better than the broader or interpretational elements. This book has taken the opposite approach, focusing on how to interpret information that is received from the multiple sources that influence software selection. I allocated much less space to the tactical elements of software selection. However, this chapter covered these tactical elements of how to manage the software selection process. The software selection coordinator role is quite important to the effectiveness of the overall software selection process. In this chapter, I laid out the characteristics that make a good coordinator. It’s relatively easy to begin a software selection process without a coordinator, however, this is misleading. As the software selection proceeds there is more coordination required, more items that must be followed up, and eventually the requirement for a mediator between all the members of the software selection team. The coordinator should have the most time allocated to the software selection project of any of the team members (and needs to have this reflected in the rest of their workload) and should be able to keep the team focused, as well as provide the team with the necessary context to both interpret information and explain to them accurately how the software selection has proceeded up until the current discussion. A coordinator can allocate the type of time to a software selection that such an initiative requires, and the type of time commitment that the executives often cannot make. Software selections can be greatly improved by increasing the use of the scientific method, which is a structured approach to the testing of any hypothesis. Science works because it follows a set of rules that allows for objective comparison of the variables to be observed. And a big part of this process is to document information that is uncovered during the software selection. This documentation helps increase the rationality of the overall process and can help to place on even footing information that is often gathered months apart. A software selection matrix, which is a condensed way of making these comparisons, is not uncommon, but documentation during the software selection process should not be limited simply to comparison spreadsheets of this type. Documents with detailed explanations of the rationale for the positives and negatives of the various compared applications should also be created and made available to the current software selection team members as well as archived for future software selections. Documentation, and reviewing this documentation, can have the positive effect of mitigating the natural tendency of one or a few individuals with strong personalities and strong opinions from imposing their selection on the rest of the team. For instance, while a person may have only one vote in terms of their official capacity on the selection team, through the force of their personality, and their ability to persuade, they may be able to coopt other members to essentially give up their vote to these individuals.
The excessive influence of some individuals on group decision-making is why the RAND Corporation developed the Delphi Method. The Delphi Method makes the “voting” private, which tends to be less influenced by factors such as being in the same room with an influential individual. This is also why political voting is performed behind a curtain of some type. Once the software selection has been made, typically the negotiation of the contract is transitioned to another group. This is where the documentation that was developed during the software selection can be put to another good use by allowing the group that performs negotiation to understand why the vendor was selected. The group that negotiates must know not only the first choice, but the second and even third choice because an agreeable arrangement may not be feasible between the buyer and the first choice. A thoroughly-filled out software selection scoring matrix explains not only the sequence of the choices, but also the degree of preference for one vendor over another. However, someone who was part of the software selection team should keep in contact with the group that performs the negotiation. There are some with expertise in this area that believe that a person from the software selection team is in the best position to perform the negotiation with the software vendor, however, this is both often not feasible because of how companies are structured, but I question if this is the right approach because negotiation is really a different skill than software selection. If I take myself as an example, while I am qualified to perform any supply chain planning application software selection I do not negotiate contracts and would be at a distinct disadvantage performing the negotiation against a software vendor as they would certainly have a professional negotiator. I would think the best possible combination would be for the member of the software selection team to work with the individual with negotiating expertise in order to perform the negotiation together. Furthermore, the software selection team should be careful to communicate the TCO aspect of the selection (which is a criterion in my software selection matrix that I showed in this chapter), because without this context the negotiating team may be blind to these costs and make their negotiating decision based upon the initial purchase price only.
Software Selection Book
Enterprise Software Selection: How to Pinpoint the Perfect Software Solution Using Multiple Sources of Information
What the Book Covers
Essential reading for success in your next software selection and implementation.
Software selection is the most important task in a software implementation project, as it is your best (if not only) opportunity to make sure that the right software—the software that matches the business requirements—is being implemented. Choosing the software that is the best fit clears the way for a successful implementation, yet software selection is often fraught with issues and many companies do not end up with the best software for their needs. However, the process can be greatly simplified by addressing the information sources that influence software selection. This book can be used for any enterprise software selection, including ERP software selection.
This book is a how-to guide for improving the software selection process and is formulated around the idea that—much like purchasing decisions for consumer products—the end user and those with the domain expertise must be included. In addition to providing hints for refining the software selection process, this book delves into the often-overlooked topic of how consulting and IT analyst firms influence the purchasing decision, and gives the reader an insider’s understanding of the enterprise software market.
By reading this book you will:
- Learn how to apply a scientific approach to the software selection process.
- Interpret vendor-supplied information to your best advantage. This is generally left out of books on software selection. However, consulting companies and IT analysts like Gartner have very specific biases. Gartner is paid directly by software vendors — a fact they make every attempt not to disclose while consulting companies only recommend software for vendors that give them the consulting business. Consulting companies all have an enormous financial bias that prevents them from offering honest advice — and this is part of their business model.
- Understand what motivates a software vendor.
- Learn how the institutional structure and biases of consulting firms affect the advice they give you, and understand how to properly interpret information from consulting companies.
- Make vendor demos work to your benefit.
- Know the right questions to ask on topics such as integration with existing software, cloud versus on-premise vendors, and client references.
- Differentiate what is important to know about software for improved “implement-ability” versus what the vendor thinks is important for improved “sell-ability.”
- Better manage your software selection projects to ensure smoother implementations.
- Chapter 1: Introduction to Software Selection
- Chapter 2: Understanding the Enterprise Software Market
- Chapter 3: Software Sell-ability versus Implement-ability
- Chapter 4: How to Use Consulting Advice on Software Selection
- Chapter 5: How to Use the Reports of Analyst Firms Like Gartner
- Chapter 6: How to Use Information Provided by Vendors
- Chapter 7: How to Manage the Software Selection Process