- Gartner states that their ombudsman protects against their massive financial bias.
- We evaluate the accuracy of Gartner’s claim.
The area that Gartner most often refers to as their check on the bias is the Office of the Ombudsman, which manages against conflicts of interest and ensures fairness. Before we get into what the Gartner Ombudsman is, let’s start by analyzing the terminology.
An “ombudsman” is normally appointed by a government body and provided with the independence to investigate rights violations. The term “ombudsman” is infrequently used in the modern business environment. An ombudsman is similar to an Inspector General, something that everyone has heard of with respect to the United Nations. The definition of the Inspector General from the UN website is informative:
“UNHCR’s Geneva headquarters has three mandated functions: to assess the quality of UNHCR’s management, including identifying measures to prevent mismanagement and waste of resources; to investigate allegations of misconduct by anyone working for the agency, including non-permanent staff; and to conduct inquiries into violent attacks on UNHCR staff and operations, as well as other types of incidents that could affect the reputation of the organization.”
Gartner describes their Ombudsman as follows:
“The Gartner Office of the Ombudsman has been modeled after that of the news industry, in which the ombudsman takes action on issues of accuracy, fairness, tone and balance submitted by newspaper readers or radio listeners. As in the news industry, the Gartner ombudsman recommends approaches to address these issues via content or process updates.
At Gartner, these issues may be raised by clients, non-clients or employees. We don’t simply react to comments from these audiences—we invite these comments with a dedicated Web site and staff that address issues as they arise.
Given the nature of the current environment, we have formalized and publicized our commitment to objectivity to benefit both our associates and our clients. The ombudsman simply makes more public the ethics and standards we established when we were founded in 1979 and which continue to evolve.”
Sounds good, doesn’t it? But, the explanation must be examined within the context of how different Gartner is from most media outlets, in particular with the statement that…
“The Gartner Office of the Ombudsman has been modeled after that of the news industry….”
Modeled on the News Industry?
While it may be true that the office was “modeled” on the news industry, the position of Gartner vis-à-vis news industry firms is completely different; news organizations do not see every entity about which they write an article as a potential customer. To see why this is the case, let’s perform a thought experiment. Imagine that Time Magazine was receiving payments from many of the entities about which they wrote articles. Of course, in a way Time Magazine does accept payments as they accept advertising. But, not every entity that Time reports on is a potential revenue source. Over at Gartner, however, every entity they report on is either a current or a potential revenue source. In addition, how much an advertiser advertises in a media outlet is a matter of public record. One can easily tell by reviewing the magazine, newspaper, television channel or website and checking who has advertised; this is why we know the relationship between financial contributions and the effect on media output. However, which software vendors pay Gartner—and how much they pay Gartner—is not a matter of public record.
The Value of the Ombudsman?
Now comes the interesting part of the thought experiment. Could Time Magazine assuage people’s concerns about reporting bias by simply saying, “It’s OK. We have an ombudsman?” I don’t think it could. In fact, let’s take the example of the Swedish press Ombudsman.
“Pressombudsmannen (or press Ombudsman) is a person whose role in the Swedish print media is to determine whether the actions of a newspaper are in line with good journalistic practice. Complaints regarding the practices of print media can be reported by the general public to the Pressombudsmannen who determines whether a complaint should be brought before the Swedish Press Council (PON). The PON can issue fines of up to SEK 2,000 and publish a rejecting opinion.” – Wikipedia
However, notice that the Press Ombudsman can refer the matter to the Swedish Press Council and a fine can be issued. There is no possible referral to anyone outside of Gartner system. In fact, the actual complaints are not published, nor are the rulings and transparency of the Gartner Ombudsman is nil. While Gartner states that they have a dedicated website, it is used primarily for public relations to tell people about the effectiveness of the Gartner Ombudsman. There is no publishing of information regarding current complaints.
Impartiality is Proven by Gartner How?
What is really known about the ombudsman in terms of his or her impartiality? For instance, let’s imagine that a large vendor has a complaint regarding the fairness of a rating, while a smaller software company that does not pay Gartner (or pays a much smaller amount) has the same complaint. Which complaint would be a higher priority for the ombudsman? Anyone can guess, but there is no way to know for sure. Research requires data points, and Gartner does not publish the complaints, nor does it show how the ombudsman ruled. In a court case brought against Gartner by ZL, which claimed that Gartner’s Magic Quadrant was “…misleading and favored large vendors,” Dennis Howlett, writing for ZDNet, had the following to say about the ombudsman.
“Gartner currently has an employee act as ombudsman to handle disagreements. The conflict of interest is self-evident in the way ZL’s concerns were summarily dismissed with little supporting evidence. There is a crying need to establish an impartial ombudsman similar to those found in public media, in order to ensure purchasers that they are receiving impartial analysis.”
The Ombudsman Called out as a Phoney and Biased Entity
So ZL proposed in the action against Gartner that the Gartner Ombudsman is simply fascia and does not meet the standards of ombudsman in other areas, particularly in print. In this regard, ZL must be considered to be correct. Gartner could do a number of things to improve the Office of the Ombudsman.
- The Office of the Ombudsman could be placed outside of Gartner and funded by software buyers.
- The Office of the Ombudsman could publish the complaints and the rulings on these complaints on a public website.
As a research organization, Gartner should understand that their statements regarding the impartiality of their “ombudsman” are insufficient, for the same reason that Gartner should not accept a vendor’s statement that they have “the best software in the world” or that they have “one hundred percent customer satisfaction,” without actually asking for evidence of these claims. If research were just repeating what entities said about themselves, it would not be called research.
The PR Function of the Ombudsman
Gartner’s description of the ombudsman function focuses on how the ombudsman rules on fairness complaints that are made to Gartner. However, this leaves out the ombudsman’s quite significant public relations function. For example, the ombudsman—currently Nancy Erksine—is active on Internet comment sites stating that Gartner’s ratings are never for sale. She also comments on the Ombudsman website, generally saying good things about Gartner and so on. The Ombudsman website also informs visitors that Gartner analysts are not allowed to own stock or have other types of investments in vendors.
Would the Ombudsman Every Criticize Gartner?
The statements made by the ombudsman are designed to counter arguments that are critical of Gartner’s objectivity. The entirety of evidence presented by Gartner regarding its objectivity essentially comes down to the ombudsman’s word that Gartner has a number of controls in place, and therefore its research can be trusted to be without inappropriate influences. These statements are not auditable except by looking at Gartner’s media output and by interviewing a number of people at vendor and buyer organizations, much of which would contradict the statements made by the ombudsman.
The Objectivity of the Gartner Ombudsman?
Another problem with the ombudsman’s statements is objectivity. An individual has objectivity when they can reasonably be expected to be indifferent as to which way the results swing. For example, salesmen cannot be said to be objective between the vendor for which they work and the vendors that are competitive because they are paid to sell the products of one vendor. They could not hold the position that the software of competing vendors is superior and continue to be employed. The same is true for the Gartner ombudsman. If Nancy Erksine were to begin telling members of the media that Gartner’s vendor ratings can be purchased or that Gartner analysts take positions in the stock of vendors prior to the publication of Magic Quadrants, how much longer would Nancy Erksine be the Gartner Ombudsman or be employed at Gartner? Since the answer is “not long,” it is clear that the Office of the Ombudsman is not objective. Being objective means that it is at least possible for you to hold the opposite view from that which you currently hold.
Gartner has borrowed the name of “ombudsman” for their mediating position, but have not imbued the role with the required independence to meet the definition of the term as it is used, not only in the media but in other areas as well.
What the Ombudsman Means for Vendors
Vendors should view the Gartner Ombudsman as principally a public relations mechanism to make buyers comfortable with Gartner’s reports and advice. If a vendor does not rank as desired in a report, appealing to the ombudsman will do little. All vendors want to be rated superior to other vendors regardless of whether there is any justification for a superior rating. Gartner analysts are constantly hearing from vendors about how they think they should be rated higher than they were in the last review, so after some point, analysts will tune these vendors out. To improve one’s rating in the future, a far more effective use of time and energy would be to hire Gartner to better understand their ratings and then do what it takes to score higher in these ratings.
Search Our Other Gartner Content
Financial Bias Disclosure
Neither this article nor any other article on the Brightwork website is paid for by a software vendor, including Oracle, SAP or their competitors. As part of our commitment to publishing independent, unbiased research; no paid media placements, commissions or incentives of any nature are allowed.
Gartner is the most influential IT analyst firm in the world. Their approval can make or break a vendor in an application category, or at the very least control their growth. Gartner has been behind most of the major IT trends for decades. However, many people read Gartner reports without understanding how Gartner works, how it comes to its information, its orientation, or even the details of the methods it uses for its analytical products. All of this and more is explained in this book.
Table of Contents
- Chapter 1: Introduction
- Chapter 2: An Overview of Gartner
- Chapter 3: How Gartner Makes Money
- Chapter 4: Comparing Gartner to the RAND Corporation, and Academic Research
- Chapter 5: The Magic Quadrant
- Chapter 6: Other Analytical Products Offered by Gartner
- Chapter 7: Gartner’s Future and Cloud Computing
- Chapter 8: Adjusting the Magic Quadrant
- Chapter 9: Is Gartner Worth the Investment?
- Chapter 10: Conclusion
- Appendix a: How to Use Independent Consultants for Software Selection
- Appendix b: What Does the History of Media Tell Us About This Topic
- Appendix c: Disclosure Statements and Code of Ethics