Gartner

How Gartner is a Fake Research Entity Like JD Power and Associates

Executive Summary

  • JD Power and Associates is an automotive polling company that sells its ratings to the highest bidder.
  • The similarities between JD Power and Gartner are striking.

Introduction

A big part of what Gartner does is help defend legacy or obsolete vendors from technological change. This is an excellent market to be in because obsolete vendors always need “new paint jobs,” and they are willing to pay a significant amount of money to have Gartner confuse their clients as to what is happening in each technological space. In this article, we will cover the example of Equinix.

See our references for this article and related articles at this link.

What JD Power is Known For

JD Power is known for putting cars that rank poorly in dependability at the top of its dependability ratings. These are the same cars that perform very poorly in the results from Consumer Reports and are also known anecdotally as unreliable.

JD Power is a primary mechanism that makes such as GM, Ford, and Dodge fake their reliability and quality ratings. 

Chevrolet’s Deceptive Advertising

Chevrolet’s ads are widely considered a joke and widely lampooned on YouTube. Naturally, the claims that Chevrolet, one of the lowest ranking automotive brands often pays JD Power to provide them with good outcomes. The Chevrolet ads propose that Chevrolet routinely outscores brands like Toyota and Honda, that are widely reputed for their superior quality. 

Chevrolet takes the inaccurate JD Power and associate polls and then further exaggerates them. This video above shows exactly how. I have found the same thing with Gartner (and Forrester). In the article How Accurate Was The Forrester HANA TCO Study? we noted how Forrester specifically stated its estimates were projections. However, when SAP publicized the study, they changed it from a projection to a proven fact. No matter what an IT analyst writes, no matter how rigged, the IT analyst can depend upon the Marketing department at the vendor to further exaggerate the positive aspects of the finding.

Who is Ipsos in Automotive Quality Surveying?

In this commercial, they instead hired a company called Ipsos to fake the poll the way Chevrolet wanted.

The video makes a good point. Ipsos is not at all involved with automotive polling. We typed in Chevrolet into the Ipsos search engine and could not find any study about automotive reliability. This supposed study was showcased in a major national television advertisement, and it appears not to exist. 

And that is not the only strange thing about the supposed study.

The site you are supposed to go to, ChevyReliability.com, now does not exist and just forwards to Chevrolet.com.

What happened to the site with all of the evidence? It seems far easier when you are presenting a study to a group of people you paid to react to your statements (apparently). Notice that none of the people in the advertisement asked for details on the study.

The mystery exists at neither the described site nor at the research firm that supposedly performed the study.

Very similar to Gartner, all of the supporting data is hidden. Gartner uses the excuse that only a small portion of its research ever gets published. This is how Gartner hides the fact that there is very little supporting things like its Magic Quadrant.

Here it appears that JD Power created an award that Chevrolet could do well in, which is “initial quality” that is parodied in this video. 

The misleading initial quality rating is explained in this quotation.

Hardly a month goes by when there isn’t a JD Power & Associates survey on car reliability. Some of the results may not report what you think. Studies conflict each other to the extent that a top-five vehicle may be below average on a different study. That’s especially the case with the late June Initial Quality Study: It reports the number of “problems” cited per 100 new vehicles in the first 90 days of ownership. – Extreme Tech

Gartner and Forrester and IDC all create award categories that are designed to make one of the vendors they want to get income from to be in the leader category.

JD Power, like Gartner, has multiple revenue streams.

Power ranks all automotive brands top to bottom, but breakouts for individual segments only show the top three in many cases, and not those below average. – Extreme Tech

This is clearly promotional, as it prevents JD Power from losing the makes that score low in a particular segment. Notice this approach is entirely different from Consumer Reports, which is a real research entity, and publishes the complete list and shows how everyone scored. Another way that JD Power differs from Consumer Reports (which is never paid by vendors, but instead is funded by subscribers) is that Consumer Reports does not allow any vendor to use its conclusions in advertizing.

Consumer Reports explains its policy about not having its rankings used in advertising in the following quotation.

“We feel this would create an implication among consumers that we have a relationship with the company,” said Charles Daviet, Consumer Reports’ director of survey research. – SFGate

More deceptive advertising from Chevrolet is seen in the following video, which refers to JD Power.

https://www.youtube.com/watch?v=tp6EW-F3OSA

This quotation describes how the people in the advertisement were encouraged to give canned responses.

It’s hard to say how many people know what J.D. Power and Associates is. Just last year, Chevy debuted a commercial where “real people” discovered that Chevy received more J.D. Power Initial Quality Awards than any other car company. The commercial shows consumers responding with low-energy exclamatives like “wow,” “I’m speechless,” and the somehow even more robotic, “That really makes me a lot more confident in Chevy as a brand.”

Later, the A.V. Club interviewed one of the real people (who was in fact a random person with no ties to the company) from the commercial, and found that none of the participants actually knew what J.D. Power was prior to filming. The participant told the A.V. Club, “[The spokesperson] was asking us all these buzzword questions, and then after a while he was like, ‘Do you know what a J.D. Power And Associates Award is?’ And we all go, ‘No.’ Because we all know they don’t really matter because nobody really knows what they are.” Even if you’ve heard of it, its function is still a mystery. – Vox

Continuously Being Used in Advertizements

Another strong similarity between JD Power and Gartner is how it continually pops up in mentions in advertising. We cover in the article How Vendors like Palo Alto Networks Share Illegitimate Gartner Output, because Gartner’s rankings are very much influenced by vendor money. Because Gartner is the best-known name in the IT analyst space, it is constantly used in advertising.

These types of LinkedIn shares are extremely common. The vendor does not disclose their financial relationship with Gartner and presents the rigged report as informing the reader. The vendor then receives contact details in return for what is actually advertising, but advertising created by Gartner. Observe the introduction.

Gartner’s latest research to understand the evolution of cloud-native infrastructure.

The following quote describes the same relationship between JD Power and automotive advertisements.

Over the next decade, J.D. Power became an industry name, but a mention during a well-timed 1984 ad made the market research firm a household name. During Super Bowl 18, Subaru aired the first commercial to boast its J.D. Power rankings. Since then, J.D. Power has been mentioned in more than 350,000 television ads and 2 billion print ads, according to its website. – Vox

How this system works is described by the following quotation.

If you pay attention to all the various automobile ads on TV, you’d probably notice that just about every make and model of vehicle has won some sort of award from some magazine, or that its been “rated highest in its class” by any number of organizations.

But what many people don’t know is that these awards and thumbs-up ratings almost always cost the car companies a pretty penny.

Take as an example Consumers Digest (which has absolutely nothing to do with our kin at Consumer Reports or our benevolent benefactors at Consumers Union). The publication doles out awards for vehicles it deems “Best Buys,” and then charges the manufacturers thousands of dollars just to mention the awards in their advertising.

By its own admittance, Consumers Digest typically charges $35,000 for the first award and $25,000 for each subsequent award. Which means GM, which had 15 vehicles on the Digest’s 2010 Best Buy list, likely paid around $385,000 to Consumers Digest for the right to mention the honors. – Consumer Reports

And they don’t know, because these firms, posing as research firms, don’t tell consumers and of course, the car companies go along with the pretense that no money is exchanged.

Being Paid While Also Being Promoted by Vendors

Like Gartner, JD Power receives an enormous amount of free advertising, as vendors that have already paid these entities to score well in their output, promote the entity as is explained in the following quotation.

The cost of a 30-second commercial at Super Bowl 18 was $368,200. The beauty of J.D. Power’s Super Bowl debut is that the firm likely didn’t pay directly for a cent of the ad — if anything, it probably got paid. At the time, J.D. Power charged companies like Subaru $350 to mention its study for 10 months. Power saw the bigger benefit as publicity, rather than money, but soon the firm upped that number. (Today J.D. Power does not disclose how much an automaker pays to include the company’s findings. – Vox

Proposing a Fake Chinese Wall to Cover for Financial Conflicts of Interest

JD Power tries to cover up its lack of disclosure for who pays them and how much by claiming to have “Chinese Walls” setup between its research and its income operations. This is explained in the following quotation.

Also, companies honored by J.D. Power with top rankings routinely pay it additional sums for extra copies of its foot-high, crystal trophies.

Jamey Power, the firm’s executive vice president and son of founder James David Power, told me he’s not worried about a perceived conflict arising from J.D. Power & Associates taking money from the companies it judges.

“We have walls set up in our organization that separate the different business divisions from these issues,” he said.

But consumer advocates say it’s difficult to maintain the public’s trust if you enjoy a financial relationship with the firms you claim to be independently reviewing.

“Even if the whole thing’s on the up and up, there’s at the very least an appearance of impropriety,” said Ken McEldowney, executive director of Consumer Action in San Francisco.

“Some consumers might wonder if J.D. Power gives awards only to companies that can pay thousands of dollars in licensing fees.” – SFGate

This is eerily reminiscent of Gartner’s ombudsman, which Gartner proposes removes the need of Gartner to disclose how much each vendor pays them.

In fact, every single Gartner publication entirely leaves out the fact that any money is exchanged in any way. But when asked, they invariably point to the Gartner ombudsman, which is a position that is an employee of Gartner as we cover in the article How to Best Understand Gartner’s Ombudsman.

Claims That Money From Vendors Don’t Influence Them

Both JD Power and Gartner state that the money paid to them never influences them. Notice how in the case of JD Power this proposal is contradicted by the following quotation.

Consumers Digest Editor Rich Dzierwa said the fees don’t influence its picks, and each winner is described in its magazine. But its Web site displays only the reviews of cars from auto makers that paid a licensing fee. Winners who don’t pay the magazine are listed on the site as winners, but no further information about those models is given. – WSJ

This statement self falsifies. Only paying winners have a full an actual profile. It should go without saying that no legitimate research would have two ways of presenting the findings. That is one way for entities that paid and a second for those that did not pay. In the case of Gartner, vendors come in and drop out entirely from Magic Quadrants depending upon the year. And it is not as if the vendor stopped doing business. This appears to be something similar. Some years vendors pay and get profiled and other years they don’t and are excluded.

Being able to exclude vendors is important. Without the ability to excluded or reduce the scores of the vendors, both JD Power and Gartner would lose their power over vendors — and realistically, if vendors did not think that they could influence JD Power and Gartner, they would not pay them.

Not Knowing The Method Used to Develop the Ranking or What Research Is

It has been the position of Brightwork Research & Analysis for years that the majority of those that are influenced by Gartner never spend the time to investigate how Gartner functions, and also are not familiar with the rules of research. This also applies to those that are impressed with JD Power’s ratings.

This is illuminated in a similar way about JD Power.

Jeff Galak, an associate professor of marketing at Carnegie Mellon University, says it’s this exact lack of knowledge that makes the award such a trustworthy prize. “People have no idea what it is,” he says. “If you say to somebody, ‘I have an award for the best cow,’ they don’t know how to evaluate a cow. But this award is ostensibly official-sounding.” – Vox

In both cases, Gartner and JD Power tout the fact that their audience believes each does not change their rankings based upon their vendor compensation.

Some biggie at J.D. Power stands by the company’s assertion that the survey and consulting sides of the business are separate entities.

“Core to our success is that our clients believe that the research is entirely independent,” he said. “If they felt that you could buy a better score, then the score would become worthless.” – Consumer Reports

Neither JD Power nor Gartner cares about those that know their research is rigged and part of the advertising for vendors. They only care that the majority not know how their system works. In numerous discussions with Gartner personnel over a decade, I have noticed the exact same phraseology. The masses don’t think that Gartner does not sell its ratings, therefore it must be true.

Conclusion

Both Gartner and JD Power are money-making operations that put research a distant second in comparison to profit maximization.

Neither of the audience for Gartner or JD Power put much effort into understanding the validity of their output. Neither Gartner nor JD Power discloses the fact that they receive a considerable percentage of their funding from vendors. And as we cover in the article How Gartner Helps Vendors Co-Opt New Things They Are Unrelated To, vendors are also very careful never to disclose that they pay or, of course, how much they pay to be profiled. This is as dishonest on the part of the vendors as it is of Gartner.

Ratings of all types seem to be scams frequently. It is clear that companies can buy any auditor they want to sign their books, as we cover in the article Financial Audits as So Corrupt to be Useless.

This quote describes other scam rating entities.

Another scam? The Better Business Bureau. It’s simply a franchise with protected regions. Anyone with the funds can become a BBB and start selling memberships and plaques for your business wall. Even 60 Minutes nailed them ten years ago. They are simply in the business of selling their so-called ratings.

Another one is JCAHO, the private hospital accrediting company. You know how many hospitals have been closed by JCAHO?
Zero.

Hospitals pay THEM for their ratings and inspections. My hospital paid $30,000. – I Am Incorrect

In each case, the story is the same.

  1. The entities or vendors being rated by the pretend research entity pay the pretend research or rating entity.
  2. In each case, the rating entity and the vendors do not disclose the payments to readers/subscribers.
  3. In each case, the vendors use the rating in their advertising.

It is becoming increasingly apparent that buyers are accepting falsified research conclusions from entities that they do not understand, and that this applies across industries. These ratings continue to be effective due to the lack of research or understanding of what is valid versus invalid research on the part of buyers.

The Problem: Thinking that Gartner is Focused on What is True

Gartner is hired by companies who fundamentally don’t understand how Gartner functions. Gartner has virtually no first-hand experience in the technologies that they evaluate and get most of their information from speaking with executives at buyers or executives at vendors and consulting firms. Gartner is also not a research entity. They compare very poorly to real research entities once you dig into the details as we did in the article How Gartner’s Research Compares to Real Research Entities. Gartner serves to direct IT spending to the most expensive solutions as these are the companies that can afford to pay Gartner the most money. Gartner has enormously aggressive internal sales goals that place accuracy far below revenue growth in importance.