Gartner

The Brightwork Research & Analysis Rating of IT Analysts Like Gartner, Forrester, and More

Executive Summary

  • After spending many years reviewing IT analysts’ work, I began to wonder if any IT analyst firm was producing non-corrupt work.
  • I review and rate some of the best known IT analysts.

Introduction

Back in 2013, when we began researching Gartner at Brightwork Research & Analysis, and what we found was shocking. Gartner was clearly corrupt and followed no research rules. At this time, we had not spent much time analyzing the other IT analysts. However, as time has passed, after having reviewed more and more material from IT analysts large problems at all of these IT analysts came into focus. The studies by IT analysts referenced by software vendors were rigged to meet the software vendor’s specification. You will find our observations of the work of some of the best-known IT analysts.

Our References for This Article

If you want to see our references for this article and other related Brightwork articles, see this link.

Lack of Financial Bias Notice: The vast majority of content available on the Internet about Gartner is marketing fiddle-faddle published by vendors who republish reports they paid Gartner to publish, or Magic Quadrants they paid Gartner to score well. The IT industry is petrified of Gartner and only publishes complementary information about them. The article below is very different.

  • First, it is published by a research entity.
  • Second, no one paid for this article to be written, and it is not pretending to inform you while being rigged to sell you software or consulting services as a vendor or consulting firm that shares their ranking in some Gartner report. Unlike nearly every other article you will find from Google on this topic, it has had no input from any company's marketing or sales department.

IT Analyst #1: Gartner

The Death Star of IT analysts. I call Gartner a Rubix Cube of corruption. Why? Because every time I analyze Gartner, I find another dimension to their corruption. I have documented many areas of Gartner’s corruption, but they are so numerous that if you ask me to list them all extemporaneously, there is no way I can do so without leaving some out.

Gartner’s best-known output is the Magic Quadrant. This item is entirely fake—Gartner rigs their Magic Quadrants backward from which vendors pay them the most. One of the strongest pieces of evidence for this is that they leave out open source options from their Magic Quadrants, as we cover in the article Can Anyone Make Sense of the ODMS Magic Quadrant? This database MQ leaves out all open source options, even though they are some of the most popular. Gartner’s excuse is that they only include vendors, not products.

Gartner’s advice is of little value because Gartner’s analysts do not actually know that much and spend most of their time talking to executives. And second, because they are corrupt and dishonest.

IT Analyst #2: IDC

IDC is the second largest IT analyst firm in the world. They cover many topics, even things like projecting the number of laptops manufactured in a given year.

Like the next to be discussed Forrester, specialize in sponsored studies where they write something very complimentary about a vendor or category of software. How Gartner and IDC Help Vendors Co-Opt Things, They are Unrelated To.

IDC has no integrity and can be hired to write just about anything.

Owned by a Chinese Company?

IDC has an additional problem, and it is curious never to see it discussed.

The combined conglomerate (IDC/IDG, the IDG side controls some of the world’s largest IT media outlets) was acquired by a Chinese company. As every Chinese company eventually reports up to the CCP in China, IDC’s databases are certainly being used for espionage against US firms. IDC produces garbage, and I can’t recall reading any IDC report without seeing that it was rigged.

IT Analyst #3: Forrester

Forrester used to be a true competitor with Gartner. However, with Gartner’s growth, and continual acquisitions, Forrester is now tiny compared against Gartner in revenues. Gartner produces comparisons like Gartner called Forrester Waves. But, they are not very popular, and it is rare to hear anyone refer to them, while the Gartner Magic Quadrant is constantly referred to.

As noted in the IDC entry, Forrester is another entity that can be relied upon to publish anything that a vendor pays them to publish. Forrester will absolutely humiliate itself in its sponsored studies, which you can check in the article How to Understand Forrester Fake S/4HANA TCO Study.

How Forrester Likes to Keep it at a 30,000 Feet Level

A final point about Forrester is they function at a very high level. This is even apparent from how they market their services.

Forrester’s marketing is often very head in the clouds. I don’t know how long they have marketed this way. But they appear to overlap very strongly with strategy type consulting. 

I believe most of Forrester’s revenue comes from either scamming companies on their soft serve future predictions, or selling vendor-sponsored studies, that the vendors use as marketing collateral.

IT Analyst #4: G2Crowd

G2Crowd is not technically an IT analyst. However, they are prevalent, and they do cover and rate enterprise software, so I have them included here.

G2Crowd pretends to be an objective user review based website for software comparison. However, they are, in fact, a lead generation site for software vendors. Vendors are paying Their business model to showcase their products.

I only found this out through reading a G2Crowd press release that was not intended for users of G2Crowd and is something I covered in the article The Problem with G2Crowd’s Financial Disclosure.

G2Crowd manipulates the reviews but allows consultants to write reviews, and consultants only are interested in promoting their preferred applications.

There is some decent information available at G2Crowd. Though you are often reading a review from a consultant, the company and the website are completely dishonest, and they are lying to the people who use their website.

IT Analyst #5: Nucleus Research

Nucleus Research is a smaller IT analyst that at one point had a good reputation. However, sometime after 2007, they became corrupt. One strong piece of evidence is how they began estimating the ROI of applications after this time. There has been a huge increase in ROI in their studies/white papers.

The following is a tabular form of the studies/white papers produced by Nucleus Research on CRM that I could find.

Nucleus Research CRM ROI Output

YearTitleDescriptionEstimated ROI
2007ROI Case Study Microsoft Dynamics CRM Roland DGA CorporationA well done case study with a reasonable ROI, and very substantiated support.63%
2011CRM Pays Back $5.60 for Every Dollar SpentIs similar to the 2014 study/white paper.560%
2014CRM Pays Back $8.71 for Every Dollar SpentHigh level study/white paper with no details about the samples used, funding, etc..870%
2019SALESFORCE ESSENTIALS 5P CONSULTINGAppears to be very much a sponsored study by Salesforce. Implementation duration was not at all realistic.998%
2020Microsoft Dynamics 365, on average, Delivers $ 16.97 for Every Dollar SpentThis is a ROI estimate for Dynamics 365, of which Dynamics CRM is one component.1697%

Something that should immediately come across is that the ROI estimate from the studies after 2007 is extremely high. Let us chart the ROI estimates.

Here is a question. How did Nucleus Research go from an estimate of 63% ROI to 1697% CRM ROI in the space of 13 years?

IT Analyst #6: G2Crowd

Reviewing A Table of the IT Analysts

Now that we have described each entry let us review a table where some characteristics are applied.

How The IT Analysts Stack Up

CompanyCorruption LevelDid Good Work At One Time?Owned by a Company in a Country with No Freedom of Speech?
GartnerTotalNoNo
IDCHighNoYes
ForresterHighNoNo
G2CrowdHighNoNo
Nucleus ResearchHighYesNo
Panorama ConsultingHighNoNo
Constellation ResearchModerateYesNo

IT Analyst #7: Constellation Research

Constellation Research was founded by Ray Wang, who is a bit of a conference celebrity due to this speaking ability. However, when it comes to the research produced by Constellation Research it follows the standard model of trying to keep vendor relationships and maximizing financial opportunities rather than telling the truth and following a high integrity research model. The coverage of SAP by Constellation Research tends to repeat SAP’s assertions, even when the claims are false, and comes across as highly subordinate to SAP. I understand that SAP has very persuasive relationship managers, but they don’t know anything about the reality of SAP’s products, which is why Brightwork Research & Analysis does not spend time speaking with any vendor’s representatives.

Ray Wang began his career at Gartner, which is a problem, as Gartner (as we covered) is corrupt.

I would place Constellation Research’s output significantly above that of Gartner or Forrester. And Constellation Research should be commended for not publishing the completely transparent “pay for study/marketing white paper” as do Gartner, IDC, Forrester, and Nucleus Research. However, when it comes to facing down the mega-vendors, Constellation appears to buckle and give the politically correct and low-risk analysis.

Conclusion

My conclusion is there is not a single IT analyst that has not become corrupt. This means that these companies produce unreliable information to IT decision-makers. It is a huge and damning statement, but that is where the evidence points.

The Broader Problem with Private Research

This says something broader about how poor to non-existent real research entities are in the private space. Private research mostly consists of pretending to be a research entity and then functioning as a marketing front end for the industry you cover. There is little difference between any of these companies and JD Power and Associates, which performs the same function but for consumer products.

These entities do not follow research rules and often write about topics of which they know nothing and can be paid to present pretty much any view. And Forrester and IDC play the same game.

This is why profit-maximizing research entities and the entities on this list are very much about profit maximization, which can never lead to good research. It is also why private research has such poor outcomes.

Brightwork Research & Analysis is a research entity. And we have tried to get funding for real research in a variety of technology areas. And what we have found is there is no interest in real research, or that is where the conclusion is not predetermined. This is why nearly all of the research on the website is self-funded. What is generally called research in IT space is predetermined outcome reports that can be used as marketing collateral for vendors and consulting firms? People who reach out to me are often hyperventilating sales directors who want to know if they can have a written report that helps them in upcoming sales cycles. That is where the market really is.

Are Only These Companies to Blame?

Here is a problem with only blaming IT analysts. Vendors often point out that IT analysts are corrupt (at least in private). However, how can IT analysts make money without in some way selling out? The vendors are themselves corrupt, and they, in turn, corrupt the IT analysts. As with slavery, morality applies to both sides of the transaction, not only the person buying the slave but also the person selling the slave.

And the one who corrupts an entity is as guilty of corruption as the entity that is corrupted.