The Brightwork Research & Analysis Rating of IT Analysts Like Gartner, Forrester, and More

Last Updated on April 29, 2021 by HostingandOther

Executive Summary

  • After spending many years reviewing IT analysts’ work, I began to wonder if any IT analyst firm was producing non-corrupt work.
  • I review and rate some of the best known IT analysts.

Video Introduction: The Brightwork Research & Analysis Rating of IT Analysts Like Gartner, Forrester, and More

Text Introduction (Skip if You Watched the Video)

In 2013, when I began researching Gartner to prepare a book on the company, I found it shocking. It became apparent that Gartner corrupt and followed no research rules. At this time, I had not spent much time analyzing the other IT analysts. However, as time has passed, and after reviewing more and more material from IT analysts, large problems at all of these IT analysts came into focus. The studies by IT analysts referenced by software vendors were rigged to meet the software vendor’s specification. In this article, I review other IT analysts along with Gartner.

Our References for This Article

If you want to see our references for this article and other related Brightwork articles, see this link.

Lack of Financial Bias Notice: The vast majority of content available on the Internet about Gartner is marketing fiddle-faddle published by vendors who republish reports they paid Gartner to publish, or Magic Quadrants they paid Gartner to score well. The IT industry is petrified of Gartner and only publishes complementary information about them. The article below is very different.

  • First, it is published by a research entity.
  • Second, no one paid for this article to be written, and it is not pretending to inform you while being rigged to sell you software or consulting services as a vendor or consulting firm that shares their ranking in some Gartner report. Unlike nearly every other article you will find from Google on this topic, it has had no input from any company's marketing or sales department. 

IT Analyst #1: Gartner

The Death Star of IT analysts. I call Gartner a “Rubix Cube of Corruption.”

Why a Rubix Cube?

Because every time I analyze Gartner, I find another dimension to their corruption, and it’s difficult to keep “all of the sides of the cube” in one’s head. I have documented many areas of Gartner’s corruption, but they are so numerous that if you ask me to list them all extemporaneously, there is no way I can do so without leaving some out.

Gartner’s best-known output is the Magic Quadrant. This item is entirely fake—Gartner rigs their Magic Quadrants backward from which vendors pay them the most. One of the strongest pieces of evidence for this is that they leave out open source options from their Magic Quadrants, as we cover in the article Can Anyone Make Sense of the ODMS Magic Quadrant? This database MQ leaves out all open source options, even though they are some of the most popular. Gartner’s excuse is that they only include vendors, not products.

IT Analyst #2: IDC

IDC is the second largest IT analyst firm in the world. They cover many topics, even things like projecting the number of laptops manufactured in a given year.

Like the next to be discussed Forrester, specialize in sponsored studies where they write something very complimentary about a vendor or category of software. How Gartner and IDC Help Vendors Co-Opt Things, They are Unrelated To.

IDC has zero integrity and can be hired to write just about anything.

Owned by a Chinese Company?

IDC has an additional problem, and it is curious never to see it discussed.

The combined conglomerate (IDC/IDG, the IDG side controls some of the world’s largest IT media outlets) was acquired by a Chinese company. As every Chinese company eventually reports up to the CCP in China, IDC’s databases are certainly being used for espionage against US and European firms. How the US FTC approved IDC’s acquisitions makes one wonder if US authorities either understand how China is continually performing espionage and IP theft against the US and Europe or if all of the money China distributes throughout Washington DC is providing them with the ROI that they had predicted.

There are plenty of books on the topic of Chinese industrial espionage. This one above is quite good. The Chinese’s position is that the IP of the world is theirs to steal by any means necessary. This is why it is not a great idea to allow any Chinese company to purchase a company like IDC/IDG. One, because IDC has large amounts of information about US and European companies, and two, because the IDG part of the company publishes 8 of the top 20 IT media websites.

And now that company is owned by a company in China, fourth from the bottom in press freedom in the world. There is little doubt that the Chinese government has full access to IDC/IDG’s database and that any information submitted to IDC for analysis is also used for other purposes. China puts anyone who publishes what the Chinese government does not like into prison. And the FTC officials that approved this merger know all of this. Remember that they have access to US intelligence reports which means they know a lot more than I am putting in this article as I am using only public sources of information on the topic. And they still concluded (allow me to paraphrase).

This is a great idea to allow this acquisition to take place.

I have been curious if North Korea, which is roughly equivalent to China in press freedom, would be allowed to acquire Forrester.

Overall, IDC produces very low-quality work, and I can’t recall reading any IDC report without seeing that it was rigged.

IT Analyst #3: Forrester

Forrester used to be a true competitor with Gartner. However, with Gartner’s growth, and continual acquisitions, Forrester is now tiny compared against Gartner in revenues. Gartner produces comparisons like Gartner called Forrester Waves. But, they are not very popular, and it is rare to hear anyone refer to them, while the Gartner Magic Quadrant is constantly referred to by buyers and vendors alike.

As noted in the IDC entry, Forrester is another entity that can be relied upon to publish anything that a vendor pays them to publish. Forrester will absolutely humiliate itself in its sponsored studies, which you can check in the article How to Understand Forrester Fake S/4HANA TCO Study.

How Forrester Likes to Keep it at a 30,000 Feet Level

A final point about Forrester is they function at a very high level. This is even apparent from how they market their services.

Forrester’s marketing is often very “head in the clouds.” I don’t know how long they have marketed this way. But they appear to overlap very strongly with strategy-type consulting. 

I believe most of Forrester’s revenue comes from either scamming companies on their soft serve future predictions, or selling vendor-sponsored studies, that the vendors use as marketing collateral.

IT Analyst #4: G2Crowd

G2Crowd is not technically an IT analyst. However, they are prevalent, and they do cover and rate enterprise software, so I have them included here.

G2Crowd pretends to be an objective user review based website for software comparison. However, they are, in fact, a lead generation site for software vendors. Vendors are paying Their business model to showcase their products.

I only found this out through reading a G2Crowd press release that was not intended for users of G2Crowd and is something I covered in the article The Problem with G2Crowd’s Financial Disclosure.

G2Crowd manipulates the reviews but allows consultants to write reviews, and consultants only are interested in promoting their preferred applications.

There is some decent information available at G2Crowd. Though you are often reading a review from a consultant, the company and the website are completely dishonest, and they are lying to the people who use their website.

IT Analyst #5: Nucleus Research

Nucleus Research is a smaller IT analyst that at one point had a good reputation. However, sometime after 2007, they became corrupt. One strong piece of evidence is how they began estimating the ROI of applications after this time. There has been a huge increase in ROI in their studies/white papers.

The following is a tabular form of the studies/white papers produced by Nucleus Research on CRM that I could find.

Nucleus Research CRM ROI Output

YearTitleDescriptionEstimated ROI
2007ROI Case Study Microsoft Dynamics CRM Roland DGA CorporationA well done case study with a reasonable ROI, and very substantiated support.63%
2011CRM Pays Back $5.60 for Every Dollar SpentIs similar to the 2014 study/white paper.560%
2014CRM Pays Back $8.71 for Every Dollar SpentHigh level study/white paper with no details about the samples used, funding, etc..870%
2019SALESFORCE ESSENTIALS 5P CONSULTINGAppears to be very much a sponsored study by Salesforce. Implementation duration was not at all realistic.998%
2020Microsoft Dynamics 365, on average, Delivers $ 16.97 for Every Dollar SpentThis is a ROI estimate for Dynamics 365, of which Dynamics CRM is one component. 1697%

Something that should immediately come across is that the ROI estimate from the studies after 2007 is extremely high. Let us chart the ROI estimates.

Here is a question. How did Nucleus Research go from an estimate of 63% ROI to 1697% CRM ROI in the space of 13 years? Nucleus Research noted in their later studies the software in the CRM category was getting better. However, this massive increase strains credulity. 

IT Analyst #6: G2Crowd

Reviewing A Table of the IT Analysts

Now that we have described each entry let us review a table where some characteristics are applied.

How The IT Analysts Stack Up

CompanyCorruption LevelDid Good Work At One Time?Owned by a Company in a Country with No Freedom of Speech?
GartnerTotalNoNo
IDCHighNoYes
ForresterHighNoNo
G2CrowdHighNoNo
Nucleus ResearchHighYesNo
Panorama ConsultingHighNoNo
Constellation ResearchModerateYesNo

IT Analyst #7: Constellation Research

Constellation Research was founded by Ray Wang, who is a bit of a conference celebrity due to this speaking ability. However, when it comes to Constellation Research’s research, it follows the standard model of trying to keep vendor relationships and maximize financial opportunities rather than telling the truth and following a high integrity research model. If we look at one specific area of coverage, say of the SAP by Constellation Research, I find that it tends to repeat SAP’s assertions, even when the claims are false, and comes across as highly subordinate to SAP. I understand that SAP has very persuasive relationship managers, but they don’t know anything about the reality of SAP’s products, nor do they care what is true.

Ray Wang began his career at Gartner, a problem, as Gartner (as we covered) is highly corrupt.

I would place Constellation Research’s output significantly above that of Gartner or Forrester. And Constellation Research should be commended for not publishing the completely transparent “pay for study/marketing white paper,” as do Gartner, IDC, Forrester, and Nucleus Research. However, when facing down the mega-vendors, Constellation appears to buckle and give the politically correct and low-risk analysis.

Conclusion

This analysis concludes that none of these IT analysts is uncorrupt. I could add more entities into this study, but the results end up being the same no matter who I look. This means that these companies produce unreliable information to IT decision-makers. It is a damning statement, but that is where the evidence points.

The Broader Problem with Private Research

This says something broader about how poor to non-existent real research entities are in the private space. Private research mostly consists of pretending to be a research entity and then functioning as a marketing front end for the industry you cover. There is little difference between any of these companies and JD Power and Associates, which performs the same function but for consumer products.

These entities do not follow research rules and often write about how they know nothing and can be paid to present pretty much any view. And Forrester and IDC play the same game.

This is why profit-maximizing research entities and the entities on this list are very much about profit maximization, which can never lead to good research. It is also why private research has such poor outcomes.

Brightwork Research & Analysis is a research entity. And we have tried to get funding for real research in a variety of technology areas. We have found there is no interest in this research, or that is where the conclusion is not predetermined. This is why nearly all of the research on the Brightwork website is self-funded. What is generally called research in the IT space is predetermined outcome reports used as marketing collateral for vendors and consulting firms? People who reach out to me are often sales directors who want to know if they can have a written report that helps them in upcoming sales cycles. That is where the market really is.

Are Only These Companies to Blame?

Here is a problem with only blaming IT analysts. Vendors often point out that IT analysts are corrupt (at least in private). However, how can IT analysts make money without in some way selling out? The vendors want positive coverage and eventually promote IT analysts to provide that coverage. Over time, even though the IT analyst may have started with good intentions (although not all, G2Crowd began as exactly what it is today), eventually, the market forces push it towards selling its brand and ratings.