The Problems With Peer Review Research

Executive Summary

• Peer review journals are typically held up as the gold standard of research while having many problems that tend to be overlooked.

Introduction

When some readers have a serious disagreement with an article at Brightwork Research & Analysis, sometimes the opposing and often offended individual will state that the articles at Brightwork are not “peer-reviewed.” People often make this claim with no academic background in research and typically no understanding of research and those who have never worked for or generally read peer-reviewed journals.

These blind spots mean that the persons overstate the validity of the peer review system and use the lack of peer review against articles that they simply disagree with.

In this article, I will highlight problems with the peer-review process.

Our References for This Article

If you want to see our references for this article and other related Brightwork articles, see this link.

Problem #1: Who Funds Large Scientific Studies?

The first part left out of the peer review discussion is who funds peer-reviewed studies. It usually is drug companies or medical device companies in the medical area. This is explained in the following quotation.

The bigger and the higher quality a scientific study has, the more expensive it is. This means that most big high quality studies are carried out by pharmaceutical companies. Obviously this is a problem because the companies have a vested interest in making their products look good. And when companies carry out studies, they don’t show their drugs in the best light that don’t show their drugs in the best light they will usually try to bury the data. This contributes to a problem known as publication bias. When a publication when publication bias, what publication bias means is that studies that show good effect are much more likely to get published and studies that show no effect. This is both because the people who conducted the study are more likely to push for it to be published. And because journals are more likely to accept studies that show benefit, because those studies get much more attention than studies that don’t show a benefit. And the more attention a journal gets, the more advertising money it gets. So one thing to be aware of before you start searching for scientific studies in a field is that the studies you can find on a topic often aren’t all the studies, you are most likely to find the studies that show the strongest effect. The effect of an intervention in the published literature is pretty much always bigger than the effects subsequently seen in the real world. – Why Most of What You Know About Covid is Wrong

Publication bias is a type of cherry-picking. And it dramatically benefits those companies with financial resources. A pharmaceutical company can run many studies and report those that support their commercial interests. If one runs seven studies, one of them will show a benefit just by luck. However, when the study is published, the pharmaceutical company or the individuals funded do not declare that one of only seven studies for that drug showed a benefit.

Secondly, many people in the peer review process also have financial conflicts and seek to protect and keep their relationships with pharmaceutical companies in good standing.

Problem #2: The Problem of Lack of Incentive on the Part of the Peer Reviewers

Peer review provides a sort of stamp of approval although it is questionable how much that stamp is worth. Basically peer review means that someone who was considered an expert on the subject of the article, but who wasn’t personally involved with it anyway in any way. Reach through and determines if it is sensible and worth publishing. Generally, the position of peer review of a peer reviewer is an unpaid position and the person engaging in peer review does it in his or her spare time, he or she might spend an hour or so going through the article before deciding whether it deserves to be published or not. Clearly, this is not a very high bar. Even the most respected journals have published plenty of bad studies containing manipulated to fake data, because they didn’t put much effort into making sure the data was correct. The early part of the COVID pandemic saw a ton of bad studies which had to be retracted just a few weeks or months after publication, because because the data weren’t properly fact checked before publication. – Why Most of What You Know About Covid is Wrong

The time estimation to review an article or study is far higher in this quotation.

It takes time – at least seven to eight hours per paper done properly, with no remuneration or recognition for the reviewer and hence rarely regarded as a priority in a busy academic schedule. As a result, scientific rigour can be lost when reviews become fast-tracked. – The Conversation

When I have been a peer reviewer, I have spent around 3 hours reviewing the article. However, the time-consuming part comes in when the article author responds to my comments, and I have to defend my initial observations.

When doing this, and in light of adding comments to a blog post, I have wondered why all of this has to be carried out in secret, without the readers seeing the differing views. Even YouTube has comment ability. This realization is dawning on many people and is explained in the following quotation.

Post-publication peer review in its most validated form, involves a journal such as Frontiers asking academics to perform a published interactive dialogue with authors during the review process, giving a level of accountability and responsibility.

Other journals such as Faculty of 1000 Research, Copernicus and PLOS ONE publish papers with minimal evaluation. This shifts the focus towards post-publication peer review – authors, reviewers and readers critique and comment on the paper to judge its scientific merit in the public domain. – The Conversation

Pub Peer simply allows registered commenters to analyze each article, so their credentials are known. Each person must be fully identified, so there is no anonymous commentary. 

Another response to the article. Another good thing about this is it does not keep the peers secret.

Here is a response to an objection at Peer Pub.

My Observation of This More Public Presentation of Disagreements and Article Feedback

I like this approach much better than the peer-reviewed journal approach. I do not have experience with this type of publishing, as most of my experience in reading research is going to a research search engine and finding articles that have been published in peer-reviewed journals.

I found the following positive features to Peer Pub.

1. Opening the Idea of Nonfinality: After reading several articles on Pub Peer, the interactive nature gave a much less final feel.
2. The Public Nature of the Discussion: I prefer the interaction and debate on articles to be public. The fact that the critiques of an article are not published gives a false impression of authority regarding an article.
3. More Interested Commenters: Most people who work as peer reviewers view their responsibilities as more of a chore. The commenters are far more voluntary than journal-selected peers in an internet forum-type publishing platform. This also makes them more motivated to put effort into the review or their comments.
4. The Ability of Commenters to Benefit From Their Efforts: As their comments are public, there is much more motivation for the commenters to showcase their knowledge and receive some type of benefit for verifying the article or study.
5. Increased Input: There is a likelihood of many more commenters than would be the number of peers involved in a peer review.
6. Speed: The process is much faster. The article can be published, and commenters can add comments while the article is already visible.
7. Reduced Control and Censorship: The lowered motivation of the publishing entity to police the articles.

This final point brings up a related problem: journals often force the author or authors to change some things in the article to gain approval. This reduces the work’s authenticity and is a long-term issue for both academics and peer-reviewed publishing. I once knew a Ph.D. candidate who completely changed their dissertation because they had been in school for so long and had to obtain their Ph.D. to get employed. The dissertation was changed by her to “fluff” their advisor (by referring to the advisor’s work) and by fitting in with the status quo.

Problem #3: Most Studies Eventually Get Published in Some Journal

If the peer review or at one journal says no to a scientific study, the researchers will generally moved on to another less prestigious journal. And we’ll keep going like that until they can get the study published. There are so many journals that everything gets published somewhere in the end, no matter how poor the quality is, the whole system of peer review builds on trust. The guiding principle is that the eye is the idea and it bad studies will be caught out over the long term because when other people try to replicate the studies, they won’t be able to. There are two big problems with this line of thinking. The first is that scientific studies are expensive. So they often don’t get replicated, especially if they’re big studies of drugs. For the most part, no one but the drug companies themselves has the cash resources to do a follow up study to make sure the results are reliable. And if the drug company has done one study, which shows a good effect, it won’t want to risk doing a second study that might show a weaker effect. The second problem is that follow up studies aren’t exciting. Being first is exciting and generates a lot of media attention being second is boring, and no one cares about the people who read did a study and determine what the results actually that the results actually held up to scrutiny. – Why Most of What You Know About Covid is Wrong

Again, this plays to the advantage of companies with financial resources, as they can afford to cascade through the journals until they can finally publish the study. The fact that 3 or 4 journals rejected the study before being published is left out of the publication notes.

Problem #4: The Surrogate Endpoint in a Study – Cherry Picking the Time Point At Which the Scientific Study Ends

Surrogate endpoints are an alternate endpoint and stand in for the thing that actually matters to patients. An example of a surrogate endpoint is looking at whether a drug lowers cholesterol instead of looking at the thing that actually matters overall mortality. By using a surrogate endpoint, the researchers can claim that the drug is successful when they have in fact showed no such thing. Another example of a surrogate endpoint that is frequently used in studies of cholesterol lowering drugs is looking at cardiovascular mortality instead of overall mortality.

The alternative to a per protocol analysis is an intention to treat analysis and this analysis everyone who started the study is included in the final result, regardless of whether they dropped out or not, this gives a much more accurate understanding of what results can be expected when a patient starts a treatment and should be standard for all scientific studies in health and medicine.

Pharmaceutical companies engage in a lot of trickery when it comes to the method and in the interpretation of the results, but I think it’s uncommon for them to engage in outright lying when it comes to the hard data presented in the result tables. There is, however, one blatant manipulation of the results that happens frequently. I’m talking about cherry picking the time point at which a scientific study is ended. This can happen when researchers are allowed to check the results of their study and when it is still ongoing. If the results are promising, they will often choose to stop a study at that point in claim that the results were so good that it would have been unethical to go on. Because of a statistical phenomena known as regression to the mean, basically, the longer a scientific study goes on, and the more data points that end up being gathered, the closer the result of the study is to the real result. Early on in the study, the results will often swing wildly just due to statistical chance. So studies will tend to show bigger effects early on and smaller effects towards the end. This problem is compounded by the fact that if a study at an early endpoint, at an early point shows a negative result or a neutral result or even a result that is positive but not positive enough, the researchers will usually continue the study in hopes of getting a better result. But the moment the result goes above a certain point they stopped the study and claimed maximum benefit from their treatment. That is how the time point in which a study is stopped and stopped ends up being cherry picked. This is why the plan length of a study should always be posted in advance on clinical trials.gov. And why researchers should always stick to the plan length and never look at the results until the study has gone on for the plan length. If a study is stopped early at a time point of the researchers choosing the results are not statistically sound no matter what the P values may show. Never trust the results of a study that stopped early. – Why Most of What You Know About Covid is Wrong

Changing the endpoint is a constant feature of pharmaceutical studies. It was used to get the dangerous drug Remdesivir through the FDA and has been used for decades to get chemotherapy drugs approved that focus on the endpoint of shrinking tumors while ignoring the overall health outcomes for patients.

Peer review does not catch cherry-picking when the study ends. The peer reviewers only see what is published in the study, not what the submitter leaves out.

And this leads to the next problem.

I will provide an example.

Problem #5: The Peer Reviewers Do Not Know What Was Left Out

When drug companies perform studies, they only submit the ones they want to the journal or the FDA. That means that all drug companies have large numbers of studies that will never see the light of day, as the study is only worth publishing if it can show a benefit or a faux benefit that can be profited from. Therefore, if a drug company runs four studies, and three have bad results that show no effectiveness, but one good study that does show effectiveness, they only submit one good study for publication. However, even within the studies where effectiveness is established, the drug company can simply remove data points that make the treatment look worse. This is shown in the following example.

Example of How Little Peer Reviewers Know About the Study

This is an analysis of a study into Remdesivir. The manufacturer, called Gilead Sciences, paid for it. Most of the people involved in the study have financial conflicts with Gilead Sciences, something The New England Journal of Medicine had no problem with.

In this study, which Brad Geyer is critiquing (see the whole critique here), you can see from the underlined area that the data from 8 of the 61 patients “could not be analyzed.” 

Now here is a question. Could this data not be analyzed because.

• a.) The data was lost.
• b.) The patients/test subjects had negative reactions?

In other studies on this drug, it had very problematic side effects. However, in this study, 8/61 = 13% of the data is lost. This would be a perfect place to hide people who had their organs damaged by the drugs, which this drug is known to do. 

The reality is, those that peer-reviewed the study, who also may have had financial connections to Gilead Sciences, adding on to the overall journal that has enormous financial biases in favor of pharmaceutical companies that buys reprints of published articles from NEJM, have no idea if the data was stated as lost, when in fact it was just not included. Again, the peer reviewers do not participate in the study and have to decide whether to approve of the study being published based upon what is included in the report. These obviously deceptive studies are published in the top journals all the time. And they are respected because they are “peer-reviewed.” 

Secondly, it makes sense that NEJM only includes peer reviewers who are “in the club” and won’t question these studies too much. Ultimately, NEJM wants the revenues from pharmaceutical companies to keep flowing. 

Problem #6: Only 50% of Scientific Studies Can be Replicated

One thing to be aware of is that a large share of studies cannot be successfully replicated. Some studies have found that more than 50% of research results cannot be replicated. That is, in spite of a cut off, which is supposed to cause this to only happened 5% of the time. How can that be?

I think the three main reasons are publication bias, vested interests that do what they can do to manipulate studies and the inappropriate use of the 5% p value cut off. That is why we should never put too much trust in a result that has not been replicated. We discussed statistical significance a lot now, but that isn’t really what matters to patients, what matters. And what patients care about is clinical significance, ie if they take a drug will it and have a meaningful impact for them. Clinical significance is closely tied to the concept of absolute risk and relative risk. Let’s say we have a drug that decreases your five year risk of having a heart attack from 2%. From point 2% 2.1%. Now the absolute risk reduction when you take this drug is 1% over five years. Not very impressive, right? Would you think it was worth taking that drug? Probably not? What if I told you that the same drug actually decreased your risk of heart attack by 50%? Now you definitely want to take the drug right? How can a drug only decreased risk by point 1%, and yet at the same time, decrease risk by 50%. Because the risk reduction depends on if we’re looking at absolute risk or relative risk. Although our imaginary drug only causes a point 1% reduction in absolute risk, it causes a 50% reduction in relative risk. drug companies will generally focus on the relative risk when discussing the benefits of their drugs, because it makes a benefit sound more impressive and absolute risk when discussing harms because it makes a harm sound small. When you look at an advertisement for a drug always look at the fine print, or they’re talking about absolute risk or relative risk. – Why Most of What You Know About Covid is Wrong

The idea that 50% of peer-reviewed studies cannot be replicated makes one question the validity of the peer review to catch errors. This is an enormous problem and yet is barely discussed.

One of the problems is the overemphasis on statistical significance, which is an arbitrary cut-off.

Several methodologists have pointed out [9–11] that the high rate of nonreplication (lack of confirmation) of research discoveries is a consequence of the convenient, yet ill-founded strategy of claiming conclusive research findings solely on the basis of a single study assessed by formal statistical significance, typically for a p-value less than 0.05. Research is not most appropriately represented and summarized by p-values, but, unfortunately, there is a widespread notion that medical research articles should be interpreted based only on p-values. – Ioannidis

As has been covered in earlier problem explanations, peer reviewers cannot expect to catch items that aren’t even published in the study.

And this problem is expounded upon in the following quotation.

In many ways, journals don’t even pretend to ensure the validity of scientific findings. If that were their primary goal, journal policies would require authors to share their data and analysis code with peer reviewers, and would ask reviewers to double-check results. In practice, reviewers can only judge the science based on what’s reported in the writeup, and they usually can’t see the details of the process that led to the findings. – Wired

Therefore, the quality checking is much less than generally presented by the journals and much less than commonly thought by the readers.

Problem #7: The Lack of Standardization for What Peer Review Is

Peer review can mean a lot of different ways of accomplishing the task. Generally, few people who have not worked either for a peer-reviewing journal or as a peer reviewer realize how arbitrary the peer-review process is run in journals.

This is expressed very well in the following quotation.

The editor looks at the title of the paper and sends it to two friends whom the editor thinks know something about the subject. If both advise publication the editor sends it to the printers. If both advise against publication the editor rejects the paper. If the reviewers disagree the editor sends it to a third reviewer and does whatever he or she advises. This pastiche—which is not far from systems I have seen used—is little better than tossing a coin, because the level of agreement between reviewers on whether a paper should be published is little better than you’d expect by chance.

That is why Robbie Fox, the great 20th century editor of the Lancet, who was no admirer of peer review, wondered whether anybody would notice if he were to swap the piles marked `publish’ and `reject’. He also joked that the Lancet had a system of throwing a pile of papers down the stairs and publishing those that reached the bottom. When I was editor of the BMJ I was challenged by two of the cleverest researchers in Britain to publish an issue of the journal comprised only of papers that had failed peer review and see if anybody noticed. I wrote back `How do you know I haven’t already done it?’ – NCBI

Peer Rings?

Peer rings have been found where biased peers are reviewing their friends’ papers. This is explained in the following quotation.

In some cases, journals encourage authors to suggest reviewers’ names. However, this feature, initially introduced to help the editors, has been unfortunately misused to create peer review rings, where the suggested reviewers were accomplices of the authors, or even the authors themselves with secret accounts. – The New Republic

Problem #8: The Overestimation of Peer Review Published Articles

This quote describes how many people turn their brains off in critical thinking when they learn a paper or study was published in a peer-reviewed journal.

Before this ideal system is put in place, there’s one thing we can do immediately to make peer review better. We need to adjust our expectations about what peer review does. Right now, many people think peer review means, “This paper is great and trustworthy!” In reality, it should mean something like, “A few scientists have looked at this paper and didn’t find anything wrong with it, but that doesn’t mean you should take it as gospel. Only time will tell.” – Vox

And this quote illustrates that cracks are beginning to show in the facade of peer-reviewed journals.

Scientists allowed these myths to spread because it was convenient for us. Peer-reviewed journals came into existence largely to keep government regulators off our backs. Scientists believe that we are the best judges of the validity of each other’s work. That’s very likely true, but it’s a huge leap from that to “peer-reviewed journals publish only good science.” The most selective journals still allow flawed studies—even really terribly flawed ones—to be published all the time. Earlier this month, for instance, the journal Proceedings of the National Academy of Sciences put out a paper claiming that mandated face coverings are “the determinant in shaping the trends of the pandemic.” PNAS is a very prestigious journal, and their website claims that they are an “authoritative source” that works “to publish only the highest quality scientific research.” However, this paper was quickly and thoroughly criticized on social media; by last Thursday, 45 researchers had signed a letter formally calling for its retraction. The idea that journals have a special way to tell what’s good science and what’s bad has always been an illusion. – Wired

Problem #9: The Establishment, Anti-Controversy and Politically Correct Orientation of Journals

Many of the quotes in this article have focused on scientific research. However, peer-review journals are found in every scholarly discipline.

When I participated as a peer-reviewing myself for a journal, I found a strong bias from other reviews that worked for software vendors to understate the problems on implementing different technologies. Overall, I believe that software vendors and consulting firms are far too influential in setting the narrative for software capabilities, outcomes, and implementation realities — and this is to simply increase the sales of software and services. They exert enormous pressure on IT media and journals in the field, as they are the primary funding sources.

Many of the articles published at Brightwork Research & Analysis question powerful entities or are otherwise politically incorrect. There would be no way many of the articles this website published in a peer-reviewed journal because the topics are too controversial, too contrary to considerable commercial interest, or just too un-PC, and they threaten the revenue sources of any journal, which relies upon these powerful and corrupt software vendors and software consulting firms.

Pharmaceutical companies are well known to have rigged the medical/science peer-reviewed journals, and peer-reviewed journals do not like to offend advertisers and disdain controversial topics. However this question of commercial influence is not limited to journals that publish on drug studies.

Getting Along By Going Along – And Falling into the Consensus View

Therefore, another problem with peer review is that journals are essentially part of the establishment. Demanding that all research be peer-reviewed pushes the publication of information through a censoring process, where specific unfashionable observations or statements are “rounded off” to appeal to the broadest possible demographic and keep from offending potent entities.

This is illustrated in the following quote from the article Why Has Innovation Disappeared From Medical Research?

Note: Pierre Kory’s “The War on Ivermectin” provides the most concise but comprehensive summary of the systemic corruption within our medical journals that sustained the pandemic narrative. Its story provides a valuable lesson because the same principles at work throughout COVID-19 were at play long before it started. For example, “peer review,” which we place so much weight on, typically only signifies that your findings conform to the narrative of your peers (Malcolm Kendrick provides the most potent critique I have come across on the merits of peer review here).

Because of these biases, you often must read between the lines when reviewing a paper. For example, it is extremely common that if a study obtains results that threaten the dominant narrative, the introduction and conclusion of the article will conceal those results and instead argue in favor of the narrative so the paper can make it through to publication (most people only read the summaries of a study without dissecting it).

Pollack was the one who first introduced me to the idea the grant system had removed the innovation within our scientific apparatus. Since I greatly respect his integrity and he has a significant degree of firsthand experience with this issue, I asked him to provide a hypothetical story to illustrate the problem:

Suppose you’re inclined to believe the earth is round, while everyone around you knows it’s flat. You’ve seen satellite photos showing curvature. You’ve flown westward, eventually returning to the same point where you started. With such “preliminary data” you’re convinced that flat-earth paradigm may be erroneous, and because of the subject’s importance, you’re compelled to explore further.

To do so, you need funding. So you prepare a grant proposal to a national funding organization. The organization’s gate-keeper receives your proposal. Noting its revolutionary character, the official recruits the shape-of-the-earth field’s top experts, in order to determine whether the proposal is flaky or serious.

Who are those experts? Inevitably, they’re the major proponents of the prevailing view: the flat-earth theory. Your proposal gets evaluated by the very people you’re challenging. Their response can be imagined. Who likes to get dethroned?

By inviting evaluation by the very people under challenge, the grant system ensures that revolutionary proposals will rarely receive funding. It’s effectively a dead end.

The upshot?

Few revolutions. The status quo prevails — endlessly. Public-relations folks may create the impression that science is moving at a rapid clip, but it’s mainly incremental. The truly major scientific advances come from the giant steps that we call scientific revolutions.

The scientific enterprise is assuredly active. But the dearth of modern scientific revolutions implies that something has gone seriously awry. Science should be producing major revolutions, as was the case in the early 20^th century when physics revolutions were coming practically annually. Think Einstein, Planck, Bohr, etc. That kind of progress has all but vanished, and I argue that a significant culprit is the very system supposedly designed to foster revolutions — the granting system.

The problems facing our world today are immense. Overcoming those problems will require never-before-conceived technologies. Inevitably, scientific revolutions produce those technologies: Who would have thought the discovery of semi-conduction would produce today’s laptops? Some attempts to spawn revolutionary science have surfaced, but it may be time to consider the major culprits in the crime: the federal granting agencies. By (inadvertent) design, those agencies are inhibiting scientific revolutions. That design needs to change.

Problem #10: The Lack of Scientific Evidence That Peer-Reviewed Journals Lead to Reliable Outcomes

To understand this topic, it is necessary to delve into how peer-reviewed journals got their start in the first place.

Understanding the History of Peer Review

Peer review journals and those that support peer review seem to assume a sturdy process for developing peer review. This is an inaccurate assumption, as the following quote explains the history of the development of peer review as being more of a holistic process.

Once the study is finished, the researchers will usually try to get it published in a peer reviewed journal. The first scientists back when modern science was being invented in the 1600s mostly wrote books in which they describe what they had done and what results they had achieved. Then after a while, scientific society started to pop up and started to produce journals. Gradually, science moved from books to journal articles. In the 1700s, a journal started to incorporate the concept of peer review as a means to ensure quality. As you can see, journals are an artifact of history there is actually no technical reason why studies need to be published in journals, particularly in a time when most reading is done on digital devices. It is possible that journals will disappear with time to be replaced by online science databases. – Why Most of What You Know About Covid is Wrong

This is why it is surprising to many people that little research has gone into demonstrating that the peer review process is effective and leads to accurate outcomes, which is illuminated in the following quotation.

It might seem surprising to those outside the academic world, but until now there has been little empirical investigation on the institution that approves and rejects all scientific claims.

Some scholars even complain that peer review itself has not been scientifically validated. The main reason behind the lack of empirical studies on peer review is the difficulty in accessing data. In fact, peer review data is considered very sensitive, and it is very seldom released for scrutiny, even in an anonymous form. – The New Republic

Inserting Multiple Quality Checking Steps

One of the significant problems with peer review is that it eliminates other quality-checking methods narrow-mindedly. This issue is explained in the following quotation.

We think that’s closer to the solution. Science would probably be better off if researchers checked the quality and accuracy of their work in a multi-step process with redundancies built in to weed out errors and bad science. The internet makes that much easier. Traditional peer review would be just one check; pre-print commenting, post-publication peer review, and, wherever possible, highly skilled journal editors would be others. – Vox

This website catalogs the scientific studies on Covid. Some of the studies are in peer-reviewed journals, and some are not. The internet provides a way of seeing a comprehensive view and an updated view that was not possible when journals first began to spring up in the 1700s. Journals are, in a way, a feature of a different time when information was more controlled. 

Many medical authorities are lying publicly about the effectiveness and safety of many drugs and treatments, and they pretend that other people can’t check the evidence. This is particularly problematic with MDs with pharmaceutical connections or funding from the NIH brought onto news programs of major media outlets but without declaring or being made to declare their financial conflicts.

Problem #11: Promotion of Groupthink

I produced the most accurate set of predictions on SAP that has been published anywhere over a more than 10-year period.

• You can validate this in the article A Study of SAP Accuracy.
• You can compare this accuracy to another person who was prominent in the SAP space in this article A Study into John Appleby’s HANA Accuracy.

My predictions were not “peer-reviewed” and no one had any input to them.

If these predictions had been pushed through the sausage machine by including other peers, it would have included SAP resources who all had a financial bias (as they had the domain expertise). The forecast accuracy of the predictions would have been ruined, as all of my predictions would have been reversed — in order to help each individual maximize their ability to make money from SAP.

Here are other examples.

• Dr. John Ioannidis correctly called out the fraud of Theranos, while others drank the Kool-Aid, and he was in the distinct minority. Ioannidis did it again by demonstrating that much of the information presented to the public on covid was false and that lockdowns were counterproductive. Combining the forecast of Ioannidis with his “peer group” would have only ruined Ionnidis’ forecasts.
• Establishment economists were not able to predict the 2008 housing bust. Again, a major reason is the financial bias of the “experts.”

Peer review ignores this history and holds up the mob mentality over individuals free to think independently from mob pressure. Peer review is, in this way, more similar to Asian than Western culture, where everyone needs to agree, and those from the top of the hierarchy must agree before something is considered to be true.

Peer review is a bastion of conformists and people who are uncomfortable taking positions that other people do not agree with. For example, of the people who have brought up the peer review argument to me, as I said, none of them had any background in research, and they were all women. This makes sense, as women are far less likely to be willing to stand out alone on a platform if they don’t feel they are in the majority. Women typically seek out the majority or “safe” view in order to determine their own views in a process referred to as triangulation. Women have also been shown to be much more likely to stigmatize views that are not part of the in-group. And this is not to say it is only women; it is just a matter of degree.

Conformity is the norm with humans. By copying others, we increase our safety and skip doing our research on every topic. The following videos show how deep conformity goes. 

This woman is standing up and down on command for no other reason than others are doing it. This is a person who will be highly impressed by peer-reviewed research. 

Recall that these subjects faced no intimidation or threats to their livelihood or careers. This conformity increases extraordinarily as soon as non-conformity has anything more than mild implications.

People may propose that more education may help alleviate this. However, education promotes conformity. To corporations, this is the exact desired outcome of the socialization process. This provides docile workers who will do unethical things and can be exploited.

This video shows the typical person can’t even stray from the group on how long a line is. 

This is also called herd psychology. During stressful times and when people are afraid, they follow the herd. 

You can witness this exact type of conformity on social media.

Polls often show that around 1/2 of people in the US believe they have a personal angel. Under this same logic, personal angels have also been peer-reviewed.

Problem #12: Ghostwritten Published “Peer Reviewed” Studies

The study’s sponsor ghostwrites many studies, and then the articles are “shopped” to prestigious scientists to affix their names to. They can review the study but have limited input on making changes. The “names” are incentivized to agree with the study as they are both paid for their name being attached, and the study also counts as a publication, enhances their prestige, and meets their publication requirement.

The topic of ghostwritten studies has been brought up in many journals, and the response from these journals has been that they do not see it as their responsibility to check if a study has been ghostwritten.

Conclusion

Peer review, particularly for those with little research background or who have never participated in the peer-review process to see how “the sausage is made,” is also a way of turning off one’s brain, as the concept is that the peers have quality-checked the study’s validity. However, that overstates what the peers in peer review do and misconstrues or falsely assumes that the study creators disclose everything to the peer reviewers. The peer reviewers were not involved in the development of the study, and they can’t verify if the data is accurate or is even entirely rigged.

It is probably time to stop turning off one’s brain and accepting studies because they have been published in a peer-reviewed journal. It is easy to find very low-quality and rigged research published in prestigious journals like The Lancet, The New England Journal of Medicine, and the Journal of the American Medical Association.

Furthermore, new and better ways exist to publish research and receive feedback and criticism on research articles.

Ceasing the Double Standard, Depending Upon Whether One Agrees With the Study

As was mentioned at the beginning of this article, the critique that an article is not peer-reviewed usually changes depending on whether the person agrees with the study. There cannot be logically two different standards. Any article or study that a person agrees with is immediately accepted without any concern for whether it was published in a peer-reviewed journal. Still, the peer review question appears only for those studies or articles that the person agrees with.

People accept articles published in entities with enormous financial conflicts, as I cover in the article The Rating of IT Analysts Like Gartner, Forrester, and Others.

Gartner is so corrupt with multiple financial conflicts I have called them a “Rubix Cube of Corruption,” as I cover in the article Why No One Seems to Care that Gartner Follows No Research Rules.

However, even by breaking every research and integrity rule known to man, Gartner is still very successful in selling its output. This is because for most people, whether research rules are followed or whether there is corrupt financial bias is of diminutive interest versus whether other people accept the entity as a credible source. Most of the people who deal with Gartner within the vendors are aware of the scam but are petrified to call out Gartner, as I cover in the article Why Is Their Near Radio Silence by Vendors Criticising Gartner? as they fear retribution by Gartner.

Financial Bias is Ok?

It is shocking to me how often I have had people express that they are unconcerned with financial conflicts when financial conflicts are the number one predictor of article or study inaccuracy or bias. It is straightforward. If an entity is paying for a study to be performed, that study will show a positive outcome from the product, treatment, and philosophy of the paying entity. Even the most widely respected research publications (The New England Journal of Medicine, Nature, etc..) have enormous financial conflicts.

Peer review is low ineffectiveness for validating accuracy, but checking for financial bias has proven to be high inaccuracy, which means this, not whether an article has been peer-reviewed, is far more predictive of quality than is peer review. That is, highly corrupt journals with massive financial conflicts are mindlessly accepted by the public and insiders mostly because they have significant brand awareness and secondarily because they are “peer-reviewed.”

The following quotes are also of interest on this topic.

How Large is the Peer Review Industry?

Peer review is a ubiquitous element of scholarly research quality assurance and assessment. It forms a critical part of a research and development enterprise that annually invests $2 trillion US dollars (USD) globally [1] and produces more than 3 million peer-reviewed research articles [2]. As an institutional norm governing scientific legitimacy, it plays a central role in defining the hierarchical structure of higher education and academia [3]. – Research Integrity Journal

The Impact, a Publication in a Major Journal, Can Have on a Career

For example, according to Google Scholar Metrics 2016, the journal with the highest impact factor is Nature. For a young scientist, publishing in journals like Nature can represent a career turning point, a shift from spending an indefinite number of extra years in a more or less precarious academic position to getting a university tenure. Given its importance, publishing in top journals is extremely difficult, and rejection rates range from 80 percent to 98 percent. – The New Republic

The following video explains more on conformity. As soon as these participants are told one of the identical cakes is more expensive than the other, they immediately come up with reasons why the more expensive identical cake is special.