Assessing the published literature

Though well-grounded in molecular and clinical data, the conclusions offered by Marshall Protocol researchers are sometimes met with skepticism by clinicians and fellow researchers. Some wonder how the MP science could be valid, given the existence of seemingly contradicting evidence.

Researchers who work with Autoimmunity Research FoundationNon-profit foundation dedicated to exploring a pathogenesis and therapy for chronic disease. (ARF) take no special pride in arguing that the nature of chronic disease is different than most clinicians and researchers have imagined. Indeed, it makes matters more difficult: the less familiar a conclusion is, the harder it is to persuade someone of its validity.

Using statistical inferences, John P. A. Ioannidis concluded in the prestigious journal PLoS Medicine that half of published research must be wrong.¹⁾ In grappling with a confusing study or even a field of study, it's seriously worth considering how Ioannidis could be right.

What assumptions do researchers routinely make that prevent them from embracing the science behind the MP? The following is a list of hypotheses about medical research that explains some of the hurdles that some have had to overcome thus far in accepting the MP.

1. Despite ambitious efforts, there is limited evidence that defective genes cause most human diseases.
2. The idea of permanent spontaneous remission of chronic disease is a myth.
3. Diseases and symptoms of disease may share a common pathology and are part of a single disorder. Although patients who become infected with the Th1 pathogensThe community of bacterial pathogens which cause chronic inflammatory disease - one which almost certainly includes multiple species and bacterial forms. are given a variety of diagnoses, there are often no clear-cut distinctions between one disease and the next. Rather, symptoms frequently overlap, creating a spectrum of illness in which diseases are more connected to one another than mutually exclusive disease states. One significant piece of evidence for this is the high rate of familial aggregation.
4. The era of specialization has reduced the level of interdisciplinary cooperation and consequently the ability for clinicians to appreciate the underlying connection among diseases.
5. The existence of a blood-brain barrier has been historically touted as a reason why microbes could not possibly have caused any number of neurological diseases. However, with the accumulation of additional evidence, this concept has progressively lost meaning.²⁾
6. Animal models do not always accurately represent human biology. In fact, murine (mouse) models are particularly problematic as the murine Vitamin D ReceptorA nuclear receptor located throughout the body that plays a key role in the innate immune response. is not analogous to the human Vitamin D Receptor in some key respects.
7. Only certain bacteria grow in an in vitro environment. Many fastidious forms of bacteria can only grow in the conditions offered by the human body.
8. Koch's postulates, the theory that a single species of pathogen causes a single disease appears wrong but continues to be embraced, despite strong evidence to the contrary.
9. In silico studies are a valid way to model the interactions between molecules and nuclear receptorsIntracellular receptor proteins that bind to hydrophobic signal molecules (such as steroid and thyroid hormones) or intracellular metabolites and are thus activated to bind to specific DNA sequences which affect transcription..
10. Studies touting a therapy against chronic bacterial infections which doesn't demonstrate some manner of immunopathology, or bacterial die-off, are not demonstrating that recovery is possible.

The process by which studies are designed and research is interpreted and shared has a number of liabilities.

1. One of the basic assumptions underlying many genome-wide association studies has been that the genetic makeup of all an individuals’ cells is essentially the same. In the vast majority of genetic studies to date, researchers have assumed that in sequencing DNA isolated from blood would reveal the genetic makeup of diseased tissues as well. This supposition was convenient: except for cancer, samples of diseased tissue are difficult or even impossible to take from living patients.³⁾ However, recent evidence has emerged that the genes of at least some cells from the blood and tissue do not match genetically,⁴⁾ meaning that ambitious and expensive genome-wide association studies may prove to have been essentially flawed from the outset.
2. Science is often not objective.⁵⁾ The choice of research questions, the methods to collect and analyze data, and the interpretation of results all reflect the worldview of the investigator.⁶⁾ Investigators often overemphasize the importance of their findings and the quality of their work and choose interpretations that will enhance chances of success of obtaining funds from granting bodies.
3. Other serious conflicts of interest arise when for-profit organizations, such as device, biotechnology, and pharmaceutical companies, provide funds for conducting research, consulting, and attending scientific meetings. In the past 20 years, there has been an 8-fold increase in the number of trials for which authors declare industry affiliation.⁷⁾ For example, one of the chief vitamin D researchers also sells a product containing vitamin D. Investigators accepting funds risk conflicts of interest. Even more problematic, they may cede their right to directly supervise data collection, participate in or supervise data analysis, and write the research reports to which their name is attached.^{8) 9) 10)}
4. Observational epidemiological studies are compromised by bias inherent to the design of the studies themselves. It is arguably impossible to sufficiently control the socioeconomic factors, which drive a person to participate in a therapy or take a supplement. Patients who take drugs or supplements, as recommended by experts, are more likely to have better education, better access to health care, etc.
5. While much excitement has been generated surrounding evidence-based medicine, internal documents from the pharmaceutical industry suggest that the publicly available evidence base may not accurately represent the underlying data regarding its products.¹¹⁾ Publication or reporting bias, or the tendency not to publish negative results, increases the likelihood that the literature's descriptions of a therapy's efficacy are erroneous and decreases the likelihood that adverse effects are fully reported.¹²⁾
6. According to a 2009 BMJ article chronicling a conference presentation, researchers, like politicians, use “spin” in presenting their results. This is the case of studies with both non-significant and significant results.
7. Smaller-sized studies are often published without the necessary statistical power to prove the validity of an observation or therapy. For example, an article in 1994 reported that a variant of the Vitamin D Receptor gene explains most of the population's risk for having low bone-mineral density.¹³⁾ The finding made the cover page of Nature, heralding the “osteoporosis gene.” Other subsequent studies showed an opposite effect, with the same variant predisposing to stronger bones. A large-scale analysis, of 100-fold more participants than the original Nature study, showed that there is no effect at all.¹⁴⁾
8. In some fields, the reverence for experts and expert opinion has fostered mistaken consensus, even while successful cures for chronic disease remain elusive.
9. Peer review has had the effect of validating popular theories to the detriment of novel and/or contradictory theories.
10. Therapeutic interventions, for which there is a short-term benefit but long-term harm, are not accurately represented in studies over shorter time windows. Insufficient follow-up is particularly problematic in studies looking at vitamin D and other immunosuppressants. Successful treatments must be curative, not palliative.
11. Many of the articles that appear in scientific journals, under the bylines of prominent academics, are actually written by ghostwriters, who are paid by drug companies.¹⁵⁾ These seemingly objective articles, which doctors around the world use to guide their care of patients, are often part of a marketing campaign by companies to promote a product or play up the condition it treats. One recent and prominent example relates to the way studies supporting HRT were written by ghostwriters.
12. As was illustrated by a 2009 survey of 838 marketed drugs, the available genotoxicity and carcinogenicity data for these drugs is inconsistent.¹⁶⁾ Of these 838 drugs, 366 (43.7%) did not have such data.
13. According to Henry H. Q. Heng's 2008 JAMA commentary, there may be limits of experimental medical science at the current level of our understanding, which is likely much more primitive than most of us like to think. Similar to repairing a clock in which each broken part is fixed in order, investigators have attempted to discover causal relationships among key components of an individual and to treat those components accordingly. As a result of the over-reliance on reductionist approaches, doctors recommend therapies for which we lack full pathophysiological understanding.¹⁷⁾
14. As described in the section, Disease is a continuum, it is common practice to assign one group of patients participating in a controlled trial to be the “healthy control group.” While researchers are apt to make a hard distinction between health and disease, this dichotomy is contrary to what we know about successive infectionAn infectious cascade of pathogens in which initial infectious agents slow the immune response and make it easier for subsequent infections to proliferate.. The process of successive infection does not just occur in sick people or people who are symptomatic. In healthy subjects, subclinical infection is not the exception, but the rule. From even before birth, every human is constantly acquiring new microbes.
15. Corrections come slowly if at all according to a March 2011 paper looking at a papers published in Nature or Science and since rebutted (described here. According to the authors:

For those convinced that science is self-correcting, and progresses in a forward direction over time, we offer only discouragement. We had anticipated that as time passed, citations of the original articles would become more negative, and these articles would be less cited than other articles published in the same journal and year. In fact, support for the original articles remained undiminished over time and perhaps even increased, and we found no evidence of a decline in citations for any of the original articles following publication of the rebuttals…. [A major news report] trumpeting “Fish stocks eaten to extinction by 2050″ (Leake 2010), based on a highly contentious projection… [fails] to mention any of the 11 rebuttals that question this projection, but it misses the later consensus paper by the same author and many of his critics that reverses the earlier projection of collapse, and instead expects rebuilding to occur in 5 of 10 well studied ecosystems.

Science is the belief in the ignorance of experts.

Richard Feynman, The Pleasure of Finding Things Out (1999)

Ioannidis et al.'s 2008 Science analysis shows the long length of time that passes between discovery and translation into common practice. This work also shows how after a number of interventions supported by high-profile clinical studies, subsequent evidence from larger and/or better studies contradicts the earlier work.¹⁸⁾

• Lies, Damned Lies, and Medical Science – Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science.
• Control Group: Patients Take Biomedical Research into Their Own Hands – Using forums like PatientsLikeMe (an online patient community), Facebook and Twitter, patients with ALS, cancer and a range of other conditions are turning biomedical research— a process once mystified by jargon, privacy, peer review and expensive journal subscription fees—into an open-source and collaborative effort.
• Drugs on test – randomized controlled trials may have been elevated to an “undeserved pedestal”
• Spielmans and Parry offer a robust critique of the series of assumptions upon which evidence-based medicine is based in their 2010 Bioethical Inquiry paper available here.
• Scientists at Rensselaer Polytechnic Institute have found that when just 10 percent of the population holds an unshakable belief, their belief will always be adopted by the majority of the society.
• Dissent and heresy in medicine: models, methods and strategies – an excellent review of challenges to medical orthdoxy

===== Notes and comments =====

— Sallie Q 08.19.2017 removed erroneous links (are usually but not always, to newspapers)

• Publication bias – A 2012 TEDMED talk by Ben Goldacre

The pervasive influence of the market can and has compromised doctors' judgment.

• Legacy content
  • f12
  • s363
  • s152
  • s275
  • s139
  • s148
  • e57
  • e66
  • s138
  • s134
  • s142
  • s144
  • s147
  • s145
  • f120
  • s143
  • f275
  • s140
  • s156
  • s146

• Sir William Osler, a great judge of human nature, once mused that it was the taking of pills that separated humans from animals

The Emergence of Translational Epidemiology: From Scientific Discovery to Population Health ImpactAm J Epidemiol 1 September 2010: 517-524.

Traversing the Valley of Death: A Guide to Assessing Prospects for Translational Success

Sci Transl Med 9 December 2009: 10cm9.

Translational informatics: enabling high-throughput research paradigms

Physiol. Genomics 6 November 2009: 131-140.

Evaluating the causal relevance of diverse risk markers: horizontal systematic review

BMJ 5 November 2009: b4265.

https://www.sciencemag.org/content/321/5894/1298.full.pdf

https://www.sciencemag.org/site/includefiles/help/holmes_sla_stm_ppt.pdf

A systematic examination of the citation of prior research in reports of randomized, controlled trials.

Robinson KA, Goodman SN. Ann Intern Med. 2011 Jan 4;154(1):50-5

Abstract BACKGROUND: A randomized, controlled trial (RCT) should not be started or interpreted without accounting for evidence from preceding RCTs addressing the same question. Research has suggested that evidence from prior trials is often not accounted for in reports of subsequent RCTs. OBJECTIVE: To assess the extent to which reports of RCTs cite prior trials studying the same interventions. DESIGN: Meta-analyses published in 2004 that combined 4 or more trials were identified; within each meta-analysis, the extent to which each trial report cited the trials that preceded it by more than 1 year was assessed. MEASUREMENTS: The proportion of prior trials that were cited (prior research citation index), the proportion of the total participants from prior trials that were in the cited trials (sample size citation index), and the absolute number of trials cited were calculated. RESULTS: 227 meta-analyses were identified, comprising 1523 trials published from 1963 to 2004. The median prior research citation index was 0.21 (95% CI, 0.18 to 0.24), meaning that less than one quarter of relevant reports were cited. The median sample size citation index (0.24 [CI, 0.21 to 0.27]) was similar, suggesting that larger trials were not selectively cited. Of the 1101 RCTs that had 5 or more prior trials to cite, 254 (23%) cited no prior RCTs and 257 (23%) cited only 1. The median number of prior cited trials was 2, which did not change as the number of citable trials increased. The mean number of preceding trials cited by trials published after 2000 was 2.4, compared with 1.5 for those published before 2000 (P < 0.001). LIMITATION: The investigators could not ascertain why prior trials were not cited, and noncited trials may have been taken into account in the trial design and proposal stages. CONCLUSION: In reports of RCTs published over 4 decades, fewer than 25% of preceding trials were cited, comprising fewer than 25% of the participants enrolled in all relevant prior trials. A median of 2 trials was cited, regardless of the number of prior trials that had been conducted. Research is needed to explore the explanations for and consequences of this phenomenon. Potential implications include ethically unjustifiable trials, wasted resources, incorrect conclusions, and unnecessary risks for trial participants.

===== References =====