Flaws in Validating Medical Studies
An article in the Boston Globe, Flaws are found in validating medical studies: Many see need to overhaul standards for peer review by By Michael Kranish (August 15, 2005) looks at the way medical research is published and questions reliability of the peer review process.
Prestigious peer reviewed journals published articles that declared that women who underwent hormone replacement therapy, and people who ingested large amounts of Vitamin E, had relatively low rates of heart disease; only to discover later on that subsequent research refuted and contradicted those highly regarded studies. Researchers are wondering whether the peer review process has failed and whether it needs to be re-tooled so that more reliable research can be published.
Peer review is only as good as the person doing the review and their quality and expertise can vary. "There is no governing body that defines what constitutes good peer review, or that demands that certain standards be followed."
The Boston Globe article also cites John P. A. Ioannidis, Why Most Published Research Findings Are False. PLoS Med. 2005 Aug 9;2(8):e124 [Epub ahead of print] PMID: 16060722 who says that "flaws" in the peer review system may only be ''part of the puzzle" that should be examined to improve research.
According to Ioannidis:
- The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.
- The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.
- The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true.
- The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.
- The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.
- The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.
Given the high number of studies the Globe and Ioannidis state as either wrong or flawed, it is no wonder that many in the medical profession, as well as patients, can be easily confused as to what really is the best course of treatment. Who do you you trust? Don't forget it wasn't so long ago that HRT was considered the cure all.
1 Comments:
Peer Review Failure at JAMA
The Center for Regulatory Effectiveness www.TheCRE.com has revised the Wikipedia entry on peer review to discuss and document peer review failure at JAMA. The revised entry exposes a politically biased Journal of the American Medical Association (JAMA) article [JAMA, May 24, 2006; 295(20): 2407 - 2410] on the Data Quality Act and atrazine that contains numerous factual errors and misrepresentations. A crucial, obvious error in the article was the assertion that atrazine is being phased out by the European Union because “Atrazine...has been repeatedly demonstrated to be a potent endocrine disruptor....” JAMA’s peer review process accepted this claim even though the Official Journal of the European Union explicitly stated “In the 70s, a political decision was taken to reduce to ‘zero’ the presence of pesticides, independent of their toxicity.” [Emphasis added]
The CRE Wikipedia revisions also include a discussion of the peer requirements imposed on federal regulatory agencies by the Office of Management and Budget. Federal agencies cannot use or rely on scientific information that does not meet the OMB peer review requirements. Many peer reviewed journals do not meet the OMB peer review requirements.
For more information about the failed peer review at JAMA, please contact William G. Kelly, Jr., wgkelly@tetontel.com. For more information about the US government’s peer review requirements, please contact Scott Slaughter, Slaughter@mbsdc.com.
Wikipedia article on peer review
Post a Comment
<< Home