Googling vs. PubMeddling
There is still a lot of conversation about the BMJ article regarding Googling for a Diagnosis which I and other have previously blogged about. One interesting letter, posted on BMJ's site, PubMed vs. Google -a brief comparison by Reinhard Wentz, indicated that searching PubMed (even without using MeSH) identified potentially relevant studies in 23 cases, an 88% success rate.
In a MEDLIB_L post Wentz said, "I was astonished to see that in some contributions to medical librarians' discussion lists it was suggested (among other criticisms) that Tang and Ng's results are no better than 'flipping a coin'. Surely, during a case presentation or when ‘conducting a diagnostic exercise' more than two possible diagnoses are discussed. A success rate of 58% on Google is impressive and better than just flipping a coin."
I guess I would be one of those coin flipper people... My point was that when you flip a coin you have a 50% chance of getting heads, so 58% success rate of choosing the right diagnosis still does not give much better odds than if I flipped a coin. My thought was that a 58% success rate is miserable when I know there are better tools out there. I chose the "flipping a coin" to illustrate my point, of course I know when conducting a diagnostic exercise there are more than two possible diagnoes available.
I am glad that Wentz, illustrated how a simple quick PubMed search produced better results than Google, but I have to respectfully disagree that "a success rate of 58% on Google is impressive." I still think it is rather pathetic for something being used as clinical decision tool.
2 Comments:
58% is pathetic. Would like those odds when your pilot is landing the plane!?
Just look at the odds of what happens when "we" are 99.9% successful:
- 2 unsafe plane landings per day at O'Hare Intl. Airport
- 20,000 incorrect drug prescriptions per year
- 500 incorrect surgical operations each week
- 50 newborn babies dropped at birth by doctors every day
- 22,000 check deducted from the wrong bank account each hour
(from http://www.inc.com/magazine/19890401/5589.html)
While it is extremely difficult to have even close to a 99.9% success rate, the 58% success rate should not be touted as impressive in this instance.
The article is ridiculous, the statistic isn't. A Google-influenced lookup for diagnosis that is in error is no more likely to lead to mistreatment than a traditional-resources-based error in diagnosis. You are led with Google to expect the most common cause, and it is generally successful. But in practice, of course this would be tested.
The "furor" over this article is a joke. Google is an easy to use information seeking tool. Like all information-seeking tools, it has disadvantages and weaknesses.
How much would the figure have gone up using Google Scholar? How often does the first diagnosis in a clinical setting prove incorrect? What was the point of this article other than to belabor the point that among the porn and ads on the internet, there is a wealth of useful medical information, and it has found an audience. People subscribe to BMJ for this?
Post a Comment
<< Home