Wolfram Alpha has been popping up all over my online current awareness feeds and honestly the first thing that came to mind was it sounded like an evil computer created by the equally evil law firm Wolfram & Hart. But since David Boreanz is now on the T.V. show Bones, I decided that it was unlikely that Wolfram Alpha was an elaborate marketing plot for another Joss Whedon show.
It turns out Wolfram Alpha is a “computational knowledge engine,” this is not to be confused with a regular ol’ search engine. A search engine craws over the web filing and indexing data. Wolfram Alpha relies upon the data inside it (entered by employees) that it scrutinizes and compares to draw conclusions about overlapping and intersecting details.
It has been in development for five years and it is still very picky about search terms and how people search it. It often misunderstands queries or search terms. Wolfram Alpha prefers small simple search strings and it seems to do well with searches the produce specific quantifiable results.
PCWorld does a good job explaining how somebody could use Wolfram Alpha to find overlapping and comparative information.
Then pick another term that will produce overlapping or comparative results. Try ‘California income’. Simple enough. Each search result includes a pop-up window that identifies its source, in case you ever want to dig into the origins of Wolfram Alpha’s information.
Now try another overlapping term, such as ‘California New York income’. Wolfram Alpha generates a simple table for comparing income in the two states. Now, you may begin to see its potential.
The site is admittedly young and is versed in only certain topics. Thus, a search for ‘San Francisco income’ comes up empty. If you cut a search back to its core and Wolfram Alpha still has nothing to offer, that entire topic might be missing from its current database. Visit more of the site’s examples to see whether a similar subject is available.
The folks over at the Dragonfly blog (Pacific Northwest Region NNLM’s blog) have begun to look and even post a link to PatrickMD.netwho wrote “Why You Shouldn’t Trust Wolfram|Alfa for Medicine.” He tested it using several different medical health queries and he found that once you strayed from their examples Wolfram Alpha had problems finding the information. Additionally Patrick discovered serious questions about the quality and how it interprets its data using W|A’s own example searches.
Their example of “Mayo Clinic, Olmsted Medical Center” is supposed to compare two large medical centers in Rochester, Minn. However, it actually compares the Mayo Clinic satellite in Jacksonville, FL, with Rochester. Even that apples-to-oranges comparison is hampered because there is no data in WA for Mayo in Jacksonville. Try finding data on Mount Sinai Hospital in Miami — the only Mount Sinai that WA admits to knowing is in New York City (and has no affiliation with the one in Miami.)
This kind of problem with non-clinical information me leery of trusting its other results especially clinical results Patrick tests Wolfram Alpha further by looking at the risk of heart disease for a male nonsmoker.
WA says it calculates heart disease risk based on the Framingham study, but I get different results. (Assuming LDL 111, HDL 54, BP 120/80, nonsmoker, not diabetic.) Using the male score sheet from Wilson, Prediction of Coronary Heart Disease Using Risk Factor Categories. Circulation 1998 97 (18): 1837-1847., I get 6%, versus WA’s 4.6%.
As the score sheets just return whole numbers, WA is likely using the Framingham model which is discussed in the paper. However, even using that I get 5.4%, a solid 0.8% more than WA’s result. (for sticklers, my work is after the “more”.)
Based on Patrick’s testing, my testing (which is much more basic), and othersit seems that W|A is just in its infancy and has a very long way from being any sort of real tool for medical purposes. Even if the data was correct within the system, Wolfram Alpha also has another large problem, it is too complicated to search. You really have to search it in a very specific manner in order to get results. That kind of a search just doesn’t fly with the regular public and many professionals. Just look how hard it is for us to get our users to search using MeSH! Look at the trends at tagging in libraries and the web, people want to use their own terms and their own search methods. Wolfram Alpha fails at this type of searching completely. It makes a poor ILS system look easy to use.
According to Wolfram Alpha’s FAQ page it is free to use for personal noncommerical use, subscriptions will be available in the future for enhanced versions and large scale commerical use. Yet without inconsistant and unverified data and a extremley fussy searching feature, I don’t see many people wanting to pay to use it. It is free and I can’t think of a reason to use it for my job as a librarian.
Who knows maybe in the far future we will have something like Star Trek’s computer system where we can just orally ask it a question and it will answer us back in our own language.