More Details on OvidSP's Basic Search
I have been playing around OvidSP to try and get to know it better. I am sure other librarians have been doing the same. As heavy Ovid users, the librarians around here have been trying OvidSP and sharing the experiences, comparing searches and their results. We are still trying to figure out what the man behind the curtain is doing. I figure if we are still wondering about it, then there must be others who are as well. So, I decided to re-post another of Ovid's emails about OvidSP.
---------------from Ovid--------------
More Details on OvidSP's Basic Search:
OvidSP's Basic Search continues to generate a lot of buzz in the librarian community. One specific feature that is causing a lot of interest is the new Include Related Terms option. Some customers have asked: Why do I sometimes get fewer search results when I include related terms? The following explanation should shed some light on this.
The NLP-based Basic Search was designed to find the most relevant results, not all results. Since some searches could return an almost infinite set of results, the system has a manageable cut-off point of 500 results ranked by relevancy.
Each search result is assigned a score based on its relevancy to the search query and all results with the same score are grouped together. The number of groups (let's call them "Scoring Groups") varies depending on the number of results that are assigned the same score. We stop after the first group that brings the total result count past 500. If that group is large, the ending count may be a lot higher than 500.
Example: The query "computed tomography {No Related Terms}" returns 900 equally relevant results in MEDLINE. The query doesn't provide much information, so the system can't whittle the result down to the 500 or so that it prefers to return; all 900 results are equally relevant, so they're all returned.
Here is another hypothetical example. Suppose that for a particular search, the scoring of the results returns the following:
---------------from Ovid--------------
More Details on OvidSP's Basic Search:
OvidSP's Basic Search continues to generate a lot of buzz in the librarian community. One specific feature that is causing a lot of interest is the new Include Related Terms option. Some customers have asked: Why do I sometimes get fewer search results when I include related terms? The following explanation should shed some light on this.
The NLP-based Basic Search was designed to find the most relevant results, not all results. Since some searches could return an almost infinite set of results, the system has a manageable cut-off point of 500 results ranked by relevancy.
Each search result is assigned a score based on its relevancy to the search query and all results with the same score are grouped together. The number of groups (let's call them "Scoring Groups") varies depending on the number of results that are assigned the same score. We stop after the first group that brings the total result count past 500. If that group is large, the ending count may be a lot higher than 500.
Example: The query "computed tomography {No Related Terms}" returns 900 equally relevant results in MEDLINE. The query doesn't provide much information, so the system can't whittle the result down to the 500 or so that it prefers to return; all 900 results are equally relevant, so they're all returned.
Here is another hypothetical example. Suppose that for a particular search, the scoring of the results returns the following:
Five of the results for this query were assigned a score of "1"; 14 a score of "2"; 37 a score of "3"; and so forth. We can see that the "Scoring Group" that brings the total over 500, Scoring Group "7", contains 367 results, bringing the total number of results OvidSP shows to 805. If the last group, the one with a score of "7" had returned only 200 results, the total number of results shown would have been 638 (438 + 200). If Scoring Group "6" had included enough results to bring the total to over 500, say 359 for example, the total would have been 508 (149 + 359). No results from Scoring Group "7" would have been included.
Please note: the Scoring Groups are a "behind-the-scenes" tool and do not represent the number of stars assigned to a particular result in the result set and which range from 5 to 1 for each result from a Basic Search.
Why was 500 chosen as the cutoff point for Basic Search results? Based on our extensive end-user research, we found that users who are looking for the most relevant articles to their query− rather than doing an exhaustive literature search− rarely look at more than 100 results before accepting what they have, refining their search, or trying a new search. On the other hand, we want to have a sufficient amount of results. Based on this research, it was determined that five hundred is an appropriate set of results for Basic Search.
If you have any questions about Basic Search or any other aspect of OvidSP, please contact your Ovid Account Representative or support@ovid.com. And if you haven’t already done so, click the Try OvidSP!
---------------
I'd like to say this makes sense to me, but I am not sure it does. I had to re-read the "scoring" explannation a few times to get a basic understanding of how they determine relevancy. However, I have to admit I still don't understand how I get fewer search results when I include related terms? The more frustrating part is that OvidSP on its website suggests this as a way to broaden the search. My guess is that patrons are going to be frustrated if they click this and it doesn't broaden their search, or worse gives them fewer results. They are not going to sit through and read about the relevancy scoring.
I am going to have play around with OvidSP some more. I am a heavy Ovid user, so it behooves me to figure this thing out, but I am far from comfortable about using OvidSP Basic. I am still very concerned about how poorly my asthma and ragweed in children search panned out. I tried it again today to see if there were any changes and I am still getting the same poor results. While I am concerned about what is going on behind the scenes with the terms, I am more concerned that the combine feature just isn't working correctly.
Why was 500 chosen as the cutoff point for Basic Search results? Based on our extensive end-user research, we found that users who are looking for the most relevant articles to their query− rather than doing an exhaustive literature search− rarely look at more than 100 results before accepting what they have, refining their search, or trying a new search. On the other hand, we want to have a sufficient amount of results. Based on this research, it was determined that five hundred is an appropriate set of results for Basic Search.
If you have any questions about Basic Search or any other aspect of OvidSP, please contact your Ovid Account Representative or support@ovid.com. And if you haven’t already done so, click the Try OvidSP!
---------------
I'd like to say this makes sense to me, but I am not sure it does. I had to re-read the "scoring" explannation a few times to get a basic understanding of how they determine relevancy. However, I have to admit I still don't understand how I get fewer search results when I include related terms? The more frustrating part is that OvidSP on its website suggests this as a way to broaden the search. My guess is that patrons are going to be frustrated if they click this and it doesn't broaden their search, or worse gives them fewer results. They are not going to sit through and read about the relevancy scoring.
I am going to have play around with OvidSP some more. I am a heavy Ovid user, so it behooves me to figure this thing out, but I am far from comfortable about using OvidSP Basic. I am still very concerned about how poorly my asthma and ragweed in children search panned out. I tried it again today to see if there were any changes and I am still getting the same poor results. While I am concerned about what is going on behind the scenes with the terms, I am more concerned that the combine feature just isn't working correctly.
Why is it when I combine searches 1 and 2 and 3, (by clicking the boxes and hitting combine searches with And) I get 0 results, but when I type out asthma and ragweed and children I get 705 results? That does not make sense. There shouldn't be different results!
These types of incosistencies are driving me nuts. How can I show or use a product when it doesn't even combine things the way it should? It makes me question a lot of other things that I normally wouldn't.
These types of incosistencies are driving me nuts. How can I show or use a product when it doesn't even combine things the way it should? It makes me question a lot of other things that I normally wouldn't.
5 Comments:
Hi,
I am struggling also, and I have found a result in PsycInfo that illustrates it nicely:
1. stress library personnel 5469
2. stress library personnel (incl related terms) 504
Strange isn't it?
But when you follow their explanation it is the right result!
Efor instance
When somewhere in the Scoring groups the results are lets say 499, and a next Scoring group could be 4970 in the first option and only 5 in option 2.
In both cases you are exceeding the 500 limit and the search stops...
But what if you are using a lot of general terms … is 500 not a bit too little? My in between results are often very large.
I am also worrying about the ranking: the most relevant should be the first ones and in my experience they sometimes are not relevant at all.
Sometimes the first hits do not even have the search terms in the title or in the key words … so much for precision!
Hi,
Evidently, the combine function works differently using the search history box as opposed to the basic search box. It shouldn't, but there you go!
I did a similar search to your asthma example with virtually an exact result.
Jeremy
I guess you are not really supposed to combine searches in the basic search: in a natural language search, everything has to be in the first line problably...
I did a similar search to yours using "pressure sores" "diabetic" "elderly" (hardly an unusual grouping of symptoms or patients) and got similar results.
I e-mailed Ovid to see if they could explain what was happening and I received this reply:
"The difference is OvidSP is set up to give you the best results for your query. It is unlikely that the best results from a search on Diabetic would also be the best result on Elderly and the best on pressure sores. I ran a search on each of the three terms in Medline. I Used the Boolean and for the results and had 1 result. When I search the same three terms in OvidSP basic that was the top result."
Perhaps I'm becoming dull in my old age, but this explanation does not really clarify anything for me. Perhaps you or one of your readers could explain what he's talking about . . .
On the other hand, doing this search in OvidSP Syntax yielded 14 highly applicable results.
The reason why the combine doesn't work (and this is something we have to educate users about) is that in the first step, the search engine is looking for the most relevant item related to that term (i.e. asthma). This set may not necessarily have any references to articles on ragweed. (In fact, it doesn't.)
When I repeated the search combining the asthma set and the children set without combining the ragweed set, 242 items were retrieved (Medline 1950-Present). However, this was a smaller number than when I enter 'children asthma' on one line (919 items). However, this is not surprising, because, again, the initial set with the most relevant items on the topic of asthma in general will not necessarily have a lot of articles about children. In other words, the same records that are the most relevant for someone interested in asthma in general are not necessarily the same records that will be most relevant to someone interested in asthma in children.
I quite like the basic search. I've tried out searches that I've been asked for advice on in the past couple of months and been quite pleased with the results. What was retrieved would be highly suitable for someone who needs a few highly relevant items.
For an academic institution where many of our students/faculty never receive training on database searching, the basic search provides our users with a far better chance of getting a decent search result than they'd have using the Ovid Syntax mode on their own.
For 'power' searchers... We'll still teach them how to use the Ovid Syntax, and librarians themselves will continue to use it for comprehensive searchers.
Post a Comment
<< Home