Interesting SciHub News

Wow it has been a while since I have published. Hopefully, I won’t have that kind of blogging break again.

As librarians we all know there are many ways to get scientific articles, some are legit while other methods are illegal. SciHub is one of the illegal methods.  For those of you who haven’t heard of SciHub, think of it as Napster for scientific articles. Alexandra Elbakyan, participated in research forums where scientists asked each other for research papers.  Elbakyan created SciHub as an automated method to share those papers. The process made it easier for people without access to paywalled papers or difficult to find papers to download them.  Sound familiar? Napster was founded by Shawn Fanning and Sean Parker as a peer to peer sharing service “for music enthusiasts to download copies of songs that were otherwise difficult to obtain.”

As with Napster there seems to be an anti SciHub group (which obviously includes the publishers) and a pro SciHub group who want easier access to materials that are hard to obtain or they don’t want to pay for.  As an individual it can be very easy not to care about the publishers who are profit driven making money on the backs of poor researchers and academic institutions struggling to deal with publisher price gouging rate increases.  While it is easy to think that way, there are many layers to the problem that has led us to this SciHub situation. Publishers, libraries, users, changes in society, technology, politics, and money are all to blame for the evolution of SciHub.

Until recently Elbakyan has been basically untouchable by the U.S. and European courts. While there have been several lawsuits that have gone against her and SciHub, she lives in Russia and has said she plans to ignore the lawsuits. Blocking SciHub doesn’t work either, it switches domains and mirror sites.  It is kind of like playing online Whack-a-Mole, you hit one site and another pops up elsewhere.

Well it seems as if things in Russia might be changing.  According to an article in Chemistry World, SciHub is now blocked in Russia following a Russian court ruled against the site.  Moscow City Court ruled the site should be blocked in Russia following complaints from Elsevier and Springer Nature over intellectual property infringement.  As a result of the ruling, several Sci Hub and Library Genesis domains are now inaccessible by Russian internet service providers.

An article from says “Sci Hub is no to blocking efforts” and probably has other domains “up its sleeve.” TorrentFreak says those other domains can be targeted by rights holders and Elbakyan is encouraging users to “use tools to circumvent Internet censorship – which you can search for in Google or by using the bot in Telegram.”

So it looks like Elbakyan and Sci Hub may be touchable in Russia after all. I don’t think Elbakyan is a hero nor a villain. My hope is that through all of this Sci Hub  evolves like Napster did. Napster is now into the on-demand streaming business. Even if we aren’t using Napster directly as our streaming service, many  are using the pieces of Napster and don’t realize it.  The iHeartRadio All Access app is powered by none other than Napster. It took a Napster to bring about changes in online music. Hopefully Sci Hub can be the catalyst to bringing change to the scientific online paper world so that more papers are easily available and legally.



Predatory Publishers

A recent article in The Guardian “Predatory publishers: the journals that churn out fake science” reported on an investigation (in collaboration with German broadcaster Norddeutsher Rundfunk) into predatory publishers and fake science.

According to the article more than 175,000 scientific articles have been produced by the five largest “predatory open access publishers”  and 5,000 scientists at British universities have published in predatory publications in the last 5 years.  The article mentions that many of the researchers were “exploited by the publishers, who aggressively seek new business from academics who don’t know their reputation.”

Predatory publishing has been on the minds of librarians for quite some time, I often feel like it is old news.  Unfortunately, I think is still new news to many researchers and STEM authors.  I can point to examples of clinicians looking to publish a paper who didn’t even understand the difference between open access and traditional access.  In their mind a journal like NEJM appears open access to them because they are able to access it freely using the library subscription.

So when you have this access perception problem it isn’t hard to see how some can be fooled by predatory publishers.  Their game is more difficult to spot than the Nigerian Prince who just needs you to send him $1,000 for you to receive $10,000.  The problem isn’t just with publications. There at predatory conference promoters.  Back in May I posted about receiving an invitation to speak at a conference in China.  Considering I have been asked to speak in Ireland and other places it isn’t all that far fetched to think somebody from China would be interested.  After getting my hopes up momentarily, careful review led me to realize this was predatory conference spam mail.

I think as librarians we need to remember that there are still many authors who are unfamiliar with the concept of open access and as a result unfamiliar with appropriate open access article submission guidelines and expectations.  Lists of predatory publishers will come and go, we need to work with people to be able to better identify the red flags.  We need established publishers to step up their game and help with the education process. We need database providers to establish criteria for inclusion, rather than including any research article that was publicly funded.

Only by working as group can we have a hope at turning the tide.



The Donut Hole of Library Access

We have heard of the donut hole for Medicare prescription drug coverage where people experience a coverage gap for their prescription drug coverage.  I think there is a donut hole for medical information. There are doctors, nurses, researchers who are affiliated with an institution (but not officially part of the institution) or they are private practice who have privileges but are not employed by the hospital.  These people often fall in the donut hole for access to medical information.

There are more and more of these people as universities buy hospitals but the university doesn’t/can’t provide library resources to the hospitals.  Hospital systems are buying other hospital systems and wrongly assuming the library resources will be cheaper (bulk discount) or that they don’t have to pay for library resource because “aren’t we all just one system now that you bought us and we want what you have.”

Library resources are expensive and when met with the surprise cost (when a new system is acquired) administrators often do not understand the basics of library resource licensing and costs. Why would they?  The amount of a library’s budget is not even on their radar when it comes to the budget of the entire hospital.  Unfortunately, they want across the board cuts and the department manager must enforce those cuts. We have little data that says medical libraries save lives and save money in the long run.  What little data we do have hasn’t made it to the minds of administrators.

IMHO the donut hole of information access is growing.  I have people who call for access because they are doing research with somebody in my system. I can give the person in my system access but I can’t give the partner, who is outside of my system, access.  This is met with confusion and dismay.  This is one of the reasons ICanHazPDF and SciHub exist.

As hospital libraries are disappearing and budgets are shrinking, the donut hole will get bigger.  The perception that all is available free on the Internet still exists among many people and when hit with the reality they are flabbergasted by the true access costs causing them to dig their heels in deeper and not pay for anything.  I know of 2 hospitals that didn’t have libraries who were seeking to get library resources only to immediately scuttle the idea completely when faced with the costs. No concept of baby steps and ramp up to more resources. Nope, in their minds the costs are just too astronomical for resources they thought would be cheaper because they are tailgating onto another library.  They totally bail and go back to the idea that everyone will just get by use PubMed and Google, or maybe one expensive (but not as expensive as a whole library) point of care tool.

So, how do we stop the donut hole from growing? Pricing isn’t going down. (When does it for anything…cars, houses, etc.?) Our value is not on the minds of administrators. It will probably be a multi pronged approach requiring cooperation from both publishers and librarians.  What are your thoughts?

Medlibs Needs RA21 on Their RADAR

I attended remotely, the RA21 webinar Friday morning and it was interesting.  I hope they recorded it and will make the recording available for everyone because this needs to be on the radar of medical and hospital librarians…now. Those attending MLA in Atlanta there will be session Sunday 4-4:20 Leading Easy Access to Content: RA21 Pilots Transform Researcher Productivity and Privacy in the Hyatt Regency Embassy C.

Why should medlibs care? The publishers are looking to do away with the current method of online resource authentication, IP validation.  There is a whole slew of reasons as to why IP validation has problems, one of the biggest is piracy like SciHub.

What is RA21? The RA21 website goes into further details, but it is basically an initiative to facilitate seamless access to online resources while preventing piracy and improving authentication methods.

This is big, because right now many hospital and medical libraries use IP validation, EZ Proxy or both to authenticate users.  RA21 seeks to eliminate IP validation from your on campus IPs as well as your EZ Proxy.

There are privacy issues that concern many people. I am not going to focus on that. I am going to focus on just implementation issues in hospitals.

I don’t understand all of the technical nuts and bolts to RA21 but here is what I learned from the webinar and why hospital librarians need start paying attention.

  • Publishers are pushing to eliminate IP validation and the method for authentication.  This means you won’t be able to give your hospital’s IP ranges or your proxy server and have your patrons automatically access library resources (without passwords).
    • Patrons will not be able to click and access a resource simply by just being on campus.
  • RA21 will require people to validate themselves and sign into the resources.  So a doctor will click on Wiley’s Cochrane Library and be asked to login, even when on campus.
    • They mention that the doctor will only have to login once because the system will know him/her.  What doctor do you know stays in one place and uses one computer? Doctors will have to login multiple times through out the day.
  • RA21 follows the user not the user’s location. So there will need to be some database of approved users.
    • Librarians will need to maintain that. They are the one who will have to add users and delete users.
    • Some libraries are set up to be able to do this through their ILS patron database. However, other ILS systems can’t share patron database info.
    • Additionally, A LOT of hospital libraries don’t have an ILS, they still have sign out cards!
  • EVERYONE, academic medical libraries and hospitals will need something like Shibboleth or OpenAthens to be able to implement RA21.  This is not good. There are A LOT of hospital libraries who can barely afford their journals let alone OpenAthens or another product to manage online access.
  • Libraries with walk up access via their computers will have to figure out how to time out people.  The doctor is not going to logoff of a journal when they leave.

Now I am admittedly fuzzy on what authentication methods they are using.  Whether they have a database of approved users who have created their own ID and password or they have something else. Some groups seem to be talking about email addresses while other groups talk about login IDs and two factor authentication.

However, every user must authenticate. There will be no more pass throughs via IP. So every time a doctor wants to use an online resource they will have to login.  Now as many hospital librarians know, the doctors are not going to want to login to access an online resource.  I believe I heard one medical librarian say her doctors will have kittens if they had to have yet another username and password to remember just to get journal articles.  We have an online resource that contains both ejournals and ebooks within it.  The ejournal articles allow IP validation to access the PDF. The online resource used to require doctors to use a username and password to access the PDF of the books. Our doctors absolutely refused to use any of the ebooks from that product.  They didn’t want to bother with logging in. Requiring a login to view the PDF of those books impacted usage. IMHO RA21 in a hospital environment will impact usage.

What about single sign-on? RA21 keeps talking about single sign-on. Most likely hospitals will not allow us (or anyone) to tie our library login to their network login.  So there is no real “single” sign-on.  They will need to remember 2 different usernames and passwords, one to get onto the hospital network and one to get library resources. What do you think will happen?  Doctors will use the same network passwords as the passwords for the library.  That’s not good.

Hospitals tend to have extremely locked down IT environments, some hospitals more than others. I know of hospitals that can’t provide off campus access to the ejournals because their IT forbids them from using proxy access (even if it is outsourced and off site).  The folks at RA21 kept talking about working with our IT departments and it is clear that none of them have had to deal with hospital IT.  The hospital IT department does not care about the library. The hospital IT only cares about the EMR and locking everything down as tight as possible from the outside world…including medical publishers.  I know a librarian at a government healthcare agency library that routinely loses access to PubMed due to IT restrictions. Yes, a government healthcare agency library loses access to a government database because the government agency IT has things extremely locked down. So IT is not going to be on board. It isn’t in their interests which is the total security of the hospital network….not STEM piracy prevention or user experience.  This change will fall to the library staff to handle.

Now I agree that IP validation is a flawed system and we need something better.  However, I have concerns as to how it can be implemented in hospital libraries.  Not one of the RA21 Steering Committee is from a hospital library.  They are all big STEM and research and have tested it in the academic library environment.  When I asked for examples of implementation or testing in hospitals I heard nothing.  I don’t think they realize how different hospitals are.  After all, they kept presenting the idea that we can tell IT that it will be a better user experience.  IT is does not care about user experience.

I think your major hospital systems will be able to adapt.  Sure the docs will have kittens about the login requirements and usage might go down because they don’t want to bother logging in for something quick. But I really worry about the hospitals that aren’t big.  I worry about the ones with budgets that are little more than pennies.  I worry about the ones that aren’t allowed to use any outsourced resources to provide journal authentication.  I worry about the solo librarians with no contacts in IT.  How are those hospitals going to handle things?

We need to pay attention so that we can be an active partner in trying to make RA21 or whatever method for authentication something that is feasible for medical libraries.




Library Phishers

I just read the post “Silent Librarian: More to the Story of the Iranian Mabna Institute Indictment” and it was very eye opening.  The United States Justice Department, FBI, New York FBI, and US Treasury announced charges against nine Iranians for conducting a huge cyber theft campaign.  Prosecutors state the nine Iranians worked for the Mabna Institute and stole more than 30 terabytes of academic data and intellectual property from universities, companies, and governments around the world.  That is roughly the equivalent to 8 billion double sided pages.

There were more than 750 phishing attacks identified targeting more than 300 universities in 22 countries, however most the targets were located in the US, Canada, UK, and Australia.  Its not just universities that are getting hit. Medical librarians will recognize these institutions PhishLabs identified as also being targeted, Memorial Sloan Kettering Cancer Center, Ohio State Wexner Medical Center and Thomson Reuters.

The PhishLab post provides a more detailed picture on the impact of phishing campaign which targeted university professors, students, faculty, and medical institutions dating back to 2013.  The phishing attacks profiled are designed to look like emails from the institutions’ email.  The fake emails contained spoofed sender email addresses (making it appear as if it was sent from a legit institutional account) telling the target their library account has been expired and in order to reactivate they must follow the link and login with their credentials.  The URLs for the link in the email are similar looking to the correct institution’s URL.

The example they give: (note the XXXX intentionally redacted data)

  • Legit American University Library URL:
  • Fake URL:

The landing pages of the fake site is identical to the legit site (as shown on the PhishLabs post)

These stolen credentials are then sold online where buyers ask to buy specific university passwords.  Passwords to the “best” universities and rare journals are the most expensive.

Phishing attacks involving the IRS, bank accounts, and credit cards get the most press these days. While I was aware this sort of thing happened in library resources I was unaware to its extent.  It makes you realize why publishers are looking at RA21 as the answer to better authentication.  I’m not sure if RA21 is the answer.  But I will be virtually attending the FREE RA21 seminar Friday April 27th to learn more about it and see what it means for medical and hospital libraries.

Learning more about RA21 and other ways to prevent library phishing is something we need to be more involved in.  On a simple level, perhaps we need to educate our users to call us directly (like they do the banks) if they have a question about a library email account.



Submissions Wanted: JMLA Virtual Projects Section

Submit Virtual Projects for JMLA Virtual Projects Section by March 31, 2018

The Journal of the Medical Library Association (JMLA) Virtual Projects Section Advisory Committee is seeking current, innovative and notable technology projects in health sciences libraries for the 2018  JMLA Virtual Projects Section. The previous Column format for JMLA Virtual Projects is transitioning into a Section format this year, which will appear on an annual basis in the October issue of JMLA.

To be considered for the Virtual Projects Section, please submit a 200 word abstract of your virtual project, including why it is innovative/notable, and provide a link to your project web page (if possible) that describes or demonstrates the project. Send your submissions to Susan Lessick, AHIP, FMLA,, by MARCH 31, 2018.

Technology projects must have been implemented within the past two years. Submissions of virtual projects may demonstrate either the implementation of a new technology or a new application of an older technology. Focus areas for technologies of special interest to the Committee include (but not limited to) the following:

  • artificial intelligence (AI) and AI-enabled applications
  • blockchain technology for libraries
  • augmented reality applications
  • open textbooks initiatives
  • research impact tracking tools
  • social media outreach
  • voice control technology (Amazon Echo, Apple Siri devices, etc.)
  • wireless charging
  • chatbots or intelligent agents
  • patron satisfaction tracking technologies

Please consider sharing your knowledge and experience with implementing virtual projects in your library to inspire and encourage your peers, partners, and communities!

JMLA Virtual Projects Section Advisory Committee:

Patricia Anderson
Emily Hurst, AHIP
Michelle Kraft, AHIP
Susan Lessick, AHIP, FMLA
Dale Prince, AHIP
Elizabeth Whipple, AHIP

Hiring a new librarian? Are new graduates qualified?

Believe it or not some librarians are retiring and some libraries are hiring. I know, I heard the same story 20 yrs. ago in library school about the wave of librarians retiring and the need to hire a bunch of librarians to fill those open positions.  Instead of a giant wave of retirements, I think it has been a gentle rise over time.  Instead of filling every single open position retirement brought, I think there has been closing of libraries, not filling positions, or restructuring positions for a different type of fill.  However, not all positions will be left unfilled.  I know of a library that will probably have at least 4 people retire sometime in 5 years.  I don’t know if all 4 positions will be filled, but I know for certain that they won’t all go unfilled.

So this leads me to think about who will fill these positions and the others positions in the medical library world.  In the past when we have had open positions, the number of librarians applying without any library experience (volunteer, practicum, library assistant, etc.) has been large. The number of people applying without medical, health, biology, etc. (basically anything related to medical) library experience has also been large.

I realize the experience part is difficult to come by when there are few library jobs out there.  That is why I am always interested to see if the person had a volunteer position, practicum, internship….something that gives them an idea of what working in a library is like.

Twenty years ago (gah I can’t believe it has been that long) when I was in library school, cataloging was a required course. The same held true with reference. Database searching was elective, but dude…. I totally knew I had to take that class.  After seeing the resumes and speaking with some graduates I also am very concerned about what is taught in library school.  I know there are people who  graduated with a library degree who had never taken cataloging or reference.  IMHO those are the very basics of a library education and form the backbone of what you need to build upon as librarian…no matter what librarian you become (subject or position).

That is just the education for regular librarianship. I haven’t even gotten into the skills and knowledge necessary for medical librarianship.  Medical libraries (like many other special libraries) do things a bit differently. We don’t do ILL like everyone else (Docline).  We catalog differently (NLM Classification). Our reference is all medical and health issues… which is often not taught library school because it is still viewed as verboten in public libraries.

Those are just some of the easy, off the top of my head examples of things that unless you worked in a medical, health science type of library you would be totally unfamiliar with.

So as I look toward the future, I am wondering what other medical librarians are looking for when they are looking to hire an entry level librarian and do they feel the library schools are producing graduates that meet our needs?  Let me know what you think?  What is essential in a librarian? What kind of internships, practicums, volunteering are helpful?  If you offer internships, practicums, volunteering what are the basics they need to know before hand?

Comment your thoughts.

My First Year as Director: Things I Have Learned

I officially became the Director of my library in September of 2016, but I shadowed the old director until November.  So to me I kind of count November as to when I really became the Director. The time has gone by quickly. I have learned a lot of things.  I thought I would share them, just as I shared about what I learned as President of MLA.

So here are a few things I learned:

You Need Friends:
No matter if you are new to an institution or have been there for 20 years (as I have) there will be certain departments in your institution that will befuddle you and they will always befuddle you.  At some institutions, you have a better chance discovering the meaning of the four messages on the Kryptos sculpture than knowing how parking is assigned and getting a good lot/garage/spot can be akin to winning the lottery.  While at other institutions, sending a license agreement off for review to the legal department may often feel like you sent it to the Bermuda Triangle.  Every institution has its own version of Catch 22 policies and departments and you need to find friends in those departments, preferably several.  Those people will save you.  They may not be able to fix the problem themselves, but they can often get you out of no man’s land and connect you with somebody who can fix things. (Unless it is parking…nobody can ever fix parking.)

Thank yous:
This sounds like a no brainer. However, a simple thank you note or email, depending on the situation, is not only the nice and right thing to do, but it will help you win more friends.  (Remember the paragraph above about friends)  Based off my observations the art of sending a thank you is disappearing.  Perhaps we are busier than we were in the past and don’t think about it.  Maybe some of us didn’t have their mom standing over us when we were 7 yrs old writing thank you notes from our birthday party.  Some say technology is to blame.  Whatever…just send a quick thank you via email if somebody helped you out.  If they really went above and beyond, send a thank you card.

Transitioning from co-worker to boss:
This can be sticky if you are a new boss to a new institution and even stickier if you are the new boss in an institution you’ve been in for a long time.  There are lots of books on this topic. I get weekly emails from the site From Bud to Boss, while it is primarily set in the business world there are some helpful things.  I recommend working on this transition either by reading some of these books or talking to another library director you know well.  You are going to have times where you are going to have to have a difficult conversation with an employee.  I have found the book Crucial Conversations helpful and practicing the conversation with either HR or a trusted library director can be beneficial.  While it is fine to get your stuff together and practice, do not put off the conversation longer than you need to.

Going from an employee focused on your own job, to a manager or director who is focused on your own job plus everyone’s jobs means your time gets squeezed.  There are certain things that you used to do that you just don’t have time for and should be delegated to somebody else.  These things could be ordinary tasks like ordering supplies or doing the desk schedule. They could also be specialized.  For example, you may no longer be the best person to do a systematic review.

In the past, I had ideas that I would be super awesome at doing systematic reviews, I would learn how to do research, I would manage the library’s systems (web design, link resolvers, etc.) and my spare time I would devote to finding the latest and greatest resources. Ha. I don’t have time for any of that now.  Now is the time to find people who are interested in those things and encourage them and provide support for them to take on those roles for the library.

Find a mentor:
Find a library mentor (another director or manager) and find an institutional mentor.  Both are equally important.  The library mentor will help you with the library things that normals just don’t understand.  An institutional mentor will help deal with the quirks within your institution.  You don’t feel alone when you have a good mentor.  Chances are they have been in the same situation.

Make time for you:
There is always going to be something that needs your attention.  I encourage disconnecting when possible.  But it isn’t possible all of the time. For example, I have to do payroll on Christmas Eve (they switched it earlier so we didn’t have to do it on Christmas day).  Find your balance with technology in order to find your balance with your work and home.  Some people will absolutely never answer emails when they are at home.  I do.  I find if I am able to answer them on a regular basis then they don’t pile up and I don’t have to spend more time at work catching up and answering them.  If I’m not spending time at work catching up on my email then I am able to spend time at work doing the other things that need to get done.  The important thing is find what works for you to maintain a healthy work and home life balance.

I learned a lot more but these are some of the biggies that apply to everyone.  Here is to a new year, may 2018 bring us happiness, joy, and good challenges.


Concerned about Net Neutrality Contact Congress

I’m sure you have all heard about the FCC not taking (or looking) at any public comments regarding Net Neutrality.  However, you can still voice your opinion on Net Neutrality by contacting your members of Congress.  It is particularly important to contact them if your Congress person is on one of the committees that oversees the FCC. (Go to links highlighted below, click “about” and find out if your state and Congress person is listed)

(I am re-posting what I was sent from the MLA/AAHSL Legislative Task Force)

Through the ALA Washington Office we have learned that last week, Federal Communications Commission (FCC) Chairman Pai announced plans to dismantle network neutrality protections approved by the FCC in 2015 and affirmed by the federal appeals court in 2016. The new draft order is scheduled to be voted on by the five FCC commissioners on December 14th.

Why Net Neutrality Matters:

MLA and AAHSL support the net neutrality protections approved by the Federal Communications Commission in 2015 and affirmed by the federal appeals court in 2016. Net neutrality is critical to libraries, library patrons and the public.

Health sciences libraries require an open internet to provide

  • unencumbered access to the National Library of Medicine’s (NLM) almost 300 databases that support health care, education, and research; and
  • Internet access to images that support telemedicine.

The public requires Internet access without restrictions and barriers to access consumer health information.

Libraries depend on the principles of net neutrality which allow them to create and make available on their websites content that supports educational opportunities online worldwide and to provide access to datasets to promote research and collaboration.

What You Can Do (before December 13th):

As the ALA Washington Office reports, right now, the FCC is not accepting public comments (that may come later), but strong disapproval from members of Congress (especially from those who serve on committees with oversight for the FCC) could force a pause in the December 14 vote to derail net neutrality. Make your voice heard now by emailing your member of Congress ( and ( to support net neutrality protections.

Links below are provided to the following House and Senate Committees and Subcommittees with jurisdiction over the FCC.

For talking points see:

MLA/AAHSL Comments to the Federal Communications Commission re: Restoring Internet Freedom,” Docket 17-108 (July 14, 2017) and feel free to personalize your letter by addressing the impact of this potential rollback on your libraries and users.

Here are some other good video (non medical) explaining why Net Neutrality is important.

Systematic Review Search Strategy Development: (Very Nearly) A Thing of the Past?

A guest post by Rachel Pinotti, MLIS, AHIP

Recently, a faculty member sent me a copy of a June 2017 editorial published in Annals of Internal Medicine entitled Computer-Aided Systematic Review Screening Comes of Age along with the article which it accompanied.  The editorial argues, in short, that machine learning algorithms generate superior results to human-designed search strategies.  It asks (and answers), “Is it time to abandon the dogma that no stone be left unturned when conducting literature searches for systematic reviews? We believe so, because it has a deleterious effect on the number and timeliness of updates and, ultimately, patient health.” (Hemens & Iorio, 2017)

As a librarian who conducts, consults, and teaches systematic review searching, this unleashed a flood of thoughts and questions.

On a philosophical level, these authors’ thesis raised a real tension that I feel with regards to so many topics I teach about: the tension between teaching students about the way things are now vs. the way they very likely will be in the near-to-medium term future. As of now, I don’t think GLMnet and GBM, the machine-learning algorithms utilized in the original article which the editorial accompanies (Shekelle, Shetty, Newberry, Maglione, & Motala, 2017) are widely utilized for systematic review searching, but they quite possibly may be in 3-5-7 years’ time (or less).  Are students better off learning to design and execute comprehensive search strategies, a skill that will serve them in the immediate term and perhaps a few years hence or better off learning how to use GLMnet and GBM, tools that may come into wide use a few years from now?  The answer is probably that they are best off learning both.  Unfortunately I don’t know of anyone within my institution who could teach the current cohort of students these new tools.  (Maybe such people exist and I’m not familiar with them, maybe they don’t exist, or maybe they exist but exercise their skills exclusively for research, not teaching purposes….)

Even once these tools come in to wide use, I wonder if teaching students to design and execute comprehensive search strategies is a bit like teaching them long division – not something they are likely to use frequently or maybe ever in their day to day work, but you need to learn long division in order to understand the concept of division so you understand what is happening when you type 48756/38 into a calculator (or enter your initial search terms into a machine learning search tool).

On a practical level, a big concern with machine learning algorithms is whether they are able to effectively handle multiple information sources and grey literature?  Shekelle indicates, “Although initial results were encouraging, these methods required fully indexed PubMed citations.”  The algorithms could likely be adapted for Embase and other databases, though this might require permission from database providers. Grey literature (conference abstracts, theses, etc.) often does not have complete abstracts and almost by definition is not fully indexed.  Excluding grey literature from a systematic review or meta-analysis introduces a real risk that publication bias will produce a biased result, as documented by McAuley, Pham, Tugwell, & Moher, 2000 and others.

I’ve always felt that some of the best practices recommended in systematic review searching, such as the recommendation to use both index terms and keywords, are redundant and unhelpful.  So I don’t doubt – and on the contrary am quite receptive to –the argument that current best practices in systematic review searching favor sensitivity far too greatly over precision. Some filters (e.g. language) are much more effective than others (e.g. age, gender). I absolutely think that filters, especially Clinical Queries, are useful when you need to find the few most relevant articles on a topic, but do not believe that they are yet able to unearth all the studies that would have a bearing on a topic, as suggested by Hemens.

The editorial asserts, “Simple Boolean filters applied when searching increased precision 50%…while identifying 95% of the studies…” (Hemens & Iorio, 2017) with an underlying, unstated assumption that these results would hold true across fields and topics. Thinking of this in evidence-based medicine terms, these articles present case-series level evidence for the effectiveness of machine learning search algorithms.  Before changing our standard of practice, I would like to see larger scale studies (perhaps cross-over studies in which the same review is conducted both using evidence found through machine learning and traditional search methods) which indicate that machine learning search tools are as effective or more effective than traditional search methods before advocating that they be widely adopted.

The ideal outcome would be getting to a point where machine learning is used to effectively search traditional published material allowing reviewers to focus their energy on searching grey literature. However until we reach the point where these methods have been validated and can be widely used and taught, current best practice remain just that: the best methods for unearthing all evidence that may have a bearing on a given research topic.

Rachel Pinotti, MLIS, AHIP
Assistant Library Director, Education & Research Services
Levy Library, Icahn School of Medicine at Mount Sinai
Box 1102 – One Gustave L. Levy Pl.
New York, NY 10029-6574

Email: rachel[dot]pinotti[atsign]mssm[dot]edu
Phone: available via MLA members list

Follow us on Twitter @Levy_Library to learn about Levy Library events and initiatives.


Hemens, B. J., & Iorio, A. (2017). Computer-aided systematic review screening comes of age. Annals of Internal Medicine, 167(3), 210-211. doi:10.7326/M17-1295

Shekelle, P. G., Shetty, K., Newberry, S., Maglione, M., & Motala, A. (2017). Machine learning versus standard techniques for updating searches for systematic reviews: A diagnostic accuracy study. Annals of Internal Medicine, 167(3), 213-215. doi:10.7326/L17-0124

McAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000). Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? The Lancet, 356(9237), 1228-1231. doi: