Yes? Or No? Or HOW? Catching a Predator at Birth (Maybe)

Catching a Predator at Birth

I almost called this post: “Create attention for your article; write a layman’s summary,” which was the subject line from the e-mail we are discussing locally in trying to decide if it is a predatory publisher or not. (Short version of what we did for those who don’t have time to read the whole story: Identity, Authority, Credibility, Language, Editing, Timing, Licensing, Accessibility, Openness, Sources, Resources. Basically, defining a chain of trust.) I’ve blogged here before about the idea of layman’s summaries, a.k.a. plain language abstracts. They have a great tagline. It’s a great idea. My first reaction was, “How can we help?” Obviously, I think the idea is awesome, and I’ve thought so for a very long time, many years. I am far from the only person to think so. Just take a quick look at these few selected quotes.

DC Girasek: Would society pay more attention to injuries if the injury control community paid more attention to risk communication science?
“We also need to call attention to the injuries that continue to take lives, despite the fact that solid solutions for them have been published in our scientific journals. We need research on translating study findings into public action. Epidemiology and engineering remain central to the field of injury control. We must look to the social and behavioral sciences, however, if we hope to overcome the political and cognitive barriers that impede our advancement.”

Alan Betts: A Proposal for Communicating Science
“Given that the future of the Earth depends on the public have a clearer understanding of Earth science, it seems to me there is something unethical in our insular behavior as scientists.”

Jason Samenow: Should technical science journals have plain language translation?
“Some scientists might resist the onus of having to write a lay-person friendly version of their articles. However, I agree with Betts, it’s well past time they do so”

Chris Buddle: Science outreach: plain-language summaries for all research papers
“1) Scientists do really interesting things.
2) Scientists have a responsibility to disseminate their results.
3) Scientists do not publish in an accessible format.
This is a really, really big problem.”

Chris Buddle: A guide for writing plain language summaries of research papers
“A plain language summary is different because it focuses more broadly, is without jargon, and aims to provide a clear picture about ‘why’ the research was done in additional to ‘how’ the work was done, and the main findings.”

Lauren M. Kuehne and Julian D. Olden: Opinion: Lay summaries needed to enhance science communication. PNAS 112(12):3585. doi: 10.1073/pnas.1500882112
“But rather than an unrewarding burden, scientists (and journal publishers) should consider widespread adoption of lay summaries—accompanying online publications and made publicly available with traditional abstracts—as a way to increase the visibility, impact, and transparency of scientific research. This is a particularly important undertaking given the changing science media landscape.”

This is seen as SUCH an important idea that multiple grants were provided to create a tool to assist scientists in doing this well!

Center on Knowledge Translation for Disability and Rehabilitation Research (KTDRR): Plain Language Summary Tool ((science OR research) (attention OR “plain language” OR “clear language” OR layman OR journalist) (summary OR abstract)

Imagine my excitement when a colleague (many thanks to Kate MacDougall-Saylor) alerted me to a new online publication specifically for this purpose! How PERFECT for Health Literacy Month! A faculty member had asked her if it was a legitimate enterprise. So we looked at the email she’d received, and at the web site.

Dear Dr. XXX,

We are interested to publish the layman’s summary of your research article: ‘ABC ABC ABC.’ on our website.

The new project ‘Atlas of Science‘ started from 1st October 2015. It is made by scientists for scientists and the aim of the project will be publishing layman’s abstracts of research articles to highlight research to a broader audience.
Scientific articles are often difficult to fathom for journalists, due to the scientific jargon.
Although journalists like to assess the news value quickly, that is by no means simple with most research articles. Writing a short, understandable layman’s summary is a good means to reach this goal.

This makes sense, has a good message, and is accurate about the potential impact so far, but the English doesn’t read as having been written or edited by a native speaker of English, and the formatting is inconsistent. It doesn’t look as if a professional editor did a final review before promoting to the world. Warning Sign #1.

The name of the web site (Atlas of Science) is identical to the highly regarded book from MIT Press and authored by Katy Börner of the Indiana University Cyberinfrastructure for Network Science Center. At first, I thought perhaps they were connected, but quickly realized this was a separate group, simply using the same name. Warning Sign #2.

Most of the rest of the message came directly from the “For Authors” page on the web site (Why, What, Use), except for the instructions.

Submit
∙ Send your summary to info@atlasofscience.org, not later than ##/#/2015.

What do we do with your layman’s summary?
∙ We check the text, and in consultation with you we dot the i’s and cross the t’s.
∙ Your text will be available on the Atlas of Science website, www.atlasofscience.org .
We will actively promote this site to the press.

Please, let us know if you are interested and do not hesitate to contact us if you have any question (simply reply to this email).

This was less worrisome, except … the phrase “not later than” (combined with a date of just over a week to respond) seems to be pressuring the faculty member to respond quickly, without thinking it through carefully, and without time to actually create a well-done plain language summary. Warning Sign #3.

Speaking of a well-done plain language summary, do they explain how to do what they say they want? We checked on the web site. Not really. They tell you what they want, but not how to do it, and they don’t point people to any resources to help them understand what a plain language summary is, what this means, or how to do it. They define no standards, set no guidelines, make only the barest and simplest recommendations (such as word count — 600 words with 2 figures), and do not even mention appropriate reading level. Warning Sign #4.

Does the posted content on the site actually appear to match the stated goals of the site? Not remotely. The pieces posted don’t even match the minimal guidelines they stated in their own criteria. I tested a few of the newest posts. The titles alone (“Regulation of mediator’s expression and chemotaxis in mast cells”, “Minute exocrine glands in the compound eyes of water strider”, “Gene therapy not just counseling for your denim obsession”, tell you these are not plain language, but just to be fair and unbiased, I ran them through a Simple Measure of Gobbledygook (SMOG) Tool, which is only one of several tools and resources available for assessing readability.

Regulation of mediator’s expression and chemotaxis in mast cells
The SMOG index: 20.1
Total words: 766
Total number of polysyllabic words: 180
Total number of sentences: 41

Over 150 words more than the defined limit for the abstract (Warning Sign #5), and written for an audience with a reading level matching those with multiple graduate degrees. The SMOG Index, you see, displays the reading level by number of years of education. 12 is a high school diploma, 16 is a college degree, 18 is a masters, and 20 is well into PhD territory. The average reading level for adults in the United States is roughly 8th grade, meaning that a really well done plain language summary would be written to a SMOG level of 8, at most 12. 20 is a long ways from 12.

Minute exocrine glands in the compound eyes of water strider
The SMOG index: 16.2
Total words: 461
Total number of polysyllabic words: 70
Total number of sentences: 35

Gene therapy not just counseling for your denim obsession
The SMOG index: 18.7
Total words: 573
Total number of polysyllabic words: 79
Total number of sentences: 23

Save your pancreas from diabetes! Your beta cell reserve is critical for prevention and treatment of diabetes.”
The SMOG index: 19.6
Total words: 455
Total number of polysyllabic words: 100
Total number of sentences: 25

It’s easy to see that most of the authors take the word count seriously, and that some of them genuinely tried to reduce the reading level and had an idea of where to start with this. None of them came anywhere close to an 8th grade reading level, and none of them were below college graduate reading level. Warning Sign #6. The writing in the abstracts was highly variable, some included grammatical errors, and there was no sign of editorial oversight. Warning Sign #7.

You get the idea of how the checking is being done. I don’t want to walk you through the excruciating details for every piece, but here are a few more criteria, and then ending with a surprise reveal.

“About Us”: Can’t tell who they are, either individuals or institution. Improper grammar & punctuation. No contact information. Contact form has email address hidden. Warning Signs 8, 9, 10.

Content Sources: Most links are to RSS feeds from major science news services, not unique or locally produced content. For the unique content, authorship is unclear (is author of the plain language abstract the same as the author of the original article?), buried deep in the page, no editor mentioned, and no contact information given for the presumed authors. The links for the original articles go back to PUBMED, not to the original publisher, and nont of them give the DOI number for the articles. Warning Signs 11, 12, 13, 14, 15.

Licensing: For a project of this sort to have the impact it is supposed to on journalists and the public, it would need to have a Creative Commons licensing structure, presumably with attribution. Instead it has
copyright, all rights reserved,” but gives no information on how to get permission to use the content. It appears that the intellectual property rights are held by the website, not by the actual authors. This is (in my opinion) terrible. Warning Signs 16, 17, 18.

Accessibility: Problems using the site on my phone. Tested desktop view, and there are a number of fatal errors, missing ALT tags, empty links, duplicated links, etc. Sloppy, sloppy coding. Nobody’s perfect, but MEDLINEplus has zero fatal errors, just for comparison. If this is from a reputable organization, I’d expect better. Warning Signs 19, 20, 21.

Now, the big surprise! While I was digging around online, I found some of the content, almost verbatim, from an authoritative site! Virtually all of the “For Authors” page is from the Technishe Universiteit, Eindhoven (TU/e). Evidently, they have or had a requirement for graduate students to write a plain language summary of their research prior to graduation. Brilliant concept! The submitted content was reviewed, edited, and selected for possible inclusion in their university research magazine, Cursor. They also had a campus website to host the content. The link for this was broken when I checked today, but the Wayback Machine has several examples over the past several years, including just a few months ago.

The big question now is whether this project is taking the Technische Universiteit model and making it bigger for the world, or was the content stolen from TU/e? There is no way to tell by looking. If this is a genuine project from TU/e, there are some changes they could make to improve the project. If the project is not theirs, I would really love to see the National Library of Medicine recreate a project like this, but done properly. They’ve proven they can. And there is a genuine need.

Plain Language Summaries for Translation in Science

translationalhand1

At MLA last May, I was walking around the vendor hall, like most of us who attended, I assume. I was on a mission, though. I stopped by every vendor table that had anything to do with publishing or translational science, and talked with them at length about the idea of having plain language abstracts. I’ve been a fan of plain language initiatives for a long time, as evidenced by our library’s Plain Language Medical Dictionary app from some years ago. I wish I could say that I was doing this as a direct result of the PNAS article on the topic published in March, but no such luck. That would have helped make my arguments more compelling, I’m sure. I found the article today, thanks to the National Science Communication Institute retweeting Len Fisher.

A circuitous route, but effective enough to reach me. The article in question was this.

Lauren M. Kuehne and Julian D. Olden. Opinion: Lay summaries needed to enhance science communication. PNAS 112(12):3585–3586. doi: 10.1073/pnas.1500882112 http://www.pnas.org/content/112/12/3585

The article was short and sweet. It talked briefly (very briefly) about alternative modes of science communication, such as social media and blogs, and how they impact on audience, understanding, and adoption of new ideas. The authors then pointed out that these are limited to the few who choose to follow that channel, and it misses the benefits and affordances of mass media channels, a concept which they illustrated with a diagram of how they perceived the connections between the information channels and the audiences. Here’s the gist of it.

Scientists communicate with the public through these channels:
1) Social media and press releases
2) Journalist contacts
3) Lay abstracts
4) Traditional abstracts

The potential audiences are:
1) Public
2) Managers and decisionmakers
3) Scientists in other fields
4) Scientists in your own field

So far so good? There are obviously many more potential audiences as you subdivide these. In my conversations I was rather fond of mentioning insurance companies and agents as critical links in the chain of adopting healthcare innovations who are perhaps more likely to benefit from a plain language abstract. I also talked about the importance of highly motivated patients who take new articles to their clinicians as a recent and influential loop in the information chain that changes practice. For benefits to come through these channels requires not simply that there be a version of the abstract that is in plain language (a lay summary) but also, and equally important, that those lay summaries not be behind a paywall. One of the publishers was absolutely sure their abstracts were not being a paywall, and then when they went to show me, well (ahem), they found they were. As in, the abstracts were locked behind a paywall. Oops.

The most important part of the article’s diagram was the very subtle sideways dashes. Where do the journalists get the hook, the info that leads them to ask more questions and write those mass media articles that reach such large audiences? What triggers the journalist to reach out for those important conversations with the scientists? Well, the press releases, of course. That’s why our organizations work so hard on them. Seeing something posted and reposted on social media is another good way to reach them. But the traditional abstract? Not so much. The traditional abstract is crafted explicitly for other scientists in your field, and only partly for scientists beyond that. Now, a lay summary, a plain language abstract, that has HUGE potential as a way to reach journalists. It’s another marketing tool, beyond being the right thing to do to help patients, or to help get science into the hands of those who actually use it, or to help influence clinical practice and foster more rapid adoption of new discoveries and treatments.