Category: Technology

I almost called this post: “Create attention for your article; write a layman’s summary,” which was the subject line from the e-mail we are discussing locally in trying to decide if it is a predatory publisher or not. (Short version of what we did for those who don’t have time to read the whole story: Identity, Authority, Credibility, Language, Editing, Timing, Licensing, Accessibility, Openness, Sources, Resources. Basically, defining a chain of trust.) I’ve blogged here before about the idea of layman’s summaries, a.k.a. plain language abstracts. They have a great tagline. It’s a great idea. My first reaction was, “How can we help?” Obviously, I think the idea is awesome, and I’ve thought so for a very long time, many years. I am far from the only person to think so. Just take a quick look at these few selected quotes.

DC Girasek: Would society pay more attention to injuries if the injury control community paid more attention to risk communication science?
“We also need to call attention to the injuries that continue to take lives, despite the fact that solid solutions for them have been published in our scientific journals. We need research on translating study findings into public action. Epidemiology and engineering remain central to the field of injury control. We must look to the social and behavioral sciences, however, if we hope to overcome the political and cognitive barriers that impede our advancement.”

Alan Betts: A Proposal for Communicating Science
“Given that the future of the Earth depends on the public have a clearer understanding of Earth science, it seems to me there is something unethical in our insular behavior as scientists.”

Jason Samenow: Should technical science journals have plain language translation?
“Some scientists might resist the onus of having to write a lay-person friendly version of their articles. However, I agree with Betts, it’s well past time they do so”

Chris Buddle: Science outreach: plain-language summaries for all research papers
“1) Scientists do really interesting things.
2) Scientists have a responsibility to disseminate their results.
3) Scientists do not publish in an accessible format.
This is a really, really big problem.”

Chris Buddle: A guide for writing plain language summaries of research papers
“A plain language summary is different because it focuses more broadly, is without jargon, and aims to provide a clear picture about ‘why’ the research was done in additional to ‘how’ the work was done, and the main findings.”

Lauren M. Kuehne and Julian D. Olden: Opinion: Lay summaries needed to enhance science communication. PNAS 112(12):3585. doi: 10.1073/pnas.1500882112
“But rather than an unrewarding burden, scientists (and journal publishers) should consider widespread adoption of lay summaries—accompanying online publications and made publicly available with traditional abstracts—as a way to increase the visibility, impact, and transparency of scientific research. This is a particularly important undertaking given the changing science media landscape.”

This is seen as SUCH an important idea that multiple grants were provided to create a tool to assist scientists in doing this well!

Center on Knowledge Translation for Disability and Rehabilitation Research (KTDRR): Plain Language Summary Tool ((science OR research) (attention OR “plain language” OR “clear language” OR layman OR journalist) (summary OR abstract)

Imagine my excitement when a colleague (many thanks to Kate MacDougall-Saylor) alerted me to a new online publication specifically for this purpose! How PERFECT for Health Literacy Month! A faculty member had asked her if it was a legitimate enterprise. So we looked at the email she’d received, and at the web site.

Dear Dr. XXX,

We are interested to publish the layman’s summary of your research article: ‘ABC ABC ABC.’ on our website.

The new project ‘Atlas of Science‘ started from 1st October 2015. It is made by scientists for scientists and the aim of the project will be publishing layman’s abstracts of research articles to highlight research to a broader audience.
Scientific articles are often difficult to fathom for journalists, due to the scientific jargon.
Although journalists like to assess the news value quickly, that is by no means simple with most research articles. Writing a short, understandable layman’s summary is a good means to reach this goal.

This makes sense, has a good message, and is accurate about the potential impact so far, but the English doesn’t read as having been written or edited by a native speaker of English, and the formatting is inconsistent. It doesn’t look as if a professional editor did a final review before promoting to the world. Warning Sign #1.

The name of the web site (Atlas of Science) is identical to the highly regarded book from MIT Press and authored by Katy Börner of the Indiana University Cyberinfrastructure for Network Science Center. At first, I thought perhaps they were connected, but quickly realized this was a separate group, simply using the same name. Warning Sign #2.

Most of the rest of the message came directly from the “For Authors” page on the web site (Why, What, Use), except for the instructions.

Submit
∙ Send your summary to [email protected], not later than ##/#/2015.

What do we do with your layman’s summary?
∙ We check the text, and in consultation with you we dot the i’s and cross the t’s.
∙ Your text will be available on the Atlas of Science website, www.atlasofscience.org .
We will actively promote this site to the press.

Please, let us know if you are interested and do not hesitate to contact us if you have any question (simply reply to this email).

This was less worrisome, except … the phrase “not later than” (combined with a date of just over a week to respond) seems to be pressuring the faculty member to respond quickly, without thinking it through carefully, and without time to actually create a well-done plain language summary. Warning Sign #3.

Speaking of a well-done plain language summary, do they explain how to do what they say they want? We checked on the web site. Not really. They tell you what they want, but not how to do it, and they don’t point people to any resources to help them understand what a plain language summary is, what this means, or how to do it. They define no standards, set no guidelines, make only the barest and simplest recommendations (such as word count — 600 words with 2 figures), and do not even mention appropriate reading level. Warning Sign #4.

Does the posted content on the site actually appear to match the stated goals of the site? Not remotely. The pieces posted don’t even match the minimal guidelines they stated in their own criteria. I tested a few of the newest posts. The titles alone (“Regulation of mediator’s expression and chemotaxis in mast cells”, “Minute exocrine glands in the compound eyes of water strider”, “Gene therapy not just counseling for your denim obsession”, tell you these are not plain language, but just to be fair and unbiased, I ran them through a Simple Measure of Gobbledygook (SMOG) Tool, which is only one of several tools and resources available for assessing readability.

“Regulation of mediator’s expression and chemotaxis in mast cells”
The SMOG index: 20.1
Total words: 766
Total number of polysyllabic words: 180
Total number of sentences: 41

Over 150 words more than the defined limit for the abstract (Warning Sign #5), and written for an audience with a reading level matching those with multiple graduate degrees. The SMOG Index, you see, displays the reading level by number of years of education. 12 is a high school diploma, 16 is a college degree, 18 is a masters, and 20 is well into PhD territory. The average reading level for adults in the United States is roughly 8th grade, meaning that a really well done plain language summary would be written to a SMOG level of 8, at most 12. 20 is a long ways from 12.

“Minute exocrine glands in the compound eyes of water strider”
The SMOG index: 16.2
Total words: 461
Total number of polysyllabic words: 70
Total number of sentences: 35

“Gene therapy not just counseling for your denim obsession”
The SMOG index: 18.7
Total words: 573
Total number of polysyllabic words: 79
Total number of sentences: 23

“Save your pancreas from diabetes! Your beta cell reserve is critical for prevention and treatment of diabetes.”
The SMOG index: 19.6
Total words: 455
Total number of polysyllabic words: 100
Total number of sentences: 25

It’s easy to see that most of the authors take the word count seriously, and that some of them genuinely tried to reduce the reading level and had an idea of where to start with this. None of them came anywhere close to an 8th grade reading level, and none of them were below college graduate reading level. Warning Sign #6. The writing in the abstracts was highly variable, some included grammatical errors, and there was no sign of editorial oversight. Warning Sign #7.

You get the idea of how the checking is being done. I don’t want to walk you through the excruciating details for every piece, but here are a few more criteria, and then ending with a surprise reveal.

“About Us”: Can’t tell who they are, either individuals or institution. Improper grammar & punctuation. No contact information. Contact form has email address hidden. Warning Signs 8, 9, 10.

Content Sources: Most links are to RSS feeds from major science news services, not unique or locally produced content. For the unique content, authorship is unclear (is author of the plain language abstract the same as the author of the original article?), buried deep in the page, no editor mentioned, and no contact information given for the presumed authors. The links for the original articles go back to PUBMED, not to the original publisher, and nont of them give the DOI number for the articles. Warning Signs 11, 12, 13, 14, 15.

Licensing: For a project of this sort to have the impact it is supposed to on journalists and the public, it would need to have a Creative Commons licensing structure, presumably with attribution. Instead it has
copyright, all rights reserved,” but gives no information on how to get permission to use the content. It appears that the intellectual property rights are held by the website, not by the actual authors. This is (in my opinion) terrible. Warning Signs 16, 17, 18.

Accessibility: Problems using the site on my phone. Tested desktop view, and there are a number of fatal errors, missing ALT tags, empty links, duplicated links, etc. Sloppy, sloppy coding. Nobody’s perfect, but MEDLINEplus has zero fatal errors, just for comparison. If this is from a reputable organization, I’d expect better. Warning Signs 19, 20, 21.

Now, the big surprise! While I was digging around online, I found some of the content, almost verbatim, from an authoritative site! Virtually all of the “For Authors” page is from the Technishe Universiteit, Eindhoven (TU/e). Evidently, they have or had a requirement for graduate students to write a plain language summary of their research prior to graduation. Brilliant concept! The submitted content was reviewed, edited, and selected for possible inclusion in their university research magazine, Cursor. They also had a campus website to host the content. The link for this was broken when I checked today, but the Wayback Machine has several examples over the past several years, including just a few months ago.

The big question now is whether this project is taking the Technische Universiteit model and making it bigger for the world, or was the content stolen from TU/e? There is no way to tell by looking. If this is a genuine project from TU/e, there are some changes they could make to improve the project. If the project is not theirs, I would really love to see the National Library of Medicine recreate a project like this, but done properly. They’ve proven they can. And there is a genuine need.