YOUR METHODS AND MINE
“You can’t trust scientists—they’re just as corrupt as the news.”
My ex-girlfriend said this on our fourth date. I finished explaining what one of my previous college classes was about, Research Methods. It’s a class that teaches how to search for scientific papers on databases, understand the technical jargon used in papers, and how to identify a good experiment from a poorly conducted one.
She went on: “Scientists fudge results all the time. And if they don’t, then the journals that publish those papers are funded by people who want certain results. Science is all about what people want you to see.”
I sighed when hearing all this, mostly because of the irony. I’m betting she obtained this opinion from—you guessed it—the news, which is usually shared through YouTubers, family members, and social media—the epitome of “what people want you to see”. I doubted she came to this conclusion from reading actual research.
Which, to her credit, isn’t a fair thing to ask anyone to do.
Most scientific papers are not publicly available. Reading research is both a tedious and job-specific pursuit: either you’re in a field that requires keeping up with the latest findings, or you’re a college student with access to freely available databases (if not, it’s costly to subscribe for access).
Even if there was free access to academic research, it’s unreasonable to expect someone to understand papers because of the way they are written. There’s a specific structure and language that makes sections like a study’s Abstract, Introduction, Methods, Results, and Discussion difficult to understand. These seem like neat, self-explanatory section headings, but each requires a knowledge of how experiments are conducted to analyze procedures and statistical findings.
Scientific papers are the equivalent of the secret language similarly used by banks and the investment community. The comedian Lewis Black once joked about an economics class he took at Princeton University: “Are you trying to keep this shit a secret?!”
In other words, academic papers are written for the audience who best understands them: Other academics.
That’s why the news is an important communicator of scientific research. Scientific papers are like a temperamental uncle that no one wants to deal with at Thanksgiving, and the news is like that one aunt who acts as a negotiator between the uncle and the rest of the family. Without the aunt, no one would enjoy a peaceful meal of turkey and stuffing without feeling angry about what the uncle’s problem is. Believe me: we’re all grateful for the aunt that makes Thanksgiving work for everyone.
Except, the news isn’t always the best communicator. Sometimes, there’s a disconnect between what scientists and news media companies deems newsworthy. The problem often stems from scientists believing the public wants to hear all the fine details of their work (just read a 20-page study without falling asleep) and news companies finding most research uninteresting or not newsworthy.
Depending on the news company, this leads to some liberties being taken. Some change the details of a scientific paper, even the conclusions, to make sense for a broader audience. This is often done by accident, mostly from misunderstanding the results from a complicated study. Of course, after a story is published with incorrect reporting, the scientific community scrambles to fix the errors (without access to a wide audience), and an icy relationship forms between reporters and scientists who don’t see eye to eye (the uncle and the rest of the family).
Or, more nefariously, a news organization might purposely provide a clickbait title or change the content of a research paper for more views and web traffic.
A headline might run like this:
The study might have been about dogs who are measurably annoyed with their owners over being fed certain types of dog food, which results in a doggy grudge being held for a time, but this isn’t the same as dogs “hating” people. But it’s a much juicier headline to make a blanket statement that dogs, suddenly, hate people.
So, why would my former girlfriend hate on science so much? It seems like news companies share more of the blame when it comes to misinformation. Besides, I was taking a college course at the time that told me how mistaken she was.
The relationship ended long ago, but I haven’t changed my mind about the integrity of scientific journals and research.
The truth is that it’s damned hard to be fraudulent in the scientific community. With publishing, not only is the reputation of the published scientist on the line, but so are the careers of those who are asked to peer-review the paper. It’s a huge risk to publish fake findings.
And then I read a headline this morning from The Wall Street Journal:
“Flood of Fake Science Forces Multiple Journal Closures“
***
CAN’T TRUST SCIENCE
Nidhi Subbaraman’s article in The Wall Street Journal covered the recent influx of fraudulent scientific papers that have spread through the academic publishing world. Like an internet virus, hundreds of submissions flooded scientific publications for the purposes of misinforming from credible news sources. Subbaraman notes the authors of these fictional papers are not scientific researchers fudging the findings of their experiments, but are the work of independently paid writers from paper mills.
Paper mills are freelance groups or individuals that produce content for pay. An advertisement might be posted about a prospective paper with an attractive title (like: The Science of Dogs Hating People) and a paper is then written to that point. The papers are forged as credible by citing established scientists’ and researchers’ work, often to look like a well-researched paper. These papers are convincing enough to survive the submission process of many scientific journals.
The large influx of submitted and published papers has led to the closing of many science journals because of the fraud. Wiley, the company covered in Subbaraman’s article, has lost $35 to $40 million in revenue and will be closing 19 of its journals. Paper mills often target the weakest journals that have less scrutinous publishing standards. A publisher like Wiley owns and operates more than 2,000 journals under its publishing umbrella, which means peer-reviewing is lax in less prominent publications. This allows paper mill submissions to slip past proofreaders, especially if a journal is under-staffed.
This news is discouraging but not unexpected. Every industry has been preyed upon by those seeking a quick dollar (see: the continued attempts to short GameStop’s stock over the past week). However, what’s occurring in the academic publishing world points to a bleak future when it comes to safeguarding historically serious places for scientific writing and discussion.
My question is this: If the scientific publishing world is becoming more easily exploited, how can we ever let our guard down?
Maybe my ex was right.
***
PEER-REVIEWED PROGRESS
“Where do you get your information then?” I asked my ex on that date. “How do you know when to trust something?”
She paused for a second, shrugged, and said, “You just know it.”
It was at this point that I knew our relationship wouldn’t last. If my psychology background proved useful for anything it was in understanding that our intuitions are anything but accurate. The writer Kathryn Schulz’s book Being Wrong: Adventures in the Margin of Error is about how even our simplest assumptions are horribly (and often humorously) mistaken.
It’s tempting to rely on intuition in an era of conflicting news sources. I get it: misinformation feels exactly like betrayal—it hurts the same way as finding out that a close friend was lying to you. When betrayal happens, it’s easier to build up walls so that nothing else that’s harmful can get through.
When I think of why my ex-girlfriend—or anyone—actively distrusts science, it’s often about preservation more than protection. Maybe it feels justifiable to shout down or marginalize anything that has harmed us before, but it isn’t a great path forward.
That’s why the response to misinformation cannot be an existential turn inwards to judge the world from a place of safety. To do so is not only impossible but contradictory.
For example, claiming technology and news isn’t to be trusted makes little sense when we cling to devices and online services for every other area of our lives. In other words: We can’t choose to disparage one source for another when they are intrinsically the same.
We all get our news somewhere—who is to say we’re not getting it wrong all the time?
I know—the elephant in the room is how we move forward when it appears we’re being swindled at every turn.
I think the response by Wiley, the publisher in Subbaraman’s article, is a model for how to do so. In the face of a crisis over academic credibility, the publisher began restructuring the submissions program, invested in AI proofreading software to spot signs of recognizable fraudulent content, and conducted audits into the past two years of published papers.
This is what a system working to correct itself looks like.
The Wall Street Journal story is less about the faultiness of our technological world and is, instead, a tale about science behaving as it is supposed to in the face of new problems.
And that’s the key to living a better life: Learning to identify faulty information and responding wisely next time.
Like when dating someone who doesn’t believe in the credibility of science journals.
You live and learn to choose better the next time around.
***
- “Treat Ya Better” by The Criticals
- “City Lights” by Hall Johnson
- “Cameras & Coastlines” by Smallpools
***
Wishing everyone as well as you can be. You’re not alone out there,
