How to read about science
This is a short overview of a good way to start interpreting the science news you hear and see in the media. If you want a more in-depth look, try the short college course videos in the link section.
First, a few generalizations.
The media -TV, newspapers, bloggers, etc – make money by having people go to their site, watch their channel, or buy their papers, so anything that gets you there is fair game to them. They grab you by putting up a header that’s offensive, shocking, or crazy that makes you want to stay tuned, or click to the page just to yell at the TV or write an angry comment.
Often, as you might have noticed, the rest of the information isn’t nearly as exciting or offensive as they made it seem. Sometimes, however, you’re still upset. But what you have to do is take a deep breath and a step back to remove the emotions. Think about the information from a more detached point of view.
Next, look at where the information is from.
What is the main message? This may not be in the headline, and can even be as far down as the last paragraph.
Where did they get the ‘facts’ they are using? Was it a scientific paper, an interview from some ‘expert’ or are they not telling you? If it’s just some person’s opinion, then you’d skip down to the “author” section.
Once you find the article or paper they used, or if you’re starting with the article, then you want to know how reliable is the journal or magazine it’s in. Is it a well respected, peer reviewed journal referenced by thousands of other writers, or is it a three page handout printed in someone’s basement? If you’re not sure you can google the journal’s “impact factor” which tells you how much other people in the same field use papers in the journal in their own writing.
Also, you want to make sure it’s in the right type of journal. You don’t want to read about ear infections in the journal of outer space.
What kind of study is it?
Not all studies are created equal. The most reliable one is a double blind randomized controlled trial, and the least reliable is expert opinions or anecdotes (someone’s experience). Also, in experiments the ‘power’ can be very important. This is how many subjects a study needs to show a true result.
The type of study done has to do with the question they’re asking, how common is the thing they’re looking at, ethics, and funding.
Who is(are) the author(s)?
Who is the author? Is there more than one?
What are their credentials? This includes what sort of degree they have and from where, where they work now, how long they’ve been in this field, how many other papers/books have they written, and do they have any honors or awards given them by others in their field?
Do they have any conflicts of interest? This can occur when the study was paid for by a company who would like a certain result, or if the author has a stake in the company that paid for it, or works for the company that paid for it. Also there can be personal bias, which can make an author go so far as to fake data if they’re intent enough on a certain result.
What does the rest of the scientific community say?
Despite what they like to portray in the media, science is a community and they like consensus. Put a few experts in a room together, and while they may not completely agree on all the facts in a topic, they will agree on the majority.
Usually there will be several articles written about any one subject. If it’s truly new or unusual then there will at least be comments on the paper from other experts, and if it’s worthwhile there will soon be others investigating the same thing. Often in science new and groundbreaking has to be well proven before people will believe.
Since the media cycle moves quickly and has a short attention span, we rarely hear about what other experts truly say about something that may be controversial. That’s because science moves slowly. Sometimes, though, papers have been recanted-pulled from the journal/website-because it was shown to be poorly written, biased, have errors that are either by mistake or on purpose. In this case the information wasn’t worth the effort to publish it, but unfortunately it’s still in the public mind because the media has moved on.
The bottom line.
Be a skeptic. Ask who?what?why?when? Get the information, and then make up your *own* mind.
Links.
A rough guide to spotting bad science.
Skeptical Raptor. Stalking pseudoscience in the internet jungle.
How we can judge health and science information.
