A Public Service Announcement :)

Wired comes through with a primer on how to read a scientific paper, something yours truly does but with some difficulty. Now, with this little gem of an article titled Learn to Read a Scientific Report, writers and readers alike will know what to look for when wading through a tome describing the intricate mechanisms of some esoteric organism or phenomenon. To whit:

Reading an original paper isn’t the same as understanding it. Science journalists try to help on that front, but as with any filter, important bits can get lost in translation. So you need to explore a few critical elements—of the study or the media coverage about it—to determine whether it contains life-changing advice or something best deposited behind the couch in the dentist’s office. And what are those elements? Glad you asked.

Causation vs. correlation
How do you know if a study’s results answer the question it set out to ask? Sometimes an outcome is just a coincidence—there’s a correlation but no causation. Meta-analyses pool the results of smaller studies and filter signal from that kind of noise.


True size of the effect
Watch out for weasely language—a “threefold increase” might only be a shift from 1 percent to 3 percent. One recent paper reported that women’s mortality risk rose 133 percent. That sounds scary, but the elevated mortality rate was still just 1.9 percent.


Statistical power
Look at two key factors, the n and the p. The n is the number of subjects used in the study. Multifaceted experiments typically have fewer subjects than simple surveys. Genetics studies need a big n. The p value lets you know whether the result is “statistically significant”—it’s the probability of something occurring by chance alone. You want to see a p of less than 0.05. (Results can be statistically significant and still only show correlation, or have confounding factors.)

Conflicts of interest
Most journals now note this as a matter of policy. Was the company making the drug or product associated with the laboratory that did the study? Are any of the authors trying to sell a product? For example, the authors of a study exploring the effectiveness of “brain training” techniques on cognitive enhancement worked for the company that developed (and sold) those techniques. They disclosed this, but that’s still a red flag.

Any questions? :).