Statistical Street Smarts
To paraphrase Mark Twain, there are three kinds of lies: lies, darned lies, and statistics. This can be countered with another saying: numbers don’t lie. Both sayings are true in their own way because while numbers don’t lie, people can use numbers to mislead you. This is common practice in advertising and political polling. It’s a dangerous pitfall for certain scientific fields. It can be hard for a layman like yourself to spot the lies in statistics, but here are three things to watch for when you’re trying to decide whether to believe a study.
First, ask yourself where the data came from and how it was gathered. There are two sides to this question. First of all, it’s important to ask who was being asked, and how many? As a general rule, the larger the sample size (the more people asked) the more accurate the data. It’s also important to consider the questions being asked. This is especially important for in-depth political polling, as the phrasing of questions can bias the response.
Look carefully at how statistical data is presented. Graphs can be misleading, so look carefully at the labels on the x and y axes of a graph to see what they mean. What is being measured, and how? Certain information may be being left out. It’s fairly common practice, for example, to use a broken line at the x- or y-axis to show that some numbers have been skipped. This can be done for innocent reasons, like making a graph easier to read, but it can also be used to create a misleading impression. Lines on a graph are meaningless if you don’t know what they are representing.
Correlation and Causation
Last but not least, remember that a correlation does not prove causation. A line on a graph can be alarming, but don’t jump to conclusions until you have seen the proof of why. After all, the crime rate goes up in summer at the same time ice cream sales do, but that doesn’t mean that the two are in any way related.
This isn’t to say that you can or should ignore statistical data. Statistical analyses are a powerful tool for knowledge, but it’s all too common for people to misuse numbers for their own advantage. Be a wary consumer of media by asking the right questions about a statistic before you decide whether to believe it or not.