…at least according to a survey of three people in my office (including me).
People just tend to trust quantitative data. It’s reassuring in it’s exactness – one plus one equals two and all that. Numbers are black and white with no room for ambiguity. Right? Not exactly. Please don’t misunderstand, metrics and statistics are very useful tools, providing insight into trends and behaviors beyond what simple human observation allows. However, just like most things in life, statistics should always be taken with a grain of salt and reviewed with a discerning eye.
Flashy headlines and advertisements filled with numbers are created to shock and intrigue, but statistics are usually not as straightforward – or as monumental – as they are made out to be. What can you do to approach numbers with the right amount of caution and respect?
Keep context in mind.
Headlines are created for a single purpose: to draw people in (see a perfect example above). So, it’s really no wonder that writers highlight very significant-sounding data in them.
Never read a headline and walk away (the same applies for ads). Context is absolutely key to ascertaining what a piece of data actually means. For example, upon reading the first paragraph of an article titled “99 Percent of People Wear Pink Socks,” you may find out that this only applies to people living in North Dakota or that people only wear pink socks on Thursdays.
Many articles and ads have an angle and writers use specific statistics to their advantages in order to further their points. Move beyond the title and the bolded text to decipher how the number is being framed and why.
Do a little digging.
Any writer attempting to be taken seriously will accompany a piece of data with a link to its original source (if there’s no source whatsoever, back away slowly and move on).
Instead of taking a statistic at face value, click back to the source and get as many details as you can – assuming these details aren’t already provided in the content you’re reading. Considering the following:
- Who conducted the research?
- Was it a private company? A non-profit? A research institute? A third-party organization?
- Where was the research collected from?
- Was data collected from a reasonable sample size? Nationwide or a single neighborhood in Idaho?
- Who were the research subjects?
- College students? All females? Hispanic adults?
- How many people responded?
- 5? 500? 5,000?
Knowing as much of this information as possible can help you make sense of all of the numbers by allowing you to gain perspective on how these numbers came to exist in the first place. For instance, my above survey of three people should not be given nearly the same respect as the results of a survey of 300 people.
Be wary of outliers.
Imagine: you’ve just read about an amazing, life-altering new study that states one in three people are actually aliens from outer space. But, a month earlier, you read a different article about a study that stated only one in fifty people are actually aliens. Pretty confusing (and scary). Which do you believe?
That’s not a simple question to answer.
Either one could be right or both might be completely off base. When you find multiple statistics about the same topic that conflict, little alert bells should start ringing in your head. Take the time to compare their merits (for example, is one coming from a more reputable source?). Time might account for dramatic variations if technological or other advances have made it possible to conduct more thorough or accurate research.
I’m not a trained expert, but I’ve been duped enough times to learn a few lessons. The biggest takeaway is to never assume – you know the old adage, I’m sure. Never assume that someone offering up a piece of data is an expert; always verify and do your homework. Additionally, don’t underestimate your own intuition – if something seems fishy to you, then it probably is.
Good statistics inform, educate, and provide perspective, but fake data can lead to confusion and worse. Whether you’re dealing with an article or an ad, make sure you know the difference.