If you demonstrate statistical incompetence, why should we trust you?
People making a point tend to cite statistics.
And with simple tools like SurveyMonkey, anyone can create a poll.
But if you don’t know how statistics really work, you can seem like a fool. I don’t trust people who use statistical tools in a misleading way. Do you?
What’s wrong with this poll on LinkedIn?
The expert billed themselves on LinkedIn as “LinkedIn™ Trainer/Boosting Visibility & Business.” Their poll on LinkedIn looked like this:
Which statement best describes you on LinkedIn?
- “I’m super active on LinkedIn”
- “I’m here, I’m not active”
- “Here because I’m in sales”
- None of the above, comment
Can you guess which option came in first?
After two-thirds of the votes went to “super active,” the LinkedIn expert commented “Looks like the early winners are ‘super active,’ are you commenting and posting on LinkedIn every day? Inquiring minds want to know!”
Just for now, let’s ignore that they just cited the advertising slogan for the salacious tabloid The National Enquirer. What’s the problem with the survey?
Let’s look at an analogy. You stand outside the local hardware store. You ask each person who comes to the door, “Do you prefer going to the local hardware store, or big-box stores like Home Depot?”
After talking to 50 store visitors, you report the results: 90% of hardware buyers prefer the local hardware store over big-box retailers!
Your sample has an obvious bias. You’ve missed all the people who went to Home Depot instead.
I wanted to give the creator of the LinkedIn poll about LinkedIn a chance to understand the problem. So I commented:
Have you considered that polling on LinkedIn to see who is active on LinkedIn may have an obvious sampling bias?
They responded:
The poll is active for a week. I don’t think it’s an obvious sampling bias. Why do you say that?
Hmm.
People who are on LinkedIn a lot are far more likely to see this poll.
The LinkedIn algorithm is far more likely to show it to people who interact regularly.
People who use LinkedIn a lot are more comfortable with features like polls.
And when I pointed all this out, the original poster insisted “People who are following me, whether they are active or not have a chance of seeing this poll.”
Perhaps. But not an equal chance. A LinkedIn expert who doesn’t understand how sample bias works — and how the LinkedIn algorithm creates it — doesn’t seem like the best source of advice about LinkedIn.
Another bugaboo: the unsourced statistic
A ghostwriting agency posted this to LinkedIn:
Did you know it’s estimated around 60% to a whopping 90% of non-fiction books are ghostwritten?
Really? I have questions.
Who estimated it?
Where did the numbers come from?
What sort of non-fiction books?
Why the huge range?
I’m hugely skeptical of this “statistic.”
I tried to track it down. It’s oft-repeated, but there’s no survey. The best I could find was this 2014 NPR article in which a literary agent guesses that 60% of celebrity memoirs are ghostwritten. Ten-year-old source. No survey, just a guess. And not all nonfiction books, just celebrity memoirs.
Nonfiction ghostwriters should know how to use statistics in their writing. If my doctor says that 98% of people diagnosed with prostate cancer who have radiation treatment have a successful outcome, I know that there’s a research paper about that, and I can examine who the sample was, how they defined “successful,” and whether others have come up with similar results. If I were writing about that in a book, I’d provide both context and a footnote.
This agency provides neither. And as a result, I don’t trust them.
Statistical competence generates confidence
There’s a reason I dedicated a whole chapter in my writing book to dealing with numbers. If you don’t understand how to write about numbers, you’re not a complete writer.
Before you use a statistic, be clear about where it comes from and what it means.
And before you run a survey and shout about the results, consider what biases my exist in your sample.
Learn to use the tools properly, and your writing will be better. Fail to master those skills — and we’ll soon stop trusting your expertise.
There are so many things wrong with that poll and most others I see. Sample bias is just the beginning. Response options are neither consistent nor parallel. Two are frequency of use while the third is a reason for use (and only one at that). If I’m here for sales, I’m also a frequent or an infrequent user. Which do I check? Writing survey questions is an acquired skill and there is science behind it. The availability of easy to use tools doesn’t change that any more than word processing and layout tools made everyone a writer and content producer. Ironically the tools that make such activities easier, often have the effect of dumbing things down – in my view.
Tom