A question for thought leaders: “Yes, but how do you know?”

The number of self-described experts has multiplied along with the slots for them to express their views on blogs, podcasts, Forbes, Huffington Post, LinkedIn, and Medium. It’s great that we now get to hear from a wide variety of thinkers. But the first question you should ask of any of them is, “How do you know?”

Every thought leader needs a source of information. To generate an ongoing stream of insights, you need an ongoing stream of data.

In 2008, I had a conversation with a senior executive at a very large retailer. He claimed he had known that the 2008 recession was going to happen, before it happened. His source of information: the company’s data warehouse and point-of-sale system, which report what’s selling on a day-to-day basis. He had seen that sales of two items had spiked well above normal, and put the facts together.

What were the two items?

Home safes and ammunition.

Now that’s a powerful data stream and bit of of clever analysis.

Where do you get your insights?

In this data-drenched age, the number of data sources has multiplied. Thought leaders might know the truth and predict the future based on any of the following:

• Consumer surveys. I helped pioneer the deployment of our massive Technographics surveys at Forrester Research. They enabled us to tap into insights about consumers and their attitudes about technology in a way that wasn’t available anywhere else at the time. We built tools on top of them, too, like Forrester’s Customer Experience Index.
• Search data. Seth Stephens-Davidowitz does details analyses of who is searching for what on Google, and from which geographies. He augments that with data from Facebook and Pornhub. His clever analysis, highlighted in New York Times columns and the book Everybody Lies, reveal a lot about previously hidden topics, like how many people in different geographies are actually homosexual.
• Panels and communities. For example, Diane Hessan’s panel of 400 voters has generated a stream of insights that she has published in the Boston Globe since the election.
• Email responses. Mailchimp knows what email people respond to, because it sends billions of emails every year and observes the results.
• Poll analysis. Nate Silver and his colleagues at fivethirtyeight.com do a meta-analysis of polls, including correcting for their bias and uncertainty. This gives them unique insights into shifts in the electorate.
• Financial models. Financial analysts know things others don’t. Their models — and the information they get from corporate executives — allows them to see where stocks may be going before the rest of us do.

But big data streams are not the only possible source of insight. Other sources might include:

• Years of experience. I’ve been editing non-fiction writers for 30 years. I’ve learned a thing or two about how writing works in that time. Experience — combined with introspection and analysis — generates qualified opinions.
• Journalistic sources. Do the pundits on CNN actually know anything? To the extent that they do, it’s based on their sources that are actually on the ground dealing with what they’re talking about.
• Client relationships. If you’ve helped hundreds of clients with customer experience issues, you’re qualified to talk about what you know. That’s where Jared Spool is coming from.

The challenge with these “soft sources” is that they tend to generate intuition, rather than verifiable truth. They’re valid, but become more believable when backed with quantitative data.

The sources alone are not sufficient to be a legitimate thought leader. You must also put a lot of effort into your analyses of what you learn. Data does not become insight without the application of a lot of work and experience.

Here are some sources that I don’t find credible:

• Insights from one project at one company.
• A close reading and analysis of media sources.
• Somebody who blogs or podcasts regularly.
• Somebody who’s written a book.

None of these things qualify you to lead anything. They are not unique data sources. And no matter the level of brilliance you apply, you will likely come to conclusions that are common in the market of ideas (or are contrarian and wrong). Even if you stumble on a profound truth, you will not have access to a stream of truth. One lucky insight does not make you a thought leader.

Signs of a true thought leader

I am skeptical of everything I read. You should be, too. So how is one to tell the difference between true analysts and pretenders? Ask these questions.

• What’s the data source? Without an ongoing source of information, insights are just opinions. Any thought leader can tell you not just what they know, but why they know it.
• What’s your methodology? How do go from data to insights?
• What are your potential biases? If you analyze data, you know where the weaknesses in your analyses are. No one is always right. Smarter thinkers know where they are likely to be wrong, and why.
• How do you continually hone your insights? Insights are perishable. Most of my knowledge from ten years of analyzing the television industry is useless now, because streaming has upended the business. A true analyst is continually pushing forward, to stay ahead of developments.
• How can we test your insights? Actual insights enable you to make predictions (like “a recession is coming”). If your insights can’t be tested against reality, they are not actual insights. “Things might get better, or they might get worse” is not an insight — unless you tell us when we’ll be able to know, and how.
• What do you do when you are wrong? A thought leader is sometimes wrong. In fact, they tend to be wrong a lot. What matters is what happens next. If they cling to their ideas in the face of evidence, they are a fraud. If they explain what led them wrong, and how the new facts have changed their analysis, there is hope for them.

Test yourself against these questions. If your answers are lacking, you have a ways to go. But once you can answer these questions, you are in a position to create an ongoing platform for continuous insight.