The Right Information

From Shane Parrish’s Decision Making Course

The Right Information

The quality of our decisions depends on the quality of the information we’re using to decide with.

If the information we use is inaccurate, our predictions will be off base and the outcomes could be well out of range of what we’ve prepared for.

More information, however, doesn’t make for better information. In fact most information is irrelevant. We need to know what to keep and what to throw out. Focusing on the right information allows us to make better decisions.

Most of the time this comes from expertise. If you’re an expert in a certain domain, you know which information matters and which doesn’t. You know the rules and the exceptions.

But what if you’re not the expert? Are there reliable ways to gather high quality information so you can make more informed decisions?

Most of the information we consume is not rich sources of information- we instead consume other people’s abstractions. Abstractions are the illusion of knowledge. At best, when things go wrong, we are the line cook and not the chef.

The consumption of raw, rich, unfiltered information from direct sources, as well as knowledgeable secondary sources, puts us on the path toward understanding.

There are three ways to get better information:

Experiment
Go direct to the source and vacuum up the details.
Ask better questions.

One nuance here is you always want to consider the motivations and incentives of the source. You are responsible for filtering the information you let into your mind, and so should only allow high quality information from high quality sources in.

When you are asking experts for information, or those with experiences in similar situations, don’t ask them what they think, ask them how they think. The goal is not just to gain information, but to understand why and how they came to those conclusions.

The consumption of raw, rich, unfiltered information from direct sources, as well as knowledgeable secondary sources, puts us on the path toward understanding.

First, run an experiment.

This first way to gather better information and gather rich data is just to run an experiment. If you want to know if people will pay for something, try to sell it before you even create it. That’s what my friends at Tuft and Needle did.

Go direct to the source.

Ideally you want to get information from someone as close as possible to the problem. There is a difference between reading an academic study and reading a newspaper article on the study. The difference is details. This technique applies in the workplace too.

Getting the right information often means going to the person closest to the problem. That doesn’t mean you let them define the problem for you — remember we never let anyone define the problem for us if we’re the decision maker. And it doesn’t mean you let them determine the solution, remember we own the frame.

But you put yourself on the path to success by going to the person closet to the problem and asking the questions designed to transfer their experience and reflections to you. That means asking them detailed questions, seeking out rich information, and working to really understand the interconnections and things that matter.

So the first principle of gathering great information is: When you want to learn — truly learn — you need to go as close to the source as possible. You want the raw, unfiltered data. You want to be in the weeds. You want to see for yourself.

The closer you can get to the direct experience — either by experimenting and learning directly and reflecting upon it yourself OR by going directly to someone who has experienced it and probing them to understand how they came to their conclusions — the more potent and accurate the information will be.

For example, If you’re looking to sell your house everyone will have a different opinion on what you’ll make on the sale: the bank, your real estate agent, the buyer’s agent, friends, the home inspector, the internet, and the government. Each of them has a different motivation and incentive. Consider the source and consider how they benefit from the information they give you.

When the CEO of my company heard how I got the information about the operational issue, he knew he wasn’t getting the best information because people had an incentive to cover up their mistakes to make themselves look good. It’s natural but it’s destructive.

So, each time you gather information from someone, ask yourself questions like:

How close are they to the problem?
What’s their motivation or incentive here?
What is their source for information if it’s not direct?
If the information is direct, what do the details look like?

The exercise of evaluating the incentives is especially important when you don’t have the ability to go and confirm something for yourself. If you absolutely have to rely on someone else’s information, don’t just ask for details about the situation, question the lens through which they’re looking through. Everyone has a perspective, it’s your job to piece the perspective together to get closer to reality.

Finally, the third technique for getting better information is to ask the right questions.

Don’t ask people what they think, ask them how they think. This technique is really part of the previous one. Asking the source the right questions is extremely important, and I’ll show you how to do that below.

But I’ve separated it out, because sometimes you can’t get to the source and need to rely on second hand information, and so knowing the right questions to ask with these people is equally important.

When the source isn’t available, the best people to ask are:

First, people with broad knowledge in a different area of experience. (These are smart people who can see from outside your frame. They don’t have to be experts at the subject, just smart friends who are not involved.)
Second, people who have solved the same (or a similar problem). These are experts.

But the goal shouldn’t be just to get feedback on your particular problem or tell you what to do — it should be to learn how they would go about solving your problem and the variables they consider relevant and how they interact over time.

If you present a problem, and they tell you what to do, they’re giving you an abstraction. You haven’t learned anything, and if things go wrong you won’t know why. You’re the line cook masquerading as the chef. If you ask them how they think about the problem, you’re starting to learn.

One of the worst things you can do is ask someone to solve your problem for you. Because, when we ask for direct advice — we’re being told what to think, not how to think.

Our goal is not just to gain information — but to understand why and how they came to those conclusions.

To do this, there are three questions I recommend asking, and they apply to both direct sources and quality experts .

Notice how different these questions are than “Here’s my problem what should I do?

First, ask “what are the variables you’d use to make this decision, if you were in my shoes? How do those variables relate to one another?”

If I’m looking for the best heart surgeon, I don’t just want to ask about their opinion on my heart. I want to ask how to choose a great heart surgeon. So I might ask — What expertise should I look for in a great surgeon to handle my particular case?

Second, ask “what do you know about this problem that I (or others) don’t?” (Or what can you see based on your experience, that someone without your experience couldn’t?) In other words, what’s important and what’s not?

Finally, ask “what would be your process for making a decision if you were in my shoes? How would you go about doing it?” (Or, how would you go about telling your Mother / Friend to go about it?)

When you learn not just what someone thinks, but how they think, you start getting raw, unfiltered information and start to find hidden opportunities.

You might think, it’s too much work to do all of this and in a lot of cases you’re right. But in situations that are irreversible and consequential, you want to make the time.

Example:

To recap, let’s apply the 3 techniques of improving the quality of your information to an
example.

Let’s say ’finding a career’.

First, try to run an experiment. Shadow someone for a day or two. Or do the same type of work for an organization on a volunteer basis. The first hand information experiences like this provide is invaluable in the decision making process.

Then, you go to the source. Find someone doing the job you want. Ask them questions designed to transfer their experience and reflections to you. And remember when you are asking anyone questions, always consider the motivation and incentives of who is telling you the information, and ask how to think, not what to think. The information you’ll apply to your own problem will be powerful far beyond its current application.

Finally, in all situations, take the time to craft and ask the right questions. When you interview that person in the field you’re looking to get into, instead of asking “what’s your job like?” You ask: “Given what you know now, how would you approach getting into a position like yours if you were starting from where I am?”

You could ask them what experience and skills they’re looking for in candidates? Or what they know about this career that people who aren’t in the job don’t know? What if in one conversation you found out this company only hires interns who have worked for them, and they have a hard time filling internship positions? How much
time, money, and energy would you save?

When you learn not just what someone thinks, but how they think, you start getting raw, unfiltered information and start to find hidden opportunities. I also like to pose these questions to multiple experts within the same domain. If the same variables and processes come up again and again, I’ve got a good consensus of what I need to look for and do.

When you combine the techniques of running an experiment, going direct to the source, and asking the right questions, you’ll be deeply learning instead of simply consuming. Making you a better decision maker now and into the future.

How can you apply this personally and professionally?

Identify as many rich sources of data as possible. These are people close to the problem, first hand sources, experiments, etc.
Vacuum up other people’s experiences and reflections. Vacuum up these details personally rather than relying on other people’s abstractions of their work. Ask questions when possible around how people think, not what they think. Look for yourself. Organize these in your mind.
Reflect on the experiences of others to draw your own abstractions.
Put your abstractions into action. Test them against the real world and update them accordingly when they don’t yield the intended results. As Louis L’Amour once said, “Knowledge is like money: to be of value it must circulate, and in circulation it can increase in quantity, and, hopefully, in value.”

Asking an Expert Template:

Here’s an email template for you to use when you want to solicit advice from an expert. Notice how this shows you know exactly what you’re looking for, shows that you have already invested in solving the problem, shows you’ve done your research about them (which shows respect) and asks a very specific question.The email should contain:

1. Here’s what I’d like to work towards / see / move in the direction of as a desired outcome or preferred future.

2. Here’s what I’ve already done to work towards the desired outcome. (Even if I haven’t actually “solved” it per se — how have I demonstrated movement towards it?)

State how I would propose movement towards the desired outcome, e.g. “I’m thinking I might try…”

3. Identify their specific superpowers / expertise / insight / value I would like to draw on, and, if necessary, why.

4. Ask one of the three questions on how to think about the problem or opportunity:

What are the variables you’d use to make this decision, if you were in my shoes — and how do they relate to one another?

What do you know about this problem that I (or others) don’t? (Or what can you see based on your experience, that someone without your experience couldn’t?) In other words, what’s important and what’s not?

What would be your process for making a decision if you were in my shoes? How would you go about doing it? (Or, how would you go about telling your Mother / Friend to go about it?)

Other Good Quotes:

The desire for abstractions is natural. We feel overwhelmed by the amount of information available and unable to keep up with processing it all. But when we rely on abstractions we’re not really learning. And the further the information is from the first hand source, the larger the abstraction is likely to be.

But collecting information and learning isn’t as straightforward as it seems. When we don’t do it right, we consume massive amounts of time, energy, and money, paying a high price. If, however, we know how to learn, how to ask questions, and how to collect relevant information, we can free up time and make better decisions.

The truth is, we often treat all sources of information as equally valid, but they aren’t. If you think about it, most of what we “consume” is highlights, summaries, or distillations. We trust these abstractions will save us time and improve our thinking, but they don’t.

Most people assume that we automatically learn from experience. We have an experience and we learn. But that’s not true. We all know someone with 20 years of experience that really has one year of experience repeated 20 times. They didn’t learn the first, or second time around and as a result they make the same mistakes over and over again. Experience is necessary but not sufficient for learning.

History shows that the greatest thinkers all used personally collected details. They got out into the world and interacted with it directly. They looked for information that was raw and unfiltered by others.

Leonardo da Vinci is a great example of this approach. He kept journals throughout his life, and they contain his notes about how he pursued getting the right information. He writes things like ‘get the master of arithmetic to show you how to square a triangle’ and ‘get a master of hydraulics to tell you how to repair a lock, canal and mill in the Lombard manner’. His journals also contain hundreds of sketches and drawings based on direct and detailed observation.

Too often, however, most of us consume information and observations that are multiple degrees removed from their source. Consuming these abstractions gives us another illusion of knowledge. We gain confidence about what to do without understanding why that’s what we need to do.

My friend Tim Urban has a good metaphor to explain this concept. There are chefs and line cooks. Both can follow a recipe. When things work there is no difference in the process or the result. However when things go wrong, the chef knows why. The line cook doesn’t. The chef is the one with years of experience, experimentation, and reflection.

“The quality of my decisions can’t be better than my information. When you’re working on the ground, you know what the problem is. By the time that information gets to me it’s been filtered 4-5 times. Sometimes when people make poor decisions, everyone wonders how they could make such a stupid decision. Well, I’ll tell you how. They’re using filtered information to decide with. Your job, Shane, is to make sure I have the raw unfiltered information. Find out what’s really happening and then we’ll talk. But never make decisions based on the filtered information someone gives you without verifying it first.”

As information travels up in an organization, it goes through understanding filters, politics filters, biases, and more. Details get abstracted and signal is lost. Because your decision will only be as good as the information you possess, you need to go learn it, see it, do it yourself whenever possible.

The Learning Loop:

You can learn from experience. It just requires a couple extra steps. And by experience, I don’t just mean your own experience. We learn from other people when we read, have conversations, or watch them navigate situations and problems.

The Learning Loop goes like this:

We have an experience.
We reflect on that experience.
We draw a conclusion or abstraction from that reflection so we know what to do next time.
That abstraction becomes our “action.” The basis of our next experience.

Reflection, not experience, is what leads us to doing something differently the next time. Experience without reflection gives us an illusion of knowledge. You can see now how true learning forms a loop. The most important thing to understand at this point, is that reflection is a necessary component of learning.

One nuance here is you always want to consider the motivations and incentives of the source. You are responsible for filtering the information you let into your mind, and so should only allow high quality information from high quality sources in.

The exercise of evaluating the incentives is especially important when you don’t have the ability to go and confirm something for yourself. If you absolutely have to rely on someone else’s information, don’t just ask details about the situation, question the lens through which they’re looking through. Everyone has a perspective, it’s your job to piece the perspective together to get closer to reality.

You’re Ready to Move on to the Next Lesson When the Following Are Complete:

I’ve identified where I’m relying on abstractions as my information source
I’ve developed an experiment I could run to get me information useful for my decision.
I’ve identified direct sources of information related to my situation.
I’ve identified experts in similar situations who may be able to help me.
I’ve developed the right questions to ask the people who have the knowledge I need, covering “What are the variables? What do you know that I don’t? What would your decision making process be?”
I’ve reflected on my learning in the program so far.

How can you apply this personally and professionally?

Related