From Annie Duke’s “How to Decide“
How do you estimate probabilities? (according to Annie Duke)
First make an educated guess.
- “There’s a lot of value in making an educated guess. The more willing you are to guess, the more you’ll think about and apply what you know. In addition, you’ll start thinking about what you can find out that will get you closer to the answer.” (p. 83)
- “All your knowledge, imperfect as it might be, means that your guess isn’t random. Although you don’t have perfect information, you have a lot more than no information about what the bison weighs. That’s why I was willing to bet that you wouldn’t guess below one hundred pounds or above ten thousand pounds, because I know you know a lot of stuff.
You almost always know something, and something is better than nothing. You might not get it perfect, but when it comes to decision-making, you get credit for showing your work.” (p. 82) - “We have a way to distinguish informed guesses from uninformed ones. We call the informed guesses educated guesses. It’s not a matter of whether or not any guess is educated. It’s a matter of to what degree.” (p. 82)
- “No information would mean that you know nothing. Perfect information would mean that you know everything. For most things you are trying to estimate, you won’t be at either extreme, having no information or perfect information. Mostly, you will be somewhere in the middle.” (p. 83)
Be clear to others and yourself how educated your guess is. Check where you are on this continuum.
- “No information would mean that you know nothing. Perfect information would mean that you know everything. For most things you are trying to estimate, you won’t be at either extreme, having no information or perfect information. Mostly, you will be somewhere in the middle.” (p. 83)
- “For most things you’re estimating, whether it’s how often you’ll exercise at Sweat Sensations or whether you’ll get rich or go broke with your Kingdom Comb haircut-sharing app, you can’t come close to perfect information. Although it’s useful to make your estimates as precise as possible, it’s also crucial to be clear about—to others and yourself—how much “educated” there is in your “educated guess.” You want to be explicit about how uncertain your belief is.” (p. 110)
- “Whether you’re estimating the weight of a bison or the likelihood that Kingdom Comb will succeed, your job as a decision-maker is to figure out two things:
(1) What do I already know that will make my guess more educated?
(2) What can I find out that will make my guess more educated?” (p. 83). - “Taking aim will make you hungry to answer these questions, moving things from the “stuff you don’t know” box to the “stuff you know” box.” (p. 93)
When you realize there is stuff you don’t know, try to learn more to make your knowledge stronger.
- Your chief weapon to improve your decisions is turning some of the “stuff you don’t know” into “stuff you know.” (p. 93)
- “It might seem daunting that the speck of what you know is so small. But there’s good news, especially if you take a constructive attitude about guessing: Every time you learn something and transform some “stuff you don’t know” into “stuff you know,” you make the foundation of your decisions stronger.” (p. 94)
- “Your beliefs are part of the foundation of every decision you make. Your beliefs inform what options you think are available and the ways that your decision might turn out. Your beliefs inform how likely you think things are to occur or things are to be true. They inform what you think the payoffs are, and even inform your goals and your values.” (p. 93)
- “It’s okay to acknowledge that you’re not usually going to hit the bull’s-eye; the important thing is to take aim. Aiming for that bull’s-eye by making an educated guess gets you closer to a precise hit because it motivates you to assess what you know and what you don’t know. It motivates you to learn more.” (p. 84)
- “We have two main problems when it comes to the stuff we know. First, we just don’t know very much. Learning new stuff strengthens the foundation, making it sturdier. Second, the stuff we do know is riddled with inaccuracies. A lot of our beliefs are not perfectly true. We can think about these inaccuracies as cracks in the foundation. The only way to fix those cracks and shore up the foundation is to find the inaccuracies in our beliefs. And the only place we are going to find that information is in the universe of stuff we don’t know.” (p. 94)
State your educated guess in probabilities.
“Some people aren’t comfortable using probabilities expressed as percentages. After all, that’s part of why people prefer using these natural language terms. If that applies to you, you may find it easier to ask yourself: “For me to use this term to describe the likelihood of an outcome occurring, how many times out of a hundred do I think that outcome would be the one that happens?” (p. 101)
- “For example, if you flipped a coin a hundred times, how many times out of a hundred do you think it would land heads—and what term would you use to describe that probability?”
- “If Mike Trout came to the plate a hundred times, how many times out of a hundred would he get a hit—and what term would you use to describe that probability?”
- “If you hit a first serve in tennis a hundred times, how many times out of a hundred do you think the serve would go in—and what term would you use to describe that probability?”
- “If you passed by the breakroom at work a hundred times, how many times out of a hundred do you think you would eat a donut—and what term would you use to describe that probability?”
- “If you started a business based on the Kingdom Comb app a hundred times, how many times out of a hundred do you think you’d be offered a multimillion-dollar buyout at an early stage—and what term would you use to describe that probability?” (p. 101)
Use the “outside view” (base rates) to make your own view, the “inside view” less bias.
- Use the outside view (base rates) and anchor it with the inside view. You can find out how to do this here
Example: What is the probability you will go to the gym three times a week? You may feel there is a high probability you will go to the gym three times a week (inside view). But you should adjust this prediction because the “outside view” (base rate) shows that only 18% of people go to gym more than once a week.
“As you’re thinking about your doctor’s orders and imagining that there’s a 90% chance that you’ll go to the gym three times a week, these statistics suggest strongly that you should adjust your prediction. No matter how much motivation you think you have, it would be rare that the likelihood you’ll stick with it would be that far off the base rate.” (p. 138)
Try using a range, a lower bound and upper bound for your estimates.
- You want to be explicit about how uncertain your belief is. A convenient way to express where you are on the continuum from no information and perfect information is to offer, along with your exact (bull’s-eye) estimate, a range around that estimate. That range communicates the size of your target area by giving the lowest reasonable value you think the answer could be (the lower bound) and the highest reasonable value you think it could be (the upper bound).” (p. 110)
Example: Let’s say you are trying to guess the weight of a Bison. You know it probably weighs more than a human. So a lower bound is 100 pounds. You guess it probably doesn’t weigh more than 3500 pounds. So a upper bound is 3500 pounds. So you guess with a range is between 100 pounds and 3500 pounds. - “A convenient way to express where you are on the continuum from no information and perfect information is to offer, along with your exact (bull’s-eye) estimate, a range around that estimate. That range communicates the size of your target area by giving the lowest reasonable value you think the answer could be (the lower bound) and the highest reasonable value you think it could be (the upper bound).” (p. 110)
- “Wharton professor Abraham Wyner suggests that a good way to get to a reasonable upper and lower bound is to ask yourself, “Would I be pretty shocked if the answer fell outside this range?” If you use that as your standard, your range will naturally reflect how much educated there is in your educated guess.” (You can think of pretty shocked as meaning 90% confident) (p. 112)
- “When you express probabilities as percentages and offer a reasonable range around those probability estimates, you maximize your exposure to the universe of stuff you don’t know. That increases the chance that you uncover corrective information that will help you repair inaccuracies in your beliefs and improve the quality of your decisions.” (p. 112)
Other good quotes:
“It may be scary to open yourself up to the precision and accountability of making a specific estimate, but it’s well worth it to try. As any archer will tell you, the more you train your sights on that bull’s-eye, the more likely you are to hit it (and the more likely you are to get closer and score more points).” (p. 107)
“This way of thinking, that there is only “right” and “wrong” and nothing in between, is one of the biggest obstacles to good decision-making. Because good decision-making requires a willingness to guess.” (p. 81)
“If you dismiss making estimates of probability by saying, “I’d only be guessing,” then your’re letting yourself off the hook of trying to figure out what you know or what you could find out. Once you say “I’d only be guessing,” there’s no more work to be done. By giving up, you won’t even bother to apply the knowledge you do have to the decision.” (p. 82)