How to make decisions in a group setting

From Annie Duke’s “How to Decide”

Why is it important we get other people’s perspectives?

• “One of the best ways to improve the quality of your beliefs is to get other people’s perspectives.  When their beliefs diverge from yours, it improves your decision-making by exposing you to corrective information and the stuff you don’t know.” (p. 241)
  
  
• “Imagine if you could create a map of the facts and opinions that live in somebody else’s head and compare that with a map of your own. You would find places that overlap and places that diverge. If there’s one thing you’ve learned from this book, it’s that the way you naturally interact with the world makes it much more likely you’re going to see the places where the maps overlap, both noticing the stuff that agrees with you and actively seeking it out as well.” (p. 222)
  
  
• “You’ve probably also figured out that the exciting stuff happens where those maps diverge. That’s where you find corrective information and the stuff you don’t know. Exploring that divergence allows you to get closer to what’s objectively true.” (p. 222)

What are some of the problems people encounter when they have meetings?

1.. When someone gives an opinion, other people in the group will be in infected and influenced. Other people may not be able to give their true opinions.

• “once the first person offers their opinion, the rest of the group is infected.“
  
  
• “You have to rely on other people to tell you what they believe, and if you tell them your opinion first, they will become an unreliable narrator.” (p. 225)

What’s the solution?

• “You can accomplish this in a group setting by eliciting initial opinions and rationales from each member independently, which you then share with the team members before the group meets. This helps alleviate the incomplete-dossier problem, because each member of the group gets exposed to unshared information and the views of others.” (p. 230)
  
  
• “Research has found that people give feedback that more accurately represents their knowledge and preferences when they do it independently and privately, compared with doing it in a group.” (p. 231)
  
  
• “The only way somebody can know that they’re disagreeing with you is if they know what you think first. Keeping that to yourself when you elicit feedback makes it more likely that what they say is actually what they believe.” (p. 225)

Examples:

• “The members of a hiring committee have interviewed the finalists. Prior to any of those people having a group discussion, ask each to email their opinion about which candidate they prefer, along with their rationale. Put that feedback together and share it with the group prior to any discussion in a team setting. (p. 230)
  
  
• “An investment committee is deciding whether to make a particular investment. Elicit feedback independently and share it with the group prior to meeting.” (p. 230)
  
  
• “A legal team has been asked by the client for a recommendation about the settlement of the case. Ask each person on the team, before discussing it with the others, to give their opinion on the amount of a reasonable settlement as well as an upper and lower bound, and the likelihood the opposing party would settle in that range, along with their rationale. Have them email it to be compiled and shared with the group before a team meeting to discuss the matter.” (p. 231)

2. If you tell someone an outcome, it will infect their thinking.

• “Just as you can infect somebody’s feedback by letting them know your opinion, you can also infect their feedback if you tell them how it turned out.” (p. 226)

What’s the solution? Keep the result to yourself.

• “Knowing the outcome can ruin the feedback you get because, as you know, the outcome casts a shadow over anyone’s ability to see the quality of the decision preceding it. That’s why you should keep the result to yourself as much as possible.” (p. 226)
  
  
• “To get high-quality feedback, it’s important to put the other person as closely as possible into the same state of knowledge that you were in at the time that you made your decision.” (p. 227)

3. The way you frame questions can influence and infect other people’s thinking.

• “Sometimes, when you solicit feedback from other people, you don’t realize that the way you’re asking the question signals your belief.” (p. 227)

What’s the solution? Be careful about the way your frame the question and stay in a neutral frame.

• “You want to be careful about the way that you frame the question, because the frame you choose can signal whether you have a positive or negative view about what you’re trying to get feedback on. Try to stay in a neutral frame as much as possible.” (p. 227)

4. When a higher status person gives an opinion, people tend to agree with him and not give their own opinion.

• “Some people are more likely to cause other people’s opinions to bend toward theirs and cause others to suppress divergent viewpoints. The most contagious beliefs on a team come from higher-status individuals. Status can derive from a leadership position, experience, expertise, persuasive ability, charisma, extroversion, or even just how articulate a person is.” (p. 231)

What’s the solution? Anonymize the initial round of feedback.

• “Ideally, when you’re eliciting feedback from members of a group, an idea that’s objectively valid doesn’t become any more or less valid depending on whether it comes from the CEO or an intern. But in reality, ideas that come from people of lower status don’t get equal consideration on the merits.” (p. 232)
  
  
• “The way you get around this status-contagion problem is to anonymize the initial round of feedback, which quarantines the source from the other members of the group, ensuring that the feedback of lower-status individuals gets more consideration than it generally otherwise would.” (p. 232)

But doesn’t expertise and experience matter?

• “You may be thinking right now, “But isn’t it rational to give more weight to some feedback based on the source? If the group is getting opinions about the theory of relativity and Einstein’s in the room, isn’t what he has to say more important than the person just hired out of the intern program?”
  
  Yes, if Einstein is part of the group, there will be times when his opinions should carry far more weight than anyone else’s. Those times are when the group is talking about physics. Because of that, opinions can’t and shouldn’t stay anonymized forever. That said, there are a lot of benefits to having the first pass be anonymous.” (p. 232)
  
  
• “First, it’s hard for people to publicly disagree with team members who have more expertise or are of higher status, whether it’s Einstein or the CEO. But it’s worse than that because of the halo effect, which is the tendency to give the opinions of highly successful people much more weight across the board, even in areas in which they have no expertise.
  
  No one wants to step up and contradict Einstein’s feedback, whether it’s about the theory of relativity or whether to sue the landlord.
  
  
• “Second, for all the value that expertise provides, subject-matter experts aren’t impervious to bias. As Philip Tetlock has shown, when you’re a subject-matter expert, you have a tendency to get entrenched in your worldview and that makes it harder to climb out from that trench and see things from a perspective that differs from your own robust model of the world.
  
  This is a big advantage of anonymizing the initial round of feedback. Members will naturally see things from different perspectives. Because each member of the group doesn’t know the source of any of the perspectives, those viewpoints are more likely to get real consideration. Team members won’t know whose opinion to disregard or elevate.
  
  Group members with lower status may have different, valuable perspectives. Sometimes, they see innovative solutions others don’t see because they aren’t as anchored to the status quo. In a macro sense, when you look at the history of the world, each subsequent generation, by offering its different view, becomes responsible for innovative leaps and paradigm shifts. By initially anonymizing feedback, those outside-the-box perspectives get a chance to breathe.” (p. 232)

5. When individual members of the group have information that goes against the consensus opinion, they often won’t share it.

• “Research shows that in a group setting, even when individual members of the group have information in their possession that goes against the consensus opinion of the group, they often won’t share it” (p. 229)
  
  
• “This shows that the information that lives in people’s heads doesn’t necessarily get shared with the group, particularly as a consensus starts to form. The problem, of course, is that if you want to access the potential of the different perspectives team members have to offer, you actually have to hear about those perspectives.” (p. 230)
  
  
• “Group settings offer the potential of improving decision quality if you can access the different perspectives of the group. Often, this potential is undermined by the tendency of groups to coalesce around consensus quickly, discouraging members with information or opinions that disagree with the consensus from sharing them.” (p. 241)

What’s the solution? Anonymize the first round of feedback.

• “The question is how do you scale the solution of quarantining beliefs and outcomes if, after the first person in a group speaks, the quarantine is ruined? You can accomplish this in a group setting by eliciting initial opinions and rationales from each member independently, which you then share with the team members before the group meets.” (p. 230)
  
  
• “The way you get around this status-contagion problem is to anonymize the initial round of feedback, which quarantines the source from the other members of the group, ensuring that the feedback of lower-status individuals gets more consideration than it generally otherwise would.” (p. 232)

What if you have only a limited in time to go through this process? Try the quick and dirty.

• “When you’re in a group setting considering a decision, each member can write down their opinions and rationales on a piece of paper, passing the papers to one person, who reads them aloud or writes them on a whiteboard before discussion. That doesn’t take much extra time and gives everyone a chance to express an initial position before hearing what others think.”
  
  
• “If you want to save even more time, you can have people write their opinions and rationales down and each member can then read theirs out loud, but it’s imperative that you start with the most junior member of the group, whose opinion is least contagious (and who is most likely to get infected by the higher-status members, depriving the group of hearing their authentic perspective).” (p. 234)
  
  
• “A good group process encourages feedback that includes giving people the space to express a lack of understanding. The group as a whole benefits from that because it affords the experts the opportunity to better understand why they believe what they do, and also affords them the opportunity to transfer their knowledge to the other members of the group. And sometimes it gives them an opportunity to repair inaccuracies in the things that they believe.” (p. 233)

Before making group decisions, agree on a checklist.

• “For any decision that you have under consideration, that checklist of relevant information will differ, but it will commonly focus on the applicable goals, values, and resources, along with the details of the situation. You want to provide what the person needs to know to give worthwhile feedback—and nothing more.
  
  First and foremost, you need to communicate what it is that you want to accomplish. People have different goals and values and those things matter. The right option for one person might differ from the right option for another.
  
  If you’re asking for advice on a vacation destination, the person you’re asking should know your goals, preferences, and limitations. If you tell them you want to go to someplace that’s sunny in February and has a lot of history, but you don’t tell them that you only have three days for your vacation, they might recommend Australia, perhaps because they had a wonderful time when they spent two weeks there.” (p. 236)
  
  
• “Developing a checklist for a team can be done using the same process as eliciting any other feedback in a group setting. Ask group members independently to answer the question: “If someone came to me asking for feedback on this category of decision, what would I need to know?” Bring the answers together, anonymize them for distribution to group members, and discuss it as a team. The product of this will be a master checklist for information that has to be provided to any team members being asked for feedback.” (p. 236)

Examples:

• “If you’re making a hiring decision, it’s important to include your goals, values, and resources. Is your aim to hire someone with experience who, in turn, will help train future new hires? Do you value a sunny disposition? The relevant facts about the candidates include their CVs, what their references said, and what their interviews consisted of.” (p. 236)

Make sure everyone is accountable to the checklist.

• “Members of the group should hold one another accountable to that checklist. Anybody eliciting feedback has the responsibility to provide the information on the checklist, and anybody being asked for feedback must request that all the information on the checklist be provided.” (p. 237)
  
  
• “Whether it’s within a group or just two people asking each other for advice, it’s important to agree that everyone participating in the feedback process is accountable to the checklist.” (p. 238)
  
  
• “When I was teaching poker, students of mine would sometimes come to me and describe hands where they couldn’t remember some of the facts on the checklist. As an example, a student might ask me if they should have called a bet on the last card of a hand, but they couldn’t remember how much money was in the pot. In that case, I would refuse to tell them whether they should have called the bet, because if I didn’t know the size of the pot, my opinion would have no value and anything I told them would be gibberish.” (p. 239)
  
  
• “More important, my refusal created the opportunity for them to understand why knowing the size of the pot is crucial. In future hands, regardless of whether they asked for feedback or not, they were going to pay attention to this key detail and incorporate it in their on-the-spot poker decision-making.”
  
  
• “These benefits of adhering to a checklist accrue whether it has to do with the size of the pot in poker or how many time-outs you have left when you’re calling a football play late in the fourth quarter. Or how important cultural fit is when you’re hiring a new employee. Or how important the judge’s potential prodefendant reputation is to the value of your lawsuit. Or how important the depth of management is when you’re considering buying stock in a new electric-car company. Having a good checklist in place helps you combat biasing narratives and provides a framework within which to process information as you make future decisions.” (p. 239)

Other good quotes:

“A rational person would welcome the chance to change an inaccurate belief, but we know people like learning they’re wrong about as much as those doctors liked Semmelweis telling them they were killing patients by not washing their hands. As painful as it might be to find out something you believe is incorrect, the opportunity to change that believe will improve the quality of every single subsequent decision informed in any way by that belief.” (p. 222)

“In the hiring example, you might start by telling them the position you were trying to fill and ask what they think the key components of the job description should be and the appropriate salary range. Once you’ve gotten that feedback, you could show them the actual job description and salary range and move on to whether you should have conducted the hiring process in-house or should have hired an outside contractor.” (p. 226)

“Dan Levy, Joshua Yardley, and Richard Zeckhauser of the Harvard Kennedy School found that asking students to raise their hands (the time-honored group feedback system in classrooms) caused a herding effect, where, as soon as students saw a consensus developing, they raised their hands to join with the consensus opinion, creating super-majorities.”

“In contrast, when students answered with electronic clickers so they couldn’t see what anybody else was answering, the super-majorities that formed with public hand raising got broken up. Soliciting feedback from students independently provided instructors with a better representation of the students’ true knowledge and preferences.” (p.231)

“This is one way that keeping a record, using the decision tools developed in this book, helps you later when you’re seeking feedback about your decisions. Tools such as a decision tree, a Knowledge Tracker, or a Decision Exploration Table create a record of your state of knowledge at the time of the decision, which makes it easier to convey that information to get someone’s feedback later.” (p. 227)

Imagine if you could create a map of the facts and opinions that live in somebody else’s head and compare that with a map of your own. You would find places that overlap and places that diverge. If there’s one thing you’ve learned from this book, it’s that the way you naturally interact with the world makes it much more likely you’re going to see the places where the maps overlap, both noticing the stuff that agrees with you and actively seeking it out as well. You’ve probably also figured out that the exciting stuff happens where those maps diverge. That’s where you find corrective information and the stuff you don’t know. Exploring that divergence allows you to get closer to what’s objectively true. Where the maps diverge and your opinion and somebody else’s are far apart, three things might be true, and they are all good for improving the quality of your decisions:

• The objective truth lies somewhere between the two beliefs. When two people are equally well informed and they hold opposite opinions, the truth most likely lies between the two. When that’s the case, it’s obvious why both people benefit from having discovered the divergence. Both people get the opportunity to moderate their beliefs and get closer to the objective truth.
  
  
• You could be wrong, and the other person could be right. If you hold an inaccurate belief, the quality of any decision informed by that belief will suffer. A rational person would welcome the chance to change an inaccurate belief, but we know people like learning they’re wrong about as much as those doctors liked Semmelweis telling them they were killing patients by not washing their hands. As painful as it might be to find out something you believe is incorrect, the opportunity to change that belief will improve the quality of every single subsequent decision informed in any way by that belief. That seems like a fair trade: a little bit of pain in exchange for higher-quality decisions for the rest of your life.
  
  
• You could be right, and the other person could be wrong. When this is the case, you might think that only the person who is wrong benefits by getting the chance to reverse an inaccurate belief, because your belief was right and will remain unchanged. But actually, you benefit from the exchange as well because the act of explaining your belief and conveying it to someone else will improve how well you understand it. The better you understand why you believe the things you do, the higher in quality those beliefs become.” (p. 223)

“You can get to the outside view by asking yourself, “If somebody were seeking my opinion about this type of decision, what are the things that I would need to know to feel like I could give them high-quality feedback?” Make a checklist of those details and then provide them to anyone you are seeking advice from. You can do this for any decision, but it’s especially useful for decisions that repeat because you can think about it before you’re facing any particular instance of that decision. When you’re in the midst of a decision, you’ve probably already formed an opinion about your preferred option. Once that happens, your preference will distort what information you think you would need to know. By building this checklist in advance, you won’t be as influenced by the specifics of a decision on which you’ve already formed an opinion, making it easier to be objective and get to the outside view.” (p. 235)