How to make decisions in a group setting

From Annie Duke’s “How to Decide”

How to make decisions in a group setting.

• “One of the best ways to improve the quality of your beliefs is to get other people’s perspectives.  When their beliefs diverge from yours, it improves your decision-making by exposing you to corrective information and the stuff you don’t know.” (p. 241)

What are the problems when groups make decisions?

• “When you’re talking to somebody one-on-one, you have a simple solution to the contagion problem. Whatever the feedback is that you’re eliciting, don’t broadcast your opinion first. But this solution doesn’t scale well in a group setting.”
  
  
• “When you’re in a group discussion, you can keep your opinion from everyone, but once the first person offers their opinion, the rest of the group is infected. Our gut tells us that when it comes to decision-making, more heads are better than one. If you elicit more opinions, you get more of the outside view, a wider variety of perspectives, a broader range of beliefs, and that will improve decision quality. Yet we know that when people are in group settings, the decision quality often isn’t better, but because more people are involved in the process, the confidence in the decision goes up. That’s a bad combination: having a lot more confidence in the quality of a decision that isn’t necessarily any better. That’s why belief contagion is particularly problematic in groups.” (p. 229)
  
  
• “Research shows that in a group setting, even when individual members of the group have information in their possession that goes against the consensus opinion of the group, they often won’t share it” (p. 229)
  
  
• “This shows that the information that lives in people’s heads doesn’t necessarily get shared with the group, particularly as a consensus starts to form. The problem, of course, is that if you want to access the potential of the different perspectives team members have to offer, you actually have to hear about those perspectives.” (p. 230)
  
  
• “Group settings offer the potential of improving decision quality if you can access the different perspectives of the group. Often, this potential is undermined by the tendency of groups to coalesce around consensus quickly, discouraging members with information or opinions that disagree with the consensus from sharing them.” (p. 241)
  
  
• “The partial-information groups quickly formed a consensus based on their premeeting preferences. Once they saw a consensus forming, members with divergent information (negative on the consensus candidate or positive on the nonpreferred candidates) were unlikely to share it. Unlike in the complete-information groups, where Candidate A was the strong preference, the partial-information groups almost always chose one of the other candidates. In other words, even though the partial-information groups had in their possession all the information to determine that Candidate A was the best, they didn’t share that information and usually landed on a less favorable choice.” (p. 230)

How do you solve this problem?

• “The question is how do you scale the solution of quarantining beliefs and outcomes if, after the first person in a group speaks, the quarantine is ruined? You can accomplish this in a group setting by eliciting initial opinions and rationales from each member independently, which you then share with the team members before the group meets. This helps alleviate the incomplete-dossier problem, because each member of the group gets exposed to unshared information and the views of others.” (p. 230)

Examples:

• “The members of a hiring committee have interviewed the finalists. Prior to any of those people having a group discussion, ask each to email their opinion about which candidate they prefer, along with their rationale. Put that feedback together and share it with the group prior to any discussion in a team setting. (p. 230)
  
  
• “An investment committee is deciding whether to make a particular investment. Elicit feedback independently and share it with the group prior to meeting.” (p. 230)
  
  
• “A legal team has been asked by the client for a recommendation about the settlement of the case. Ask each person on the team, before discussing it with the others, to give their opinion on the amount of a reasonable settlement as well as an upper and lower bound, and the likelihood the opposing party would settle in that range, along with their rationale. Have them email it to be compiled and shared with the group before a team meeting to discuss the matter.” (p. 231)
  
  
• “Research has found that people give feedback that more accurately represents their knowledge and preferences when they do it independently and privately, compared with doing it in a group.” (p. 231)
  
  
• “Part of the feedback-elicitation process should include specifying the form in which people provide it. This book has described many decision tools that you can ask team members to use for offering feedback: forecasts of specific events or outcomes, outcomes on a decision tree, options to be considered, payoffs, counterfactuals, a Perspective Tracker, the output of a Decision Exploration Table (a premortem and/or a backcast) or the Dr. Evil game, or Ulysses contracts or hedges. You could also ask for feedback in the form of answers to yes or no questions or on a rating scale.” (p. 231)

Try anonymizing feedback

• “Some opinions are more contagious than others. Some people are more likely to cause other people’s opinions to bend toward theirs and cause others to suppress divergent viewpoints. The most contagious beliefs on a team come from higher-status individuals. Status can derive from a leadership position, experience, expertise, persuasive ability, charisma, extroversion, or even just how articulate a person is.” (p. 232)
  
  
• “Ideally, when you’re eliciting feedback from members of a group, an idea that’s objectively valid doesn’t become any more or less valid depending on whether it comes from the CEO or an intern. But in reality, ideas that come from people of lower status don’t get equal consideration on the merits.”
  
  
• “The way you get around this status-contagion problem is to anonymize the initial round of feedback, which quarantines the source from the other members of the group, ensuring that the feedback of lower-status individuals gets more consideration than it generally otherwise would.” (p. 232)
  
  
• “This is a big advantage of anonymizing the initial round of feedback. Members will naturally see things from different perspectives. Because each member of the group doesn’t know the source of any perspectives, those viewpoints are more likely to get real consideration. Team members won’t know whose opinion to disregard or elevate.” (p. 232)
  
  
• “Group members with lower status may have different, valuable perspectives. Sometimes they see innovative solutions others don’t see because they aren’t anchored to the status quo.”

Shouldn’t higher status people have a bigger voice?

• You may be thinking right now, “But isn’t it rational to give more weight to some feedback based on the source? If the group is getting opinions about the theory of relativity and Einstein’s in the room, isn’t what he has to say more important than the person just hired out of the intern program?” Yes, if Einstein is part of the group, there will be times when his opinions should carry far more weight than anyone else’s. Those times are when the group is talking about physics. Because of that, opinions can’t and shouldn’t stay anonymized forever. That said, there are a lot of benefits to having the first pass be anonymous. (p.232)

Here are some reasons why it’s still important to get lower status people’s feedback.

• “First, it’s hard for people to publicly disagree with team members who have more expertise or are of higher status, whether it’s Einstein or the CEO. But it’s worse than that because of the halo effect, which is the tendency to give the opinions of highly successful people much more weight across the board, even in areas in which they have no expertise. No one wants to step up and contradict Einstein’s feedback, whether it’s about the theory of relativity or whether to sue the landlord. “
  
  
• “Second, for all the value that expertise provides, subject-matter experts aren’t impervious to bias. As Philip Tetlock has shown, when you’re a subject-matter expert, you have a tendency to get entrenched in your worldview and that makes it harder to climb out from that trench and see things from a perspective that differs from your own robust model of the world. This is a big advantage of anonymizing the initial round of feedback. Members will naturally see things from different perspectives. Because each member of the group doesn’t know the source of any of the perspectives, those viewpoints are more likely to get real consideration. Team members won’t know whose opinion to disregard or elevate. Group members with lower status may have different, valuable perspectives. Sometimes, they see innovative solutions others don’t see because they aren’t as anchored to the status quo. In a macro sense, when you look at the history of the world, each subsequent generation, by offering its different view, becomes responsible for innovative leaps and paradigm shifts. By initially anonymizing feedback, those outside-the-box perspectives get a chance to breathe.” (p. 232)

Try the “Quick and dirty”- An alternative for lower-impact, easier to reverse decisions.

• “When you’re in a group setting considering a decision, each member can write down their opinions and rationales on a piece of paper, passing the papers to one person, who reads them aloud or writes them on a whiteboard before discussion. That doesn’t take much extra time and gives everyone a chance to express an initial position before hearing what others think.”
  
  
• “If you want to save even more time, you can have people write their opinions and rationales down and each member can then read theirs out loud, but it’s imperative that you start with the most junior member of the group, whose opinion is least contagious (and who is most likely to get infected by the higher-status members, depriving the group of hearing their authentic perspective).” (p. 234)
  
  
• “A good group process encourages feedback that includes giving people the space to express a lack of understanding. The group as a whole benefits from that because it affords the experts the opportunity to better understand why they believe what they do, and also affords them the opportunity to transfer their knowledge to the other members of the group. And sometimes it gives them an opportunity to repair inaccuracies in the things that they believe.” (p. 233)

Before making group decisions, agree on a checklist.

• “For any decision that you have under consideration, that checklist of relevant information will differ, but it will commonly focus on the applicable goals, values, and resources, along with the details of the situation. You want to provide what the person needs to know to give worthwhile feedback—and nothing more.”
  
  
• “First and foremost, you need to communicate what it is that you want to accomplish. People have different goals and values and those things matter. The right option for one person might differ from the right option for another. If you’re asking for advice on a vacation destination, the person you’re asking should know your goals, preferences, and limitations. If you tell them you want to go to someplace that’s sunny in February and has a lot of history, but you don’t tell them that you only have three days for your vacation, they might recommend Australia, perhaps because they had a wonderful time when they spent two weeks there.” (p. 236)
  
  
• “If you’re making a hiring decision, it’s important to include your goals, values, and resources. Is your aim to hire someone with experience who, in turn, will help train future new hires? Do you value a sunny disposition? The relevant facts about the candidates include their CVs, what their references said, and what their interviews consisted of.” (p. 236)
  
  
• “Developing a checklist for a team can be done using the same process as eliciting any other feedback in a group setting. Ask group members independently to answer the question: “If someone came to me asking for feedback on this category of decision, what would I need to know?” Bring the answers together, anonymize them for distribution to group members, and discuss it as a team. The product of this will be a master checklist for information that has to be provided to any team members being asked for feedback.” (p. 236)

Make sure everyone is accountable to the checklist.

• “Whether it’s within a group or just two people asking each other for advice, it’s important to agree that everyone participating in the feedback process is accountable to the checklist.” (p. 238)
  
  
• “Members of the group should hold one another accountable to that checklist. Anybody eliciting feedback has the responsibility to provide the information on the checklist, and anybody being asked for feedback must request that all the information on the checklist be provided.” (p. 237)
  
  
  
• “Whether it’s within a group or just two people asking each other for advice, it’s important to agree that everyone participating in the feedback process is accountable to the checklist.” (p. 238)
  
  
• “When I was teaching poker, students of mine would sometimes come to me and describe hands where they couldn’t remember some of the facts on the checklist. As an example, a student might ask me if they should have called a bet on the last card of a hand, but they couldn’t remember how much money was in the pot. In that case, I would refuse to tell them whether they should have called the bet, because if I didn’t know the size of the pot, my opinion would have no value and anything I told them would be gibberish.” (p. 239)
  
  
• “More important, my refusal created the opportunity for them to understand why knowing the size of the pot is crucial. In future hands, regardless of whether they asked for feedback or not, they were going to pay attention to this key detail and incorporate it in their on-the-spot poker decision-making.”
  
  
• “These benefits of adhering to a checklist accrue whether it has to do with the size of the pot in poker or how many time-outs you have left when you’re calling a football play late in the fourth quarter. Or how important cultural fit is when you’re hiring a new employee. Or how important the judge’s potential prodefendant reputation is to the value of your lawsuit. Or how important the depth of management is when you’re considering buying stock in a new electric-car company. Having a good checklist in place helps you combat biasing narratives and provides a framework within which to process information as you make future decisions.” (p. 239)

Other Good Quotes:

“A rational person would welcome the chance to change an inaccurate belief, but we know people like learning they’re wrong about as much as those doctors liked Semmelweis telling them they were killing patients by not washing their hands. As painful as it might be to find out something you believe is incorrect, the opportunity to change that believe will improve the quality of every single subsequent decision informed in any way by that belief.” (p. 222)

“In the hiring example, you might start by telling them the position you were trying to fill and ask what they think the key components of the job description should be and the appropriate salary range. Once you’ve gotten that feedback, you could show them the actual job description and salary range and move on to whether you should have conducted the hiring process in-house or should have hired an outside contractor.” (p. 226)

“Dan Levy, Joshua Yardley, and Richard Zeckhauser of the Harvard Kennedy School found that asking students to raise their hands (the time-honored group feedback system in classrooms) caused a herding effect, where, as soon as students saw a consensus developing, they raised their hands to join with the consensus opinion, creating super-majorities.”

“In contrast, when students answered with electronic clickers so they couldn’t see what anybody else was answering, the super-majorities that formed with public hand raising got broken up. Soliciting feedback from students independently provided instructors with a better representation of the students’ true knowledge and preferences.” (p.231)

“This is one way that keeping a record, using the decision tools developed in this book, helps you later when you’re seeking feedback about your decisions. Tools such as a decision tree, a Knowledge Tracker, or a Decision Exploration Table create a record of your state of knowledge at the time of the decision, which makes it easier to convey that information to get someone’s feedback later.” (p. 227)

Imagine if you could create a map of the facts and opinions that live in somebody else’s head and compare that with a map of your own. You would find places that overlap and places that diverge. If there’s one thing you’ve learned from this book, it’s that the way you naturally interact with the world makes it much more likely you’re going to see the places where the maps overlap, both noticing the stuff that agrees with you and actively seeking it out as well. You’ve probably also figured out that the exciting stuff happens where those maps diverge. That’s where you find corrective information and the stuff you don’t know. Exploring that divergence allows you to get closer to what’s objectively true. Where the maps diverge and your opinion and somebody else’s are far apart, three things might be true, and they are all good for improving the quality of your decisions:

1. The objective truth lies somewhere between the two beliefs. When two people are equally well informed and they hold opposite opinions, the truth most likely lies between the two. When that’s the case, it’s obvious why both people benefit from having discovered the divergence. Both people get the opportunity to moderate their beliefs and get closer to the objective truth.
   
   
2. You could be wrong, and the other person could be right. If you hold an inaccurate belief, the quality of any decision informed by that belief will suffer. A rational person would welcome the chance to change an inaccurate belief, but we know people like learning they’re wrong about as much as those doctors liked Semmelweis telling them they were killing patients by not washing their hands. As painful as it might be to find out something you believe is incorrect, the opportunity to change that belief will improve the quality of every single subsequent decision informed in any way by that belief. That seems like a fair trade: a little bit of pain in exchange for higher-quality decisions for the rest of your life.
   
   
3. You could be right, and the other person could be wrong. When this is the case, you might think that only the person who is wrong benefits by getting the chance to reverse an inaccurate belief, because your belief was right and will remain unchanged. But actually, you benefit from the exchange as well because the act of explaining your belief and conveying it to someone else will improve how well you understand it. The better you understand why you believe the things you do, the higher in quality those beliefs become.” (p. 223)

“You can get to the outside view by asking yourself, “If somebody were seeking my opinion about this type of decision, what are the things that I would need to know to feel like I could give them high-quality feedback?” Make a checklist of those details and then provide them to anyone you are seeking advice from. You can do this for any decision, but it’s especially useful for decisions that repeat because you can think about it before you’re facing any particular instance of that decision. When you’re in the midst of a decision, you’ve probably already formed an opinion about your preferred option. Once that happens, your preference will distort what information you think you would need to know. By building this checklist in advance, you won’t be as influenced by the specifics of a decision on which you’ve already formed an opinion, making it easier to be objective and get to the outside view.” (p. 235)