The following is a rough transcript which has not been revised by The Jim Rutt Show or by Annie Duke. Please check with us before using any quotations from this transcript. Thank you.
Jim: Today’s guest is Annie Duke, a retired professional poker player and she’s an author in behavioral science and decision education. She holds a World Series of Poker Gold Bracelet and won the 2004 World Series of Poker Tournament of Champions and the National Heads Up Poker Championship in 2010. She’s written a number of instructional books for poker players including Decide to Play Great Poker and The Middle Zone and has authored two books on decision making. We’re going to talk about those books today. Hi, Annie. Welcome to The Jim Rutt Show.
Annie: Hi, thanks for having me. Excited to be here.
Jim: This is great. I read these books and I go, “Wow, this is just some good stuff.” I’m really looking forward to bringing it to our audience. The two books we’re going to talk about today are Thinking in Bets and How to Decide. As we said at the intro, links to those books will be available on the episode page at jimruttshow.com, but first, before we jump into the books themselves, could you tell us the tale of how you became a professional poker player?
Annie: Yeah, that tale involves just a lot of luck. What I love about actually this tale of luck is that I’m not sure that I would have thought it was good luck at the time, which I think is actually quite a deep lesson about luck, so I’ll explain. Here’s the first lucky thing that happened which is long before I start becoming a poker player which was that my brother, when he was a teenager, was really into chess. He started playing chess tournaments. He got to a master rating and he really wanted to go study with a grandmaster. He had done an independent study program in high school where he went to New York for a few months and studied with somebody as part of this program and then he wanted to continue with that.
Annie: He had gotten into Columbia University for college and decided to defer a year. This was back in the early ’80s and he’s going to go study with a grandmaster. He does that, and instead of spending so much time with that grandmaster, he actually uses the little college fund that he has, which I think was $6,000, and loses it all at the poker table. The thing about my brother, though, is that, obviously, he was a master in chess, he’s very, very smart and very good at games. The reason why he lost all that money was because he didn’t really understand poker. He didn’t know the rules.
Annie: He found his way to a book, I think, by David Sklansky and then he found his way to a really interesting group of poker players in New York which included Erik Seidel, who I think now has 40 million in earnings in tournament poker or something under his belt, Dan Harrington who’s won the main event of the World Series of Poker and made the final table many times and super, super smart guy. He was in that group. A guy named Jason Lester, just really, really smart games players. He found his way to them and they created this learning pod and he started learning. By the time he was 23, having when he was 18 lost his college fund to poker, he had made it to the final table of the World Series of Poker.
Annie: All of that’s happening before, there’s any thought in my head that I might ever play poker. It’s totally accidental that my brother ends up in that world because this was not a time when poker was on television. It wasn’t part of the internet. You would have had to find poker through this kind of weird way and he found it through chess. As he’s doing this, actually, in the year that he made the final table, the World Series of Poker, that was my first year at Columbia myself. I followed my brother to New York and my brother didn’t really ever go to … I think he might have gone for a semester, but he certainly never graduated, but I did do my four years there. While I was there, I would occasionally go and watch my brother play poker and I would like sit behind him and watch him play.
Annie: I went off to graduate school after college at the University of Pennsylvania and I was doing my PhD work there with Lila and Henry Gleitman. At the end of the five years, here’s where the other big piece of luck comes in. At the end of the five years, I’m going off from my job talks. I had been really struggling for many months during the last year of my PhD candidacy with a stomach issue called gastroparesis. It makes it so that your stomach doesn’t really empty food out very well, so you’re pretty sick all the time. I wasn’t feeling well. I thought, “Well, I’m going to do my job talks,” and I’ve been going to a doctor, trying to figure it out and I was like, “I’ll really pay attention to this after I do my job talks.”
Annie: Well, my body had a different idea and I actually ended up in the hospital for two weeks quite sick and I couldn’t do my job talks that year. As you probably know, academics, the job cycle is seasonal. What that meant was that I was going to have to wait until the next year to actually go out on the job market and I needed a year to recuperate. I had some constraints on what I could do during that year. I didn’t want to reboot a new career because I was planning to go off and be a professor. I was sick, so I couldn’t do something that was 9:00 to 5:00 that was regular because it was going to be very likely that I was going to have to miss a lot of days.
Annie: My brother was actually the one who’s just suggested to me, “Why don’t you try playing poker? You could make some money. I’ll give you some pointers.” I played a little bit before then because he was nice enough during my graduate school days to fly me out once a year to Las Vegas when he was playing in the World Series of Poker as a nice vacation and I had played a little bit of low stakes poker, plus I had sat behind him a lot. I did understand some things about the game. He convinced me that maybe I should try that and that he would be my tutor. That’s what I decided to do while I was waiting to go back out on the job market. That’s actually how I started playing poker. It turned out that I had a knack for the game. I guess maybe it was genetic and I really, really loved it.
Annie: I ended up not going back to academics. I just never went back out on that job market. This thing that I was planning to do, in the meantime simply because I was sick and I was looking for something to supplement my income while I recovered, turned into 18 years in a pretty long career in the game. There you go.
Jim: That’s cool. We’ll get into that. How luck and skill are two things that helped form our world. Personally, I’m a pretty decent amateur poker player. In fact, I got good enough to have a surprisingly good hourly win rate at the small casinos in Northern New Mexico, although that was more a function of the amazingly bad and consistently bad pattern to play at those casinos and my own level of skill as it became clear later, which we may have time to tell or maybe not when I happen to sit down at a table with Phil Hellmuth and got quite a schooling and what I didn’t know about the game.
Annie: There you go.
Jim: There are levels and levels where I can make $175 an hour in 1/2 no limit in this shitty casinos in Northern New Mexico or I could lose $1,000 an hour playing at a 5/10 no limit game with Phil Hellmuth at the table. I can tell you that too.
Annie: First of all, let me say, those players must have been really wild for you to make that much money.
Jim: They are unbelievable. I couldn’t believe it either. I have my notebook. It’s full.
Annie: That’s a high volatility game if you’re making a $1.75 an hour, like whoa.
Jim: It’s nuts and they were so bad. The number of people typically in for the flop would be six or seven out of 10.
Annie: Oh, my gosh. Well, there you go. This is the wonderful thing about poker. Poker is just a really complex game. It’s really multilayered. It’s a game that reveals more of itself to you as you go along, right? When you start, it feels like it’s pretty simple. Then as you get deeper into the game, you discover more and more of the complexities. That’s something that makes it really wonderful. What it means is that, you say, “Oh, I ran up against finding out I wasn’t that good at the game.” I would say that’s how I would describe myself, is I’m not particularly good at that game. The nice thing is that you can usually find people who are worse than you are. It’s really this game where it’s like what’s your skill relative to other people, not what’s your skill relative to having the game solved. If you can actually place yourself in the right spot in terms of who should you be playing against, you can win a ton of money. That’s a really nice thing about that game.
Annie: I’ll tell you the funniest game I ever played in because this game that you played in sounds super wild. By the way, there was a guy who used to play in Northern Arizona who used to come up to Montana when I first started playing and his nickname was Barb Wire. I don’t know if you knew him. Anyway, I know that’s where he played. In the winters, he would go to Arizona. In the summers, he would be up in Montana. I’m sure you ran across these people. He was not the best behaved at the table and he kept having to move around from casino to casino in Arizona because he kept getting barred from places. I think he would occasionally get barred from the room that I was playing in, but it was a smaller economy, so they were a little bit more tolerant, I think, of people who maybe should have been ejected from the game. I don’t know if you knew someone named Barb Wire, but he was definitely an Arizona player.
Jim: No, we were in New Mexico.
Annie: You were in New Mexico. I’m so sorry. Now, I feel so silly.
Jim: Two different worlds entirely. Truthfully, I didn’t play that much. I would only play at the casino maybe once a month. Actually, for two reasons, one, I had lots of other busy things I was involved with and, two, truthfully I felt guilty winning money from all these Northern New Mexico painting and building contractors and small time ranchers and stuff. Here’s the scary thing. I go to the casino maybe once a month. The same people are always there. Oh, dear, right? They’re clearly there. There was a guy, a photographer from Albuquerque, terrible. There was a big painting contractor in Northern New Mexico, probably made 300,000 a year, probably dropped quarter million a year at the poker table.
Jim: The only ones you had to watch out for who were close to Los Alamos, occasionally some young physicists would come down from Los Alamos and some of them actually weren’t that good, but some of them were really good. You had to pick your table. In fact, the way you and I got connected with through a guy named Michael Mauboussin and he tells a poker story about me in one of his fairly regular business talks which was … I learned to play poker from around the family table, mostly under the influence of somebody we called Uncle Wally. He wasn’t actually our uncle, but he was a war buddy of my dad’s, an old line bachelor, came around the house. We gave him dinner once a week and had a good time, really great full life guy. He was professional gambler.
Jim: He also drove a cab. He’s going to string a luck going through, at least he could eat, but he taught us how to play, taught me how to play. By the time I was about 11, he said, “Jim, you’re probably a good enough poker player now. You should spend your time booking for weak games.” I’ve used that advice, not just in poker, but in life and it is hugely important as you indicate. I don’t have to be better than Phil Hellmuth to make money at poker. I just have to find weak games.
Annie: Well, there was a player named Eric Drache. Back in the day, no limit was not a game that was played very often and the reason is that when you have a really small economy, you want games that have more variants because what happens with no limit is that the skill differential realizes in the outcome a little bit too quickly and a little bit too in your face. It ends up causing people to come up against reality a little too quickly. It’s harder to blame your life on luck repeatedly because the percentage of times that you lose is just going to be really high. The games that were played were limit games. It allows the poorer players to get a higher percentage of wins under their belt, even though obviously, they don’t have positive expected value. It keeps the poker economy going a little bit better. I think it improves the experience for the people who were there a little bit more for entertainment purposes as well.
Annie: Anyway, there was this guy named Eric Drache and Eric Drache was an amazing poker player. Really amazing. The thing that was always said about him though was that he was the sixth best seven-card stud player in the world. The problem was that he was always playing against the other five guys.
Jim: That’s a mistake.
Annie: That’s your uncle Wally’s advice, is that so much about what you’re doing is … I don’t know if you’ve read Don Moore’s book, Perfectly Confident, but what he really talks about is this idea of you have to calibrate your confidence. Part of that is that we have this tendency for over-placement. We think that we place in in terms of what quartile we might be in or what decile we might be in higher than we actually should. That was, of course, Eric Drache’s problem. You were getting the advice of, “Don’t do that thing,” right? You should always be looking to really have a wide margin between the people that you’re playing against and you and make sure that you’re pretty sure that that margin exists, right? Play down. Don’t think that you should be playing right up against what your skill level is going to be.
Jim: I’ll give you another short little story about that. We were out, the whole big extended family, to Vegas for my mother’s 80th birthday. She loved casinos. I remember going over to the Bellagio, and actually, we were staying at the Bellagio. I walked around the poker room at the Bellagio, scoping the tables and give 90 seconds a minute, two minutes to each one I found one’s absolutely full of fish. I went up to the maitre d’ and I said, “Put me down for table seven.” He said, “There’s an hour and 45-minute wait for that table.”
Annie: That’s what happens when there’s a lot of fish at the table.
Jim: I left and went and saw those two crazy magicians, what the hell they’re named, Penn & Teller instead.
Annie: Penn & Teller who are lovely by the way. Teller, I actually haven’t met, but Penn is one of the most lovely human beings.
Jim: I took a cab over to where they have their show. I came back, and then two hours later, there were some other tables that weren’t quite as fishy as that one, but the guy was so funny. “Oh, yeah, an hour and 45-minute wait for that table.” It’s been a very interesting game. Anyway, let’s get a little deeper into the book. One of the things that you said was a direct quote for the book, “Thinking in bets starts with recognizing that there are exactly two things that determine how our lives turn out.” That’s a big claim, “Two things that determine our lives turn out. The quality of our decisions and luck. Learning to recognize the difference between the two is what Thinking In Bets is all about.” Unpack that one for us. That’s a big powerful claim.
Annie: I actually think it’s not such a big claim. I think it’s rather trivial because I think just true, right? If you think about what determines how your life turns out, well, there’s going to be the matters of luck, which is a lot of it, which has to do with lots of things like you were born a human being and not a raccoon. That’s a big thing, right? You don’t have any control over that. Not only were you born a human being, but you were born, Jim Rutt in America to your parents with whatever talents or lack of talents that you might have been brought into the world with, at a very particular time in history. For me, there’s a very big difference for somebody like me being born in 1600 when depending on where I live, women were property versus being born at the time I was when women could own property, right? That’s a really significant difference, one, which I had no control over.
Annie: Aside from just that very basic influence of luck that’s at the start of everybody’s life, as we go through, there’s just a whole bunch of stuff that we can’t control. I just want to just make clear that when I’m talking about luck, I’m talking about a pretty broad definition of luck, right? That would be the actions of others over whom you don’t have influence, right? What Vladimir Putin does in relation to me is a matter of luck from my perspective. I can’t control what he does. It’s randomness, right? Just like the stochastic nature of the world, that’s luck. When we’re thinking about luck, we’re talking about this force that’s external to you, that is acting on you and having a pretty big influence over the outcomes in your life, but you don’t have any control over that stuff. That’s one thing.
Annie: Then the other thing you have control over is the stuff that you have control over which is the quality of your decisions and how are you making decisions within that environment, right? How are you thinking about what’s the information that I have or that I have available to me? How can I think about it? What are the mental models that I can apply to this problem? What’s the data that I can bring into the situation? What is my decision process look like? How am I thinking about my ability to forecast the future, recognizing that, whatever forecasts that I have, the ultimate outcome is going to be influenced in some part by luck? If it’s the lottery, it will totally be influenced by luck. If it’s a basketball game, it will be somewhat influenced by luck. If it’s a game of chess, it’s going to be most almost completely influenced by skill, not quite all the way. You very rarely find things that are out at the end.
Annie: What I’m trying to say in these books is your job on the luck side of the equation is to see it clearly, to try as much as possible to be able to get a sense of what the spread of possible outcomes are, so that you can assign some probability to those, but even if there’s an outcome that’s going to happen 2% of the time, it’s important for you to recognize that that will happen 2% of the time and that’s how often you’re going to observe it. You need to see luck, but you need to make better decisions because that’s the part of the equation that you do have some control over.
Jim: That’s good. When I read that, I stopped and thought about and said, “It covers a lot of cases.” If I’m going to throw one-third case back at you and tell me what you think, maybe you can make it fit into one of the other and this is the one that I’ve used in my business career a lot, once it was pointed out to me and that is the coevolutionary context. It sounds like a lot of big words, but it means what’s going on from the dynamics of the external situation that you don’t have control over necessarily, but you can at least have some level of quantitative understanding.
Jim: The first example that hit me quite clearly and this was pointed out to me by a Harvard Business School professor, it was on my board, in my first startup, in 1982, 102 companies got started to build a new kind of computer hard drive called Winchester Hard Drives. As one might expect, only four of them survived. If you had stopped and listened, you could have heard the sound of dozens of people heading for that opportunity. If one understood the evolutionary context, one would have said, “Nope, not playing that game.” It’s neither luck nor I guess you could call it coevolutionary skill, but it’s evaluating the context related to looking for weak tables. I’m not going to play a game where there’s 50 companies and it looks like there’s soon going to be more in a field where only four we’re going to get through the door.
Annie: Yeah, that actually goes completely under decision quality to me. That’s what I’m trying to say about you need to see the luck and then we make good decisions, right? Because in the sense that if you think about something really simple, like a coin flipping opportunity, if I don’t properly understand what the distribution of outcomes is for a coin flip, I can’t actually make the decisions about that coin flip, even though I don’t control the outcome. This is where the intersection of these two things occurs. If I actually can see the luck properly and I can understand what the probabilities are there, then I understand the difference between laying $1.10 and taking $1.10 there, right?
Jim: Let’s jump ahead. I was going to talk about this, the four coin flips story later in the book, but since you brought it up, let’s go with it. Take the audience through what happens when they see a coin flipped four times. This was actually very interesting.
Annie: Let me think about this. I’m going to try to surmise what you think. We know that if you flip a fair coin, that it will land heads 50% of the time. If you flip it again, the chances that you get two heads in a row would be 25%, three heads in a row would be 12.5%, and four heads in a row would be 6.25%. We know that about the distribution of coin flips. There was a really interesting study that was done which had to do with people’s willingness to bet on somebody’s coin flipping skill, which there isn’t any skill to that right. There’s a skill in what the proposition is that you take, but this was simply like people were flipping coins, and then they’d say, “Heads,” and then the question just, “Did it land heads?”
Annie: Obviously, when you have a lot of people do that, you’re going to have certain people who have streaks, right? A certain percentage of the people who are calling heads on every single coin flip, let’s say, are going to call that coin correctly four times in a row. In fact, you can figure that if it’s 100 people doing that, 6.25% of them will do that. The question is, what happens when people actually have to bet on … They have a choice. They say, “You can bet on Jim,” and you just saw him, he called heads and the thing landed tails or you can bet on Morgan over here and you just saw Morgan called heads twice and it landed heads twice or called heads four times and it landed heads four times and who do you want to bet on?
Annie: What ended up happening was people wanted to bet on the people who had actually called the coin correctly which of course is totally random, but they felt that there was a greater chance that they were going to call the coin correctly going into the future. This is absolutely a pretty well-known bias called the gamblers fallacy. I think it shows how little we actually understand luck, that people are willing to do that, because of course, you could flip a coin to pick who you were going to bet flipping coins on and you would get the same result, but not only were they willing to bet on them, but they would bet more money on them.
Jim: Interesting. Very good. Let’s jump back into the book here a little bit and talk about the story that you start Thinking In Bets off with, which is, those of us who remember back to the 2015, it was a pretty big story at the time, Pete Carroll and the call in the Super Bowl. Do you have that story at your fingertips or do you want me to read it out to you?
Annie: I have that story at my fingertips.
Jim: What I wanted you tell it. You can tell it better than I for sure. You weaved the story throughout the book which I thought was quite fascinating.
Annie: I appreciate that. This story really exposes a weakness in human reasoning which is, how are we thinking about the influence of luck and skill? How are we thinking about what an outcome means? This is the problem of the coin flip that we just talked about, right? If you call a coin heads twice and it lands heads twice, what does that mean? What am I supposed to say about your coin flipping skill, right? It turns out humans aren’t great at that. There are complex answers to that and we’re not particularly good at that particular determination about what an outcome means.
Annie: Let’s get into the Pete Carroll story. It’s 2015, as you said. It is Super Bowl 49. Pete Carroll, as people might recall, has the Seahawks in the Super Bowl playing the New England Patriots. This is the New England Patriots heyday, Belichick-Brady. The New England Patriots are going for their fifth Super Bowl when Pete Carroll is going for a second. Those are two great teams, two great coaches. The Seahawks are on the one-yard line of the New England Patriots. Now, here’s the thing about this one-yard line that they’re on. It’s the fourth quarter and there are only 26 seconds left in the game. Now, they also happened to be down by four.
Annie: We can see that this limits your options, right? You can’t go for a field goal because that’s only going to get you three points. That’s not going to cover the four. They’re in a situation where they absolutely have to score a touchdown. That’s it. That’s their only choice. Now they have a clock management problem. It’s second down, so theoretically, they’ve got second, third and fourth down in order to try and get this ball across the goal line and move it that one yard. The problem is with 26 seconds left, that’s actually quite hard to do, particularly because the Seahawks only have one timeout left.
Annie: It just becomes important to thinking about the decision that we know that when you run the ball, the clock will run, even at the end of the play and you have to use a timeout in order to get the clock to stop. We know also that if you pass the ball and it’s incomplete, that the clock will stop on its own. Now, this just becomes important to this decision just because there’s this clock management problem. That’s the scenario, 26 seconds left, second down, one timeout, got to get the ball one yard against the New England Patriots down by four.
Annie: There’s an expected play here, so expected in fact that the announcers announced what the play is probably going to be beforehand because everybody thinks that Pete Carroll is going to call for Russell Wilson to hand this ball off to Marshawn Lynch. Marshawn Lynch is one of the greatest running backs of all time. He’s actually nicknamed The Beast and he also happened to have been having a good day, that day. He was moving the ball like a hot knife through butter, but you are on the Patriots one-yard line which means the defense is going to be compressed and no running back is as good in that situation as they are midfield, but that aside, everybody assumes you’re going to hand this ball off to Marshawn Lynch.
Annie: Pete Carroll does not do that. Instead, Pete Carroll calls for a pass play, very famous play. Russell Wilson passes the ball to the right corner of the endzone and the ball’s intercepted by Malcolm Butler and the game ends. The announcers go nuts. Chris Collingsworth is just saying, “I can’t believe the play. I can’t believe the play. I can’t believe the play,” and the pundits go nuts. The headlines that you see the next day are, well, most of them said, arguing that it was the worst play call in Super Bowl history. USA Today actually said it was the worst play call in all of NFL history which is a pretty spectacular claim.
Annie: Now, it was Good Morning America that Pete Carroll was actually interviewed that week. It was interesting because the interviewer was clearly trying to get him to say his mea culpas, almost like apologizing to America because, obviously, everybody was rooting against the Patriots, except for everybody who lives in New England, but the rest of the country was not rooting for the Patriots. It was like, “Don’t you agree, don’t you want to apologize? Don’t you think that was a really bad, really bad play?” Pete Carroll said something that I think is really, really insightful. What he said was, “I will agree that it was the worst result of a call in Super Bowl history.”
Annie: Then he says, “If the ball had been caught, it would have been a different reaction.” I think that we can do that thought experiment so easily, right? He does this really out-of-the-box thing and he calls for a pass play and the ball is caught for the game-winning touchdown. Does USA Today have a headline the next day saying, That was The Worst Call in NFL History, or is the headline, That was The Gutsiest Call in NFL History, This is Why He’s the Boss, This is Why He’s Going to the Hall of Fame? I think it only takes about one second of the thought experiment to really realize, of course, it’s the latter. Of course, everybody calls it the greatest play in the history of time. How different is that than the coin flipping people? “You’re the greatest coin flip caller in the history of coin flippers because you called it heads and it happened to land heads.”
Annie: There’s three different outcomes for that. It could get intercepted, it could get caught for a touchdown and complete or it could just be incomplete. There’s some probability distribution of those things happening, that it happened to be intercepted on that one particular time means very little for what the decision quality is, and yet, it means everything for the way that we see the decision quality. That’s a cognitive bias called resulting and it’s one of the things that frustrates our ability to learn from our decisions and become better decisionmakers, probably more than almost any other bias that exists.
Jim: I love that. I’ve never actually heard the term resulting before. Is that a cognitive science term or a poker term?
Annie: That is a poker term.
Jim: I suspect that is a-
Annie: But now all the cognitive scientists use it which makes me very, very happy. Basically, there’s a cognitive bias. If you talk to a decision scientist, the term they would use is outcome bias. If there’s a more intuitive term that some group of people is using to describe something that I don’t think it’s intuitive, if I say to you, “Oh, it’s outcome bias.” I don’t think it’s intuitive exactly what that would mean, but it’s the same thing as resulting. Resulting with something that me and like Erik Seidel and my brother, in our group, when we were talking about this particular problem and obviously you can imagine this issues problem, poker you’ve played, when you lose a hand. How many times have you sat at a table where some guy loses a hand, they say, “Oh, that person was an idiot”? It’s like, “Really? Were they? How do you know? It’s one hand or you lost in one session. What do you suppose to learn from that?”
Annie: This is a big problem in poker and this happens when you inject left into the equation. Resulting was actually a really important term in our group where we would say, “Well, maybe I’m just resulting. Maybe I think this. Maybe I think I played that hand well because I’m just resulting.” It was just a shorthand for being able to talk about this concept. I just felt like it was a much more intuitive term for that particular concept than outcome bias. I certainly acknowledge that it’s outcome bias. I’m not making up a new bias. I’m not telling you something that’s new. I’m just giving it a more intuitive name.
Jim: I love the name actually. You gave a million interesting examples, and I go, “Shit, our lives are dominated by this,” because an awful lot of the stuff we do is very low end, right? You talk about things like, “I quit my job and went and did a startup, which I actually did at the age of 27 and it worked.” However, the fact that it worked, as you point out, doesn’t necessarily mean it was a good decision.
Annie: Right. Nor does the fact, if it doesn’t work out, it doesn’t mean it was a bad decision, right? You can imagine, for example, we can talk about the computing example that you gave where we know that there can be certain ideas where the market gets incredibly crowded with competitors. From society standpoint, from the portfolio holder standpoint, that’s a really good thing, right? Society probably wants too many people trying to solve that particular problem where it may not be positive expected value for each individual who’s trying it, but for society, it’s really positive expected value, right? It’s an interesting thing. It depends on the perspective, but for each individual, particularly and obviously, if you have entrance into that market, it gets worse for that N+1, right? It gets worse. The respective value goes down for N+2, even more so for N+3, so and so forth.
Annie: As you get more entrance into that market, they may be negative expectancy, but it may be the person who was N+1 who actually ends up solving that problem, but that does not make it a good decision for them. That may have been really negative expected value decision for that person and it happened to work out, but it didn’t have to. Great for society, but it’s the same problem as near death experiences, right? Here’s an example, right? Muhammad Ali, we know he wins the title as Cassius Clay and then he’s forced to take time off because he refuses to go to the Vietnam War. Then he comes back and he’s an old man in boxing terms. He’s 33. Everybody’s saying, “He’s over. He’s not going to be able to do it. He really shouldn’t continue to box,” but he does, he persists and he continues and we know that he ends up beating George Foreman and he wins his title back.
Annie: Now we know that results, what feels like a pretty good decision, but then I can follow that along, which is he continued to be told, “You really should stop fighting,” in fact so much so that his trainer quit, and his team quit and Madison Square Garden was like, “We’re not going to hold the fight anymore that you’re in because it’s too awful,” and he couldn’t get licensed in America. He ended up finding a way to be able to fight offshore. We know that the result of that, which is it pretty similar decision, was that he ended up with Parkinson’s disease. Do we want to say that the first time because he happened to have won the championship back was a good decision and something to be admired and the second part of the story was a bad decision because it worked out poorly, right?
Annie: This is the problem that we always have with these kinds of things, right? It’s just hard to dig down into the decision quality because we don’t live in the matrix and we can’t see the probabilities right in front of us. We can’t see the full decision tree right in front of us. I’ve talked about this with the Pete Carroll example, I can explain why that was actually a great decision, but it requires me to walk through the decision tree. I’ve got to unpack the decision. I’ve got to talk about what the probabilities of successful run versus successful pass, what’s the probability of an interception. I’ve got to get some options theory in there actually as well. Once I do that, I can show you that mathematically it was a very good play, but that’s the conundrum that we have as human beings is that you can’t see decision quality, but what you can see is outcome quality.
Annie: The reason why we do resulting is the same reason why we do … A lot of cognitive bias comes from this thing that we do as human beings which is called substitution, that when we have to judge something that’s both subjective and complex, we will substitute in something that is less objective and less complex. We can see that with this, right? Well, the first time Muhammad Ali went against the advice and continued fighting, it must have been a good decision because it worked out well. I can see whether he won or lost. I can see whether the ball was intercepted or not intercepted. That’s not a complex judgment for me. It’s not subjective, right? It’s objective, “The ball was intercepted. I know what it was. Muhammad Ali won the title. I know what it was.” Until we make this substitution and that causes us not actually to dig down in decision quality.
Annie: Obviously, you can see why that’s really bad for the types of feedback loops that we need in order to actually improve our decision making going forward because it causes us to come to some very, very poor conclusions about whether a decision was good or not.
Jim: That’s it in a nutshell, is the thinking that you’re advocating is hard, right? Maybe we do a little sidebar here and talking about Kahneman’s System One and System Two.
Annie: System One is the substituter. What happens is that very often we’re trying to make a complex judgment and the substitutions are occurring because we’re System One thinkers. System Two just takes a lot of effort. When I start going into the analysis of the Pete Carroll play, I’m going to be recruiting a lot of System Two thinking, whereas the System One thinking is the ball got intercepted, so obviously, it was a terrible play. I’m not saying that when you make that judgment, that means that you’re not intelligent. Intelligent people do this just as much as people who are maybe less intelligent. This isn’t an IQ issue. It’s a mindware issue. It’s the way that we’re built. These things are like visual illusions. They happen and we very often don’t notice them, unless we’re like hyper vigilant for them.
Annie: That’s really what I’m trying to do and approaching this in a systems way like, “How do we actually create better decision environments, better systems, in order to help us to avoid those traps and actually improve decision quality?” This is something that we need to be very intentional about because otherwise, System One is going to win.
Jim: Absolutely, though I’m going to putting out another mildly contrarian point, we also have to decide intelligently with discernment when to use System One, when to use System Two. A good friend of mine is one of the smartest people I know and can do the analysis of decision making in a relentless way that’s very, very impressive. On the other hand, he finds himself falling into using System two thinking to decide what to have for lunch when you go out to a restaurant with him. What he’ll typically do is just point to somebody at the table and say, “I’ll have the same as they’re having,” because he doesn’t have the discernment to know when he should be in System One and when he should be in System Two. That’s an interesting and curious oddball case.
Annie: So sorry. I’d love to be able to address that, but I don’t know if you want to go there.
Jim: It’s very important. Go there.
Annie: I actually spend a whole chapter on that issue in how to decide, chapter seven, if anybody’s interested, because you’re exactly right. When should we slow down? When should we speed up? I think that this is particularly problematic in today’s world in particular for two reasons. One is that the world, it’s very complex, the world that we’re living in right now. That can cause actually the reverse problem of your friend which is we’ll just say, “It’s too complex. I’m just going with my gut,” and so will tend to go with our gut too much, right? That’s one reaction that you can get to the environment that we live in right now. The other reaction that you can get is that the world is incredibly data rich. You can get the illusion that, “If I just kept collecting more information, I could somehow get to certainty.”
Annie: That’s the gold ring on the merry-go-round, right? It’s like just within reach, but you can never quite get to it. If you keep doing that, you’re going to go around that merry-go-round like your friend and you’re never going to be able to get out of those analysis loops because we can fool ourselves into thinking that it’s a solvable problem. It pushes us to one end or the other, right? Over analyzers or gut decision makers and of course, we want to be somewhere in between. And to your point, which was super insightful, we want to make sure that we’re slowing down when we should for the right type of decisions and we’re speeding up for the right types of decisions.
Annie: So, the first thing that’s a little meta is in order to actually allocate your sort of time, your decision resources, which is like time and money and effort and whatnot. You do actually have to understand what a robust decision process would look like, so you do need to understand like, “How would I build out a pretty simple decision tree? How would I think about what my options are? What are the ways which I might compare those?” So, it’s like I can take a lot of shortcuts in math, but it’s good if I actually understood what the long way would be before I start taking the shortcuts. Right. So, it’s a similar kind of idea.
Annie: And then what we can do is start to think about two different sort of broad ways that we can consider the decisions that we face and one would be thinking about impact and the other would be thinking about optionality. And these are the two things that we want to think about as we’re thinking about how slow or faster we go. So, just in short, the impact question and this is about what should you order off the menu has to do with what’s the long-term impact on my happiness? Let’s use that as a peg for money, health, time, so on and so forth, right? So, generally, we’re going to be happier when those things are going better for us, so we’ll use happiness just kind of the proxy for that.
Annie: So, what’s the long-term impact of my happiness on this decision? And we could think about like the menu decision, like let’s say, you’re trying to decide between among some things on the menu and you order something, and it’s like crappy. It tastes really bad. I’m not assuming food poisoning here. I’m just saying like it was bad. It was yucky. And I’d see you in a year and I’d say, “Hey, do you remember a year ago when you had that crappy fish?” Did that have any impact on your happiness as it sits today? There’s, I don’t know anybody who says yes, right? Like, “Oh, yeah, that crappy fish I had a year ago really ruined my year.”
Annie: And if I see you in a month and I asked you the same question, I don’t know anybody who says, “Yeah, the crappy fish I had a month ago really ruined my month.” And I actually don’t really know anybody who would say that after a week like, “Yeah, last weekend, I had a bad piece of fish. It really ruined my week.” Again, I’m not assuming food poisoning here. And so, what that tells you about a decision, like a menu decision is just pretty low impact that whatever regret that you might feel when you actually have the bad meal in front of you and you feel like you made a mistake, which is what we say when those things happen. “I made a mistake,” which of course is resulting.
Annie: In the long run and if you can get sort of a longer time horizon on that outcome, what you find is that that that outcome really has no impact at all, at which point you should speed up and your friend has actually come up with a pretty good solution, which is about the same as coin flipping. Ask somebody else to pick for you, tell the waiter to surprise you, flip a coin, whatever, that all you really want to do is sort the menu into stuff that you like and stuff you don’t like. And once you sort of reached the threshold of “Yeah, this is something that I like,” then you can probably just go ahead and flip a coin. And one way to figure that out is to sort of look at the things that you’re choosing among and say, “If this were the only thing that I could eat, would I be pretty happy?” And if the answer is, “Yes,” go ahead and order that thing.
Jim: And while we’re on this topic, talk briefly about the 10/10/10 methodology, which I thought was a very nice simple way to do this kind of sorting.
Annie: Yeah, absolutely. So, this doesn’t come from me, this comes from Suze Orman, I believe. So, basically, it says, “Imagine how you’re going to feel about that decision and 10 minutes, in 10 months, in 10 years.” It might be 10 minutes, 10 weeks and 10 years, but anyway, the point is the same. And that’s really what the happiness test is doing for you. The more that you can sort of get away from the moment of the decision when you’re going to be thinking about you’re going to be really controlled by wants, you’re going to be really controlled by fear of regret, that immediate regret in the wake of getting the crappy meal, the less good your decision making is going to be.
Annie: And this really plays into Kahneman’s framework of inside and outside view that when we’re kind of in the decision, we’re really going to be more inside view thinking and if we can get to these sort of longer time horizons that allows us to get to the outside view to see the decision as almost like an outside observer would. And obviously, any outside observer who’s ever seen someone spend a whole bunch of time freaking out about what they’re going to order on a menu and taking 15 minutes and asking everybody at the table, “What do you think I should order? What do you think I should order?” And asking the wait staff and unable to decide.
Annie: You’re just like, “Just decide already. Who cares? It’s just a meal. You’re going to get another chance to order something. You’re going to eat something at dinner.” Like, “Could you stop,” right? And so, we want to get ourselves to be able to see ourselves from the outside a little bit better, so that 10/10/10 is actually pretty helpful for that. Yeah, so that’s kind of like the first thing that can sort of tell you when you should go faster.
Annie: Now, the second thing is optionality. And there’s two ways to think about optionality. One is if it’s repeating, so that’s like the meal thing is repeating. “If I don’t like my lunch, I’m going to get to have dinner in a little bit, so it probably doesn’t really matter if my lunch isn’t so good, because I’ll get another try really fast.” But the second way to think about optionality is really important is how quitable is the decision. So, when we’re making decisions, we’re making decisions under conditions of uncertainty and the uncertainty is coming from two sources.
Annie: One is luck, so just the world is stochastic. There’s randomness. We don’t control the actions of others. So, what that means is that the world as it is today is different than the world that will be tomorrow or in a month or in a year. And then the other thing is that we’re deciding things behind some sort of partial veil of ignorance. Just meaning that we don’t have all the information that we need in order to make a perfect decision because we’re not omniscient. So, what that means is that new information may reveal itself to us in the future. So, when the world changes or when we learn new things, which would mean we change, we may want to then change our mind. We may figure out that the thing that we chose is not the thing that we would choose today and then we want to actually switch to something new.
Annie: So, that idea of kind of quitableness, like “How easy is it for me to stop the thing that I’m doing and go on to something else and choose a different option, maybe even an option that I rejected in the first place,” actually helps you to determine how fast, like how much time you should actually spend on the decision. And it’s a pretty simple rule, the more quitable the decision is, the easier it is for you to change your mind or change course and go do something else, the faster you can go on the initial decision. The less quitable it is, the slower it should go on your initial decision. And that just has to do with when we think about impact and options that when we’re deciding under uncertainty, when we go faster, the chances for error are greater and along with the chances of error going up is that you may be increasing the probability that you get an outcome that you don’t like.
Annie: So, the question is, how much can we withstand an outcome that we don’t particularly like? And that’s why you want to think about impact and options. If the worst possible outcome doesn’t affect you very much then you don’t care much at all about increasing the error rate, because it’s not really going to matter to you and so you should just go fast. And if it’s really quitable then you also don’t care very much about the error rate because you can change your mind, so then you should go fast. And that’s really the idea behind lean startup and just push beta out as quickly as possible because you’re doing these sort of small batch development and then to a small number of customers and you can just reverse course if it turns out it doesn’t work, but the information that you get out of having done so pays you for that higher error rate, because you’re getting speedier.
Jim: Yeah, it’s a quitable. I love that. It’s another good term. I had a longtime strategy advisor who worked right at my side for many years in my businesses. He had the same idea, but he framed it slightly different. He called it, “The right to abandon.” And he would say that it’s the most overlooked optionality that’s out there. Most people miss that one and he schooled me on that and I used that relentlessly. For instance, for instance, since I retired, which is now going to be 20 years on April 1st.
Annie: Wow, Congratulations.
Jim: I retired young, I’ll confess. Ever since, I’ve used his “right to abandon” by saying, “I’m a retired guy, motherfucker. I’m not going to do anything I can’t get out of in six months.” And so that has ruled in and ruled out many opportunities that present themselves. For instance, I’ve had many options to be on the Board of Directors of publicly traded companies. I was a quite successful public company CEO. I was big demand, especially the first couple of years after I retired.
Jim: And I told every one of them, “No.” And they said, “Why did you do that? This was easy money.” I go, “Well, because it would be immoral to accept that position and then quit after six months.” And they said, “Well, why would you want to quit after six months?” And I said, “I don’t know if I would, but my rule for my retirement is don’t do anything you can’t quit within six months, that you don’t have the right to abandon.” And I’ve been very happy about using that rule.
Annie: Well, I have to say, I love that frame. So, what I love about that framework is that obviously we’re all familiar with Grit, a super popular book, rightly so. Angela Duckworth is amazing. Her work is amazing. But it’s really only the shallowest reading of Grit that would say that perseverance sort of as a default, perseverance as the virtue in the battle of that versus quitting a lot should win the day. I feel like perseverance really captures the imagination, particularly I think partly because we have this kind of like ancient thread of hero’s story, right? The person who persevere is in the face of danger and sort of crosses the chasm and ends up winning in the end.
Annie: There’s companies like this, like Rovio for example. I think Angry Birds was maybe their 51st or 52nd game that they put out and all the ones before that failed. And that’s amazing and that should be the model because we should all just sort of like persevere in the face of anything. And obviously, and Angela would not say that that’s the way that you should read her work. But we want to think about this idea of like grit and quit as complements to each other that it’s a calibration problem, that there are things that we should be gritty about and that, but then when the world tells you that you should quit, we ought to do that.
Annie: And the interesting thing in the decision science world and the cognitive science world and the behavioral science world is that the science on our inability to quit, when the world tells us that we should, is deep. It is broad, it is deep. We see it all the time. The top of Mount Everest is littered with the bodies of people who did not figure out that they might quit, but so is the business world, right? And so are like public works projects, like nuclear power plants that were supposed to cost $70 million that end up costing billions, right?
Annie: There’s so many famous acquisition stories where they’re going down the acquisition route. Snapple is one of these and the signals are really bad and yet you persist, right? And then this is true, you know this as a CEO in terms of projects, right? Like once somebody starts a project, it’s so hard for them to abandon the project, even in the face of like, “Look, there’s a lot of opportunity cost for this. It looks like it’s not working out well.” But people, once they start, they don’t want to give that up.
Annie: And it’s not just some cost, which is, obviously, a big part of the literature, but you get into this kind of deeper concept, which is escalation of commitment that as things start going wrong, we sort of actually escalate our commitment to the cause. And this is a really big problem. We need to start thinking about these things as complements to each other as opposed to sort of like this binary of one good one bad. And when you talk about this idea of right to abandon, what’s so beautiful about bringing that into your decision process in the front is that you are keeping front and center that you need to be thinking about this as, “I always have the option to quit.”
Annie: You’re probably thinking in advance, I’m guessing about, “What are the circumstances under which I would quit?” You’re probably entering it as more experimental. So, it becomes less kind of endowed to you on part of your identity, which also eases you to be a better quitter, which I think is a really good thing. So, I just love the frame that you’re offering. And it’s something I actually have been thinking about quite a bit. I touched on it and how to decide, but I’m really kind of deep into thinking about this problem right now in my life, not my personal life, in my professional life, well, in my personal life as well. But you just distilled that so incredibly beautifully.
Jim: I got another one for you. I think it’s the best talk that I give. I haven’t given it in a while. Michael Mauboussin actually saw me give it one. It’s called “Shoot the Puppy,” and then the subtitle is “The Journey From Idea To Exit in 12 Steps.” And the idea is that each of the 12 steps from the time you have an idea to the time you sell with your company and actually one step past, you sell your company, you should decide whether you should shoot the puppy or not, i.e., not keep doing this, right? And I literally have a picture of actually our dog a beautiful yellow lab with a real gun to his head as the entry screen and the last screen shows me and my wife and the dog on the couch saying, “No dogs were harmed in the creation of this presentation.”
Jim: And you’re absolutely right. I just drew a list that life is an opportunity to lay asymmetric bets, which we’ll talk about in a minute. And if an asymmetric bet looks like it ain’t going to work, the cost of the whole idea of asymmetric bets is you make relatively small bets with the chance to get big wins. And if the big win is starting to go away at a relatively rapid rate, just stop, shoot the puppy, move on to the next one. And in my kind of secondary career of advising CEOs and stuff like that, it’s amazingly hard to get people to do it.
Jim: Maybe I just have weird genetic background or something, but I always found it easy. I was found it easy to sell my companies. I found it easy to shoot them when they weren’t going right. And I would counsel other people. You know who you are Mr. CEO, but a fairly recent little company I was advising, the CEO just I’d give him credit for maniacal commitment, but he went on two years after the board said. “We ought to shut this sucker down.” And in the end, it didn’t work and we all knew it wouldn’t work, but this guy basically wasted two years of his life because he was just too hardheaded. I respect that, but it’s not how I would play the meta strategy of my life, which is to look for opportunities to lay asymmetric bets.
Annie: Yeah. So, here’s the problem that I think we have, which is that for every 10 or 20 guys like that guy, there’s the one guy who somehow pulls it out of the hat and that’s the person we hear about.
Jim: Yep. And you call that hindsight bias, so let’s talk about that. That’s, again, very closely related to resulting, but it’s not quite the same thing. And I want you to tease those distinctions apart for our audience.
Annie: Yeah. So, they’re definitely their cousins, for sure. They might be siblings, maybe. Maybe, they’re their siblings. And so yeah, so hindsight bias is essentially this problem that it’s twofold. It’s when we have an outcome. And we look, we now learn obviously new information, not the least of which is the actual outcome. We think that we should have known that beforehand or in its worst case, we sort of like have this weird memory creep, where we believe we did know it beforehand.
Annie: So, an example would be like this CEO, right? If it doesn’t work out, people look at that and they say, “I knew it wouldn’t work out. I was saying that all along,” right? But you could imagine, I’m not saying you would, but you could imagine some of those fellow board people who you worked with, who were sort of saying, like, “It’s time to shoot the puppy, right? We should really end this.”
Annie: If that CEO pulls it out of the hat and ends up with a successful company. There are some people on that board, who would say, “I knew it all along. I knew this was going to happen. I knew he was going to pull it out. I knew this was worthwhile.” And of course they didn’t, because you’re telling me they were on record as saying, “They did not actually think it was worthwhile.” But people misremember these things. Hindsight bias is incredibly powerful, right?
Annie: And this is kind of part of what’s happening. You can see that when you start to feel the sort of inevitability about an outcome, that the result of that is resulting. Because if you sort of knew all along that that ball was going to get intercepted, if you think that, “Well, everybody could see that coming,” then how could it possibly have been a good decision if it was knowable beforehand? And this is where hindsight bias and resulting are really well-related to each other, right? So, hindsight bias has also been called the creeping determinism.
Annie: And so, the way that I think about it is this that essentially, when you’re sort of thinking about a decision in prospectively, you are thinking about like, “Okay, well, we have this particular option, what do I think the possible outcomes of that are going to be?” And there’s some set of possibilities that could occur and each of those possibilities has some probability associated with it. So, we could think about Pete Carroll’s decision, and “I’m going to pass the ball.” And we know that there’s fumbles in there, incomplete pass, touchdown, or interception. Maybe a sack is in there as well. So, let’s just sort of broadly call that like five outcomes that could occur.
Annie: And each of those outcomes has some probability of occurring an interception, in this case, it’s smidge less than 2% of the time. And we know that when we go into the decision that all of those possibilities are available. But what hindsight bias does is once we know the branch that the world actually took, we essentially take like a cognitive chainsaw to all the other branches and it feels like what actually happened in that particular case, an interception was inevitable.
Annie: And along with that inevitability, you can see how then what will makes sense then that you would go backwards and say the decision must have been poor. Because if that outcome was inevitable, well, then that would mean it was knowable and that anybody could have seen it coming because it was the only thing that could have occurred, because it happens to be the only thing that did occur. So, this is a very powerful cognitive illusion. We do this all the time.
Annie: One of the best examples that I saw, and this was very much on the public stage, was with the 2016 election. So, there are a gazillion articles that are written about Hillary Clinton’s horrible campaign strategy. It was just a huge mistake that if she didn’t campaign in Pennsylvania, Wisconsin and Michigan, right? This was ridiculous. It was so horrible. And then a lot of people sort of claim that they were saying this at the time or that they knew this at the time, what an idiot she was.
Annie: What’s interesting is that when you do the Google search and you just say like, “Okay, fine. Were people say that at the time?” This incredibly obvious mistake that the Clinton campaign was so bad and it was so poorly run and how idiotic could you be. Were people say this beforehand? Because gosh knows everybody had something to say about it at that time. And when you do the Google search, what you find out is no, it was crickets. And in fact, the few articles that you do see written are actually questioning why Trump is campaigning in Pennsylvania.
Jim: I remember seeing those.
Annie: Right. And there’s a reason for that, but both things can be correct. So, the polling and at that time, people didn’t understand that there might be a sampling problem, because it was sort of the first time this really big polling error had occurred. So again, that’s information that revealed itself after the fact. And I assume, no omniscience or time machine here, so let’s assume that was unknowable beforehand that Pennsylvania, Wisconsin and Michigan were all outside the margin of error. But places like Georgia, North Carolina, Arizona, even Texas, and Florida, these were all kind of, New Hampshire, pulling as toss ups. So, if those three states were outside of the margin of error, it seems like at that time, it would have been kind of silly to put a lot of resources to that for Clinton and for her to actually focus on places where she was quite close, according to the polls.
Annie: Now from the Trump perspective, the only way he could win is if there were a polling error. That that was literally his only route to winning. So, if his only route to winning as a polling error, then he ought to campaign as if there was a polling error, which would mean that he should actually be campaigning in places like Pennsylvania. Because if there’s a polling error, those are the places that he would win. And he shouldn’t actually be campaigning in places that are close because if there’s a consistent polling error in the places that he’s closed, he would actually be quite well ahead. So, he would want to pull in like the behind by 4% to 7% range.
Annie: So, it turns out that both campaign strategies could have actually been quite wise, but the point here is that after the fact, you see thousands of articles that people talking about how Hillary Clinton is an idiot, and kind of everybody knew it and how silly of her and before the fact you see none of those. And that’s such a great example of hindsight bias that we just confuse like, “What was the state of knowledge at that time? Was that really a good decision or a bad decision?” And once we sort of know the outcome, and particularly in this case, we find out that there was a polling error after the fact, so we learned some new information. We all act like we knew that that existed beforehand and that anybody who was campaigning should have campaigned as if there was a polling error, which would have made no sense for Hillary Clinton to do. So I mean, this is a big stage where you can see these things happening.
Jim: Indeed. Two thoughts that I had when I was reading about this and also listening to you. First, I wonder if hindsight bias is related to another known cognitive bias known as recency bias. For instance, let’s take the case of the company where the board has been saying for two years, “This sucker is a dead dog. Why are you continuing to flog it?” And then somehow he pulls it out of his you know what and it works? Well, the working is more current. And one of the things we know is that information that’s more recent has higher salience in our memories.
Jim: And the memories of us having a board meeting two years ago, a year and a half ago, saying, “Why are we still flogging this dead puppy?” have gone down in significance. And so, we rate the more recent information more powerfully. It seems to me that maps pretty well on to hindsight bias.
Annie: Yeah, so all of these things are kind of related to each other. I think the issue is that with hindsight bias, it’s not so much that you might not remember saying, “We’re flogging this dead horse.” It’s that you say, “I knew this was going to win.” So, the difference is that, it’s true with recency bias, things that are more recent are more salient, will tend to overestimate the frequency of things that are recent, which creates some really bad accounting errors. It’s obviously a problem. It’s a memory issue as well.
Annie: But this is actually sort of replacing memories with something that didn’t actually happen, that happens to conform to the outcome. And so, they’re related but not exactly the same. And I think that hindsight bias is probably the bigger problem here because if we’re going to think about how can you actually learn to become a better decision maker? Well, one of the requirements I would argue is that you actually remember the past well, because as I’m starting to get feedback on the decisions that I’ve made in the past, it would obviously be quite helpful for me to remember what my state of knowledge was at the time that I actually made the decision, because otherwise it’s going to be very hard for me to learn the appropriate things from it, right?
Annie: So, you can think about it like this, like a poker hand, which happens very, very fast. And I’m sure that you’ve seen this at the table. Somebody’s playing a hand and they can only lose, “I’ve got Aces, you’ve got Queens, and you hit a Queen on the river,” And the person says, “I knew you were going to hit that Queen” Right and it’s like that’s not recency bias. That’s something altogether different, that’s hindsight bias. Now, what the recency bias would do is it might alter the way that I play aces going forward because I recently lost to somebody hitting a two outer. And now, I’m thinking that that’s going to happen to me again. So, they both create really bad errors, right? I would call those more cousins rather than siblings.
Jim: I love it. A beautiful distinction. I like that a lot. The other one I would put forth as a story about the Trump campaign in 2016 actually fits the McCain campaign in 2008 even better is a meta strategy, which I like called, “When you’re behind, raise the variance.”
Annie: Yes.
Jim: And I’m sure you can talk some good stories about that.
Annie: I can. I have a great story about that.
Jim: Yeah. Before I got involved in Poker, because I play poker, but my obsessant hobby when I was from 10 to 20, was playing Avalon Hill war games. These were kind of things people do on computers now, but me and my buddies would do them with cardboard pieces on mat boards and refight historical battles. The Battle of Stalingrad, the Battle of D-day and you rolled dice for the battle results. And there were, especially as much as we played, we were fanatic. We developed a very high understanding of the strategy space, kind of like a chess master, but because there was a lot of stochasticity in it, even if you played correctly, you could still be way behind. And one of the big insights I had when I was about 13, 12, was if you’re way behind, you should stop playing the book strategies, right? Because if you do, you’re almost certainly going to lose.
Annie: Right. The game had runaway possibilities, I assume.
Jim: Exactly, yeah, yeah. You’d lost more battles than you should have based just on bad dice rolls. So, therefore continuing to play sane and sober strategies mean you were 99.9% likely to lose. However, if you go into berserker mode and do stuff that you would never do in an even situation, maybe you had a 15% chance of victory. And so, going from 99.9 chance of loss, because you continue to play the book strategy to saying, “Wait a minute. Book strategy doesn’t apply when you’re way behind and you got to raise the variance.” So, you attack much more aggressively, giving yourself at least a small chance of rolling a bunch of ones, which are good in this particular game, and turning the situation around.
Jim: And so, one could think of the Trump case is pretty good, but the McCain case in 2008, maybe even a better fit where he was clearly well behind Obama, looked like he was going to lose, had some big positional disadvantages that had been eight years of Republicans, it was time for a change, big financial crisis, et cetera. And so he goes and does something really high risk, he picks Sarah Palin to be his Vice-President, which didn’t work. She turned out to be an even bigger idiot than he thought. And yet, when I look at that, from the framing of “Your behind, raise the variance,” it’s not necessarily a bad strategy.
Annie: Yeah, so, gosh. Okay, so I love this because first of all, I have a great story about this, and second of all, I can tie it back to the right to abandon. So, I agree with you. When you are behind, raise the variance unless you can quit. So, you’re talking about situations where quitting doesn’t work, right? You’re talking about game situation or it’s presidential, like you’re the presidential nominee. You’re not going to just be like, “I’m done in the middle, because I’m buying Obama.” That obviously makes no sense.
Annie: So, in situations where there is no available quitting option, then yes, you should absolutely raise the variance. And this actually brings up a problem for us, which actually Kahneman and Tversky have a look at it. It’s called sure loss aversion is that when we’re behind, we don’t like to quit because we don’t want to take a paper loss and turn it into a realized loss. And so, we’ll actually raise the variance in order to try to do that. But of course, what that gets into chasing good money after bad because if quitting is an option, that’s actually the option that you should take. But when you quite can’t quit, you should absolutely 100% do that.
Annie: So, I have a very good story of that. So, I was playing Erik Seidel in the NBC National Heads-Up Championship in 2010. And that tournament is one-on-one poker NCAA bracket style, March Madness style. So, you start off with the round of 32 and then the round of… well, I guess, the round of 64, 32, 16, so and so forth. So I get down to the finals with Erik Seidel. Let me just say Erik Seidel is a much better poker player than I am. So, the finals, you have to play three matches, three heads-up matches, and the best of three wins the championship.
Annie: So, in the first match, I won that match pretty easily, I think because Erik’s model of me as a player wasn’t great. It was weird because he had mentored me, but we hadn’t actually played against each other very much. So he hadn’t had a whole lot of experience with me as a player, which I think, had worked to his disadvantage. So, I had a better model of him as a player than he did of me as a player. So now, we come into the second match and he’s made every possible adjustment to my play, like so fast. I mean, this is why Erik Seidel is so good. It’s like he went off, he thought about what happened in that first match, and he came back and he just like completely neutralized me. It was horrible. And he completely crushed me.
Annie: So going into the third match of the three, I said, “Well, whatever I do, Erik’s going to be able to neutralize me and he’s going to be able to think through these problems a lot better than I can.” And so, this is really bad, so I have to increase the variance. So, how do you increase the variance in a tournament? You play very, very slowly time wise. In other words, you try not to get in very many hands for every half hour that you’re playing, because what happens in a tournament is that the blinds are going up by time, not by the number of hands that you’re playing.
Annie: So what that means is that you’re reducing the number of decisions that you have to make against somebody who is much better than you. And you’re allowing the blinds to climb to the point at which it would be mathematically okay for you just to put all your chips into the pot, which is, by definition, the fewest number of decisions that you have to make. And that’s exactly what I did. I just jacked that variance up to 11. I played really, really slow. I let the blinds go up until I could just get my chips into the middle. And I said, “Let me hope I win this hand.” And I did.
Annie: And I ended up winning that tournament, because I used exactly that strategy that you were talking about. I was like, “Oh, no. I’m in really big trouble here. I better get that. Jack that variance up through the roof. I’m going the volatility strategy here.” And that allowed me to win the tournament. But again, here, I did not have a quitting option.
Jim: Yep, that’s a perfect example.
Annie: Yeah. And if I’m playing in a cash game and I get in that situation, I’m picking up my chips and leaving the table. And we forget that we have that option. That this jack the volatility up should only occur when you have no way to get out, when there’s no quitting option. Because obviously, the quitting option would be the better option.
Jim: If you can, yes. If you can escape, if you have the right to abandon and it doesn’t cost you much, you should take it. And the beauty of poker, [crosstalk 01:12:32] it doesn’t cost you anything to get up-
Annie: And the volatility through the roof.
Jim: Yeah. I love it. That’s a perfect, perfect example. You know he’s a better player than you are if you played a thousand hands, but if you get into a situation where it’s down to three hands, you got a chance, right?
Annie: Yeah. Basically, I was like, “Look, if I can get my money in, if I’m a 60/40 dog, I’m better off than I make it if I play a thousand hands against him. If I play a thousand hands against him, I’m like 100% to lose. So, I’ll take 60% to lose over 100% to lose and then cross my fingers because by the way 40% happens 40% of the time. That’s a lot.
Jim: Yeah. Having a 40% chance of a big payday, why not? I mean, it’s exactly the correct play. Perfect.
Annie: Why not? It’s better than a 0% chance, I’ll take it.
Jim: I’ve got a million more topics to talk about. So, I’m going to skip down to which one of the ones I thought was most interesting and most applicable to actual people, and that is having a group of people to help you make good decisions. Could you talk about your circle of poker buddies and coaches and kind of the rules that you guys evolved on? How you interacted with each other?
Annie: Yeah, absolutely. I mean, I think this is a really good argument for having an excellent executive coach and because the reason is it goes back to this inside-outside view, right? That when we’re on the inside of a decision, we’re seeing it through our own perspective. And there’s all sorts of lies that we tell ourselves. Not in a malicious way. I mean, obviously, I think we all have the goal of being a better decision maker, but we all want to feel pretty good about ourselves. Nobody wants to feel like we’re making mistakes or that our beliefs are incorrect, or any of those things.
Annie: And so, when we’re thinking about all of these like overconfidence and motivated reasoning and resulting, and all this stuff, these are really inside view problems, right? They’re the way that we reason about the world in order to reinforce the beliefs that we already have. Now, the cure to that is to get some objective outside advisors who can really look at the decision in a way that’s different than the way that you would have looked at it, right?
Annie: They’re going to have different perspectives. They may have different information. They’re less endowed to the path that you’ve already taken or the resources that you already spent. They don’t care as much about your identity. They’re worried about their own, in the sense of like wanting to have that positive self-narrative or protect or beliefs. They don’t care.
Annie: So, if you can get that together with a group of people and you can see some real examples of this, like I know YPO will create these small groups of people who come together and talk frankly about the challenges that they have. If you can get a group of people who are looking at your problems from the outside or the decisions that you have to make from outside of your own biases and offer you different perspectives on it. And be willing to tell you in a pretty direct way when they think you’re wrong or when they think maybe you’re resulting, or you’re being overconfident or have you thought about this other frame? This other mental model that you could apply? Have you thought about it from a different perspective?
Annie: That that’s actually what’s going to help you get to the truth and it’s incredibly important, particularly when we’re talking about subjective judgments. Right? So, it’s nice in chess to have people that you work with in order to be a really good learning group. It’s even more important in a game like poker or when you’re making life decisions, because luck really mucks up our ability to understand what is a good decision? What is a bad decision? Why did something occur in the world?
Annie: It gives us a lot of leeway for cognitive bias to really start to take hold. In the sense of like if I lose a chess match, it’s hard for me to come up with any other reason besides I just didn’t play as well as my opponent. And so, I can go and look at that and try to figure out with a good group of people, “What were the other options?” And I can sort of explore that game space. But that’s really different than the problem in poker where if I lose in poker, I can think I played great, but I just got unlucky.
Annie: And that’s why it’s so important to create these groups of people who will look at that in a more objective way than you, so that they can start to say, “Well, what about this, though? Like, maybe you could have made a different play? Are you sure that player is as bad as you think they are?” If I just had a conversation with someone where they were thinking about whether to pursue a contract, is it a yes or a no? Right? We either have to do it now or we won’t be able to do it for 12 months.
Annie: And I said, “Well, why are you thinking about it that way? Is there a different answer to that that would be somewhere in the middle where you could sign it now, but the contract won’t commence for six months?” For example. Where you can get it signed, but just delay what the start date of the contract is. The only reason why I could do that, it’s not that I know that business better than they do or that I’m smarter than they’re, it’s that I’m not anchored to anything. Because I’m just looking at it from the outside and I’m willing to ask sort of idiotic questions, because there’s no reputational harm to me if it turns out that that’s completely impossible.
Annie: Because I just happen to be looking at it from a different perspective, one that isn’t rooted in the history of the negotiations that have occurred or how the negotiation has already been framed. And that’s why you need these outside observers because this is what allows that. All the bias to be disciplined. It’s what allows you to get to the accurate truth so much more quickly. Because if you can’t get what’s living in other people’s heads and you can’t get to other people’s perspectives, frankly, like your decision making is going to be doomed. So, you should be trying to create those groups for yourself.
Jim: I love it. We’re going to have to end it there. We could go on for another two hours. I literally have enough topics out of these new books. So, I’d like to thank Annie Duke, the author of Thinking in Bets and How to Decide for an incredibly interesting, stimulating, I hope, helpful conversation for our audience.
Annie: Thank you so much for having me. This was so fun.
Jim: It really was. One final thing before we sign off. Annie he has a great story about her dealings with a guy named Nick the Greek. Oddly enough, I also have a great story about dealing with someone named Nick the Greek. I would love to have shared those stories, but we don’t have time. Maybe when Annie comes back with another-
Annie: Well, we’ll do it like superfast. Let’s do it in a minute.
Jim: Okay.
Annie: So, I love that you have a Nick the Greek story because I say in the book that if you’re Nick, if your name is Nick and you’re Greek and you play poker, you will be called Nick the Greek. That’s just like a true thing. So, yeah, so I had a Nick the Greek who actually coming off of what we were talking about with “Why you need outside observers.” It’s so easy to start sort of offload the bad results to like, “Oh, I just got unlucky. I got bad turns of the card,” or whatever and then that can create some pretty poor decision making.
Annie: My Nick the Greek had had a few bad experiences losing with aces, and then had come to the conclusion that aces were the worst hand that you could play in the death because you always lost with them. So, Nick the Greek went broke, by the way, I just want to say that. So, this is really the big danger of these types of biases is that you can come to some pretty weird conclusions, so actually change your decision making going forward.
Annie: All right, quick. Tell me your Nick the Greek story, and then I got to go get this puppy.
Jim: I don’t have time to tell the full Nick the Greek story, it would take 10 minutes, but I will say that in my northern New Mexico poker playing, there was a guy, just like Nick the Greek who always played 7-2 because somehow, he had had it hit some time along the way. And I knew he always played 7-2 and I actually sat in, in a hand where that son of a gun hit the boat at the river and beat a high flush with a 7-2 starting hand in Texas Hold’em.
Annie: Well, I wonder if it wasn’t the same Nick the Greek. Because my Nick the Greek having had the bad experience with aces decided that Seven Deuce was the best hand in the deck and played it as if he had aces.
Jim: Yep. This guy’s name wasn’t Nick the Greek. My Nick the Greek story is not something entirely different, but it would take too long to tell. But it was a guy who always played 7-2 and played them aggressively like a moron. He was one of those guys, who were just feeding my $170 an hour win rate.
Annie: My gosh, that’s so great. So, I will say before I go, one of the things that, I use this actually in my consulting a lot is I’m sure you know this. There was always the guy who would be at the table and lean over and the board would end up like ace, 10, seven, seven deuce. And they would lean over and go, “Oh, I can’t believe it. I folded a seven deuce.” And I had a standard response to that, which was, “You know there’s a way to avoid that, right?” And they would say, “What?” And I’d say, “If you play every hand till the river.”
Jim: That’s malicious.
Annie: You will never have. That happens, too.
Jim: That’s what I’d call good strategy advice there, Annie.
Annie: Thank you. You avoid the regret. All right. I’ve got a hop and go get my puppy, but this was like so much fun.
Jim: It really was.
Production services and audio editing by Jared Janes Consulting, Music by Tom Muller at modernspacemusic.com.