The following is a rough transcript which has not been revised by The Jim Rutt Show or by Ken Stanley. Please check with us before using any quotations from this transcript. Thank you.
Jim: Today’s guest is Ken Stanley. Ken leads a research team at OpenAI on the challenge of open-endedness. He was previously a professor of computer science at the University of Central Florida, and was also a co-founder of Geometric Intelligence, Inc., which was acquired by Uber. We had one of his other co-founders, Gary Marcus on the show back in EP 25, which was also a very interesting episode. Welcome, Ken.
Ken: Thank you. Really glad to be here.
Jim: Yeah. Ken is a person who I had been following his work for many years actually. Those who listen to the show regularly know that I’m sort of pathologically obsessed with neuroevolution. And Ken is one of the major dudes of neuroevolution, the University of Texas crew, who were one of the top research centers in that in the world. And Ken is an inventor of probably the most cited neuroevolutionary tool called NEAT, NeuroEvolution of Augmenting Topologies, that is, and its follow-on, HyperNEAT. He’s also developed novelty search and POET algorithms, amongst many others. His main research areas are neuroevolution, generative and developmental systems, co-evolution, machine learning for video games and interactive evolution, quality diversity and open-endedness. Really interesting stuff.
Ken: Yeah. Looking forward to talking about some of it.
Jim: Yeah. In fact, originally when I invited Ken on the show, knowing his work and having read a lot of his stuff over the years, we originally were going to talk about his book for about half the episode. And then we were going to talk about the current state of play of neuroevolution, particularly with respect to AI. But after I read the book, I reached out to him and said, “That book is officially interesting. Let’s split this into two episodes.” So today we’re going to mostly talk about his relatively recent book, Why Greatness Cannot Be Planned: The Myth of the Objective, which he co-wrote with Joel Lehman. In about a month, we’ll have Ken back on a show just focused on the current state of play, maybe a little bit of the history of neuroevolution.
Jim: So before we jump into the book, a couple of bits of conceptual plumbing that I think might be best presented up front. One, it’s funny when I was reading the book, because I do, I create a bunch of, usually about 100 annotations on the book as I go through it in my Kindle. And I must have written seven or eight times, NFLT. Eventually you get around to it, but I think it would be useful as a tool we can refer back to in the discussion, to tell people what the No Free Lunch Theorem is.
Ken: No Free Lunch Theorem. Sure. No Free Lunch Theorem is a theorem about the ability of optimization algorithms to solve arbitrary problems in general. And basically, the idea is that there’s no algorithm that can be good for solving basically every possible optimization problem that you might face. And that means basically … In more simple language, it basically means nothing can be good at everything. So if you become good at certain types of problems, then you’re going to pay by being worse at other types of problems. It sort of was a wake up call for optimization, generic black box optimization, to say like, “Look, even if you’re doing really well on some things you’re probably paying for that by doing worse on some other things.”
Jim: I could tell my friend, Dave Wolpert out at the Santa Fe Institute, when I meet people in the data sciences or machine learning area, I immediately peg them as do they deeply understand the No Free Lunch Theorem or don’t they. And it’s quite interesting that people fall into both buckets. Here anybody claim they have the magic answer for anything, with an algorithm, I go, “No Free Lunch Theorem.” And they look kind of a little bit sheepishly and go, “Oh, yeah. Right.” But it’s really important, particularly in the context of what we’re going to be talking about today, is that there is no guaranteed right answer in general. Though, of course, the other fall off the No Free Lunch Theorem is that in any given domain, any knowledge of the domain can be used to select and pick and tune your algorithm. There is hope it’s not just endless chaos out there, but always one must be careful not to over promise about approaches or algorithms. The other concept, which we’ll probably refer to a few times comes from evolutionary computing. And that’s the idea of exploitation versus exploration. I don’t think either of those words are in the book, but maybe say a few words about that concept.
Ken: Yeah. In optimization and machine learning algorithms, these exploitation and exploration often come up and they refer to kind of two sides of the same coin is when you’re trying to look for something, sometimes what you do, you could use the word exploit, which means that you are moving in a direction that has increased your reward. It’s so things seem to be improving. And so you continue to move in that direction and that’s sort of an exploitation. In exploration, the idea is that you would kind of deviate off the path that has been so far pointing you towards increasing reward. And the idea there is that you’re exploring, so you’re trying to see maybe if there’s even better things that aren’t necessarily on the path that looks best right now. I would note though that, that distinction or dichotomy, actually, as we get into the book, will turn out to be lacking some nuance, it’s actually a more nuanced than just those two things. But I imagine we’ll get into that a little bit later.
Jim: Absolutely. In fact, I was going to make that point, which is that we’re going to talk about today is a step beyond those. And in fact, I’ve said this on the podcast multiple times, the first time I applied complexity science more broadly and evolutionary computing more specifically in my business career was right around the idea of exploitation versus exploration on a fitness landscape for mergers and acquisitions, believe it or not. And it actually worked. I wish I had the tools that we’re going to talk about today to think about at the end though, on the other hand, we’ll talk about this, when does this open-ended approach, is it really practical? So the alternative to open-endedness perhaps a more nuanced than just the straight hill climber, hill climber, meaning it just always exploited, just keep doing incrementally better, better, better.
Jim: There’s also the trick of exploring, which is intentionally moving in a direction that doesn’t send a signal that we’re getting better, better, better. But that may allow us to get better, better, better later. So I think of it as the alternative to Ken’s idea of open-endedness. So, yeah, the myth of the objective, which is the subtitle of the book in this case is actually bang on. I mean, that’s essentially the core deep idea here. Talk about what you mean when you say an objective and maybe point out how amazingly pervasive it is in how we think about the world.
Ken: Yeah. The book is really about the fact that in our society and in our culture, almost everything we do is guided in some way by objectives. So what do I mean by objectives? I mean, objectives means when you set a specific goal and measure progress towards that goal, that you can call that your objective. And then you think about some examples of things that are driven objectively, it’s like when you’re getting graded in school. I mean, you’re basically trying to maximize that grade. So you’re trying to get the highest grade you can. Or basically, if you want to get a job, like your objective is to get that job. If you want to raise your income, your objective is to get a higher income. So there’s many things that are driven by objectives. There’s a lot of the sort of societal level or institutional decisions are also objectively driven. Like, should we fund this or should we fund that ultimately and usually hinges on well, what is the objective? And do we believe this objective is achievable?
Jim: Yep. And as you point out in the book, it’s more than just the obvious kind of business things. Can we make more nails in the nail factory this month? What’s actually might be a case where incremental improvement is the way to go. Get better and better at making nails. But it slips into everything. From when we think about who we marry and dating. As you say, to education. Our careers. As I read the book, I had never really focused on the fact that it was really quite pervasive how 21st century, at least Western human beings have taken objectives to heart and organized our life almost entirely around them. So what’s the downside of that? At first it says, “Hey, getting better all the time.” That sounds good. What’s wrong with that?
Ken: Well, that’s a lot of the point of the book is that there is a big downside to doing things that way, even though at first it might seem like the obvious way to go. But the problem is that if everything is arranged around objectives, then it effectively cuts off the ability to explore and to do things for reasons that are actually different from objectives. Like for example, to do something simply because it’s interesting. And ultimately, that has a really big cost because not only … I mean, at the superficial level, like, okay, you’re not having as much fun, but at a deeper level, the problem is that there are many things that we would like to accomplish in this world, in our society, or as individuals, that we can’t accomplish as objectives because we just don’t know how to do them.
Ken: And it’s really curiosity and exploration that ultimately offers some hope that we might actually uncover some of the stepping stones that we need to achieve those much more kind of long range, blue sky types of discoveries. So objectives can actually be getting in the way of more kind of serendipitous discovery that would be available to us if we weren’t so obsessed with objectives.
Jim: Yep. And you give a very nice example from the Picbreeder project, which unfortunately it doesn’t seem to work anymore. At least it didn’t work a couple days ago and it didn’t work again this morning when I went to the site. But why don’t you tell us what Picbreeder is and how it worked and why it’s a very interesting example of how objective-based behavior misses a whole lot of possibilities?
Ken: Yeah. This is a pretty counterintuitive claim, that objectives are problematic. So it’s not something that I would have just naturally decided to promote or to be creating as a cause. If it hadn’t been for the fact that we did some experiments in the field of AI, basically, where we actually started to see some serious, serious problems with objectives that were unexpected. And really, the key place where that started to emerge was this thing called Picbreeder. And Picbreeder was a website that we put up many years ago, which is why we basically now it’s starting to have some trouble running anymore, but hopefully we can at least have it up. But when we put it up, it was a very novel system. So basically what you could do, it was you can go to this website and you could actually breed pictures.
Ken: And breeding pictures is just kind of a strange concept, but it’s sort of like breeding dogs or breeding horses. You could take a picture, you could have asked the picture to have offspring and the offspring would be a little bit different from their parents, just like with animals. And the cool thing was, though, that if you actually bred a picture into something kind of interesting, then you could publish it and it would go back to the website and other people could see that and they could actually breed from there. So people were breeding from each other’s discoveries basically.
Ken: So this was kind of a fun game and maybe a toy for some people, but ultimately it was really profoundly revealing in a way that I hadn’t necessarily expected when we started to notice this underlying phenomenon, which is just kind of mind-blowing, which is that when we looked at all of the discoveries that were really interesting on the site, and those are things like images that look like things that you see in the real world, like butterflies or cars, our skull. There was a beautiful skull that was discovered. And we looked at those images and we looked at like, well, how did people actually discover these things? How did people breed towards them? And we discovered that actually, in just about every single case, it was only when they weren’t trying to get those things that they got them.
Ken: So this is very counterintuitive. It sounds very strange. You can only find things by not looking for them. This is ties in directly to this issue with objectives, because of course, if that’s true, then it would be a really bad idea to have a particular image in mind as your objective and then try to breed towards it because you almost certainly fail because the only way to get to things is by not trying to get to them. So the implications of that are really broad. If that’s actually a general principle, that often the only way you can get to certain kinds of discoveries is by not trying to get to them, then setting objectives can be actually really bad for you if you want to discover those things, which are possible to create or discover, but just don’t happen to be convenient to discovering through objective types of search.
Ken: So Picbreeder kind of was the first reveal of this principle. And we can go into like how did we know this? I mean, obviously it sounds really strange. How did we figure this out or noticed this phenomenon, but just to begin with, I can just say that we did and it was pretty mind-blowing to me, because it basically contradicted everything I’d ever learned about how achievement works. When you go to school or getting engineering degree, I mean, you got to know what you’re trying to do in order to do it. And this just worked in the exact opposite way.
Jim: Yeah, quite interesting. Now, it’s interesting, I contemplated a similar project 19 years ago, where I was going to have an art generator that basically threw out, I think it was 50 lines drawn with Bézier curves essentially. I don’t remember if they were quadratic or linear or what. And basically they were redefined in terms of artificial DNA and you would do a four by four matrix of them and you’d select the one that looked most like what you were trying to do. And what I was going to do as the proof of principle was tried to imagine a picture of Abraham Lincoln’s face and see if I could start with 16 random sketches and converge towards a picture of Abraham Lincoln, but I never did the project.
Jim: Sounds like, based on the write-up in the book, that you actually did experiment with trying to get people to go for a specific objective rather than just to look for what might be interesting. Maybe if you could address that a little bit, that would be kind of cool.
Ken: Yeah. Once we noticed that this kind of phenomenon was happening, where people were discovering things by not looking for them, of course, we were very interested then in what happens when you actually are looking for something. And we actually did a lot of experiments along those lines. We can speak anecdotally about how hard that is. If it turns out that if you go to Picbreeder and you have a certain type of image in mind, maybe you want to get a picture of like say an apple or something like that, it’ll be really, really hard. You’ll probably fail, which is why the site was actually frustrating for a lot of people, because you see all these images there and you think, oh, this is going to be fun. I’ll just create whatever I want. And then you can’t.
Ken: But beyond just kind of this anecdotal evidence, we also did experiments where we intentionally set objectives for an automated process, meaning that we basically started a search algorithm without humans in the loop where the objective was a particular target image. And we didn’t try to use evolution because after all, evolution is what’s used inside a Picbreeder, just to move directly towards that objective based on image matching. And those also would fail miserably except for the very, very simplest images. And we understand why, which is probably more important than just the evidence, because understanding why it tells you sort of why this makes any sense at all.
Ken: And the reason is because if you think about it, the stepping stones that lead to the kinds of things that you want, in this case, images, don’t necessarily look like them. If you want to get to a skull, the bad news is that the things that lead to skulls don’t look like skulls. So that explains why it would be a really bad idea to always have the skull in your mind and say, “This is my objective.” Because as you move towards the skull and you see things that don’t look like skulls, you would discard those things and avoid those things. But those are actually the critical stepping stones that you need to cross to get to the skull. And in fact, that’s true for like every, just about every interesting image on the site. And it’s not even a surprise if you really think about it.
Ken: I mean, in a complex search space, of course, there’s going to be some deception, which is what we call that kind of a trick where the stepping stones don’t actually look like the final product, because if there wasn’t, then it would be extremely easy. I mean, all hard problems or effectively deceptive, otherwise they wouldn’t be called hard problems. And hard problems exist in complex spaces. So in some sense, this is a reasonable and expected situation, but the thing that’s sort of interesting and unique about it is that we are seeing in such a stark contrast that the objective is so pathological because in our culture, we think objectives are the means to get anywhere. And here you see the exact opposite where they’re actually something that stops you from getting anywhere or stop you from getting everywhere. So this is something to ponder, whether this actually applies beyond just Picbreeder.
Jim: Yeah. And we’ll talk about that later on. To get a sense of just how amazingly large the space of possible images is, I just did some simple calculations. Let’s take a real simple case of 1,000 by 1,000 black and white image, just black or white. The number of possible images is two to the million. What the hell? How big is that? Well, take it to what we’re thinking of powers at 10. It’s 10 to the 100,000. One followed by 100,000 zero was if I did my logarithms correctly. I think I did. And that compares to the number of fundamental particles in the universe. What’s the number of people? 10 to the 80th. Oops. 10 to be 80th. Fundamental particles of the universe. Number of quarks, number of electrons, number of protons, versus 1,000 by 1,000 black and white image, which would be considered low rez today, 10 to a 100,000. And of course, your space was even more … Was it gray scale or was it color?
Ken: Color. It’s color, yeah.
Jim: It was colors. So that was even … Not that it makes much difference you, the competent dorks explode to a ridiculous number anyway. And you had a very interesting metaphor. You called it the room of all images. Might be a Bourgeois library essentially. Maybe if you’d take us through a little bit about what the idea of the room of all images is and how the idea of stepping stones somehow emerges from that thought experiment.
Ken: Yeah. I mean, you could talk about the room of all images or even the room of all possible things. All the things that you could ever invent and if you imagine them all together in some vast, vast space. We put them in a warehouse, warehouse the size of the solar system or something or bigger, then you can imagine that that room would have some kind of organization, You’d imagine that computational devices would be in one corner and paintings would be in some other corner and they’re near each other in some way. So there’s some logic to the organization of that room. And the logic that sort of guides the organization of that space, which is the space of basically all possible things that can be created, is kind of the logic that you need to understand in order to find things in the room.
Ken: And what’s interesting though, is that the organization of that giant room of all possible things is not going to be intuitive. There are going to be some things that are near each other that initially don’t make sense. For example, the fact that, vacuum tubes lead to computer. So we can imagine in this room that there are vacuum tubes sitting near computers, because in the real history of computation, the first computers were made with vacuum tubes. But you wouldn’t think that you should go to the part of the room filled with vacuum tubes if you wanted to build a computer.That’s counterintuitive. And so this kind of phenomenon would happen over and over again. So the room is kind of a metaphor for the organization of the space of what’s possible and how just ultimately counterintuitive that organization edge, which is why it’s very difficult to navigate that room through objectives alone.
Jim: Yeah, it’s just too damn big, in some sense. It’s just like so many possibilities. Although I do want to have to toss out that while the vacuum tube was one road to computation. If we’re thinking big and open-ended, we should remember there’s at least two others, which were the analog computers of people like Vannevar Bush. He didn’t use vacuum tubes. He used gears and axles and all sort of stuff. And the other, and I did this little fun research and see where this stood, you can actually build the equivalent of a digital computer using electromagnetic electromechanical relays. And people have done so and can continue to do so. So while the space that we actually live in, the land of the transistor did go through the vacuum to, there are, no doubt, other roads to computation as well, which actually reinforces your story.
Ken: That’s going to be true with just about any complex space. I mean, with pictures, of course, there was a certain path that was followed to find the image of the skull in Picbreeder, but certainly, there’s other possible paths. No one’s found one, but of course, there’s going to be an infinite set of paths because there’s the infinite set of trajectories that could lead to the same point. But one thing we can be fairly certain about, this is important, is that almost just about all those paths are going to be counterintuitive in some way. So it doesn’t help you escape from this, the fact that there may be multiple paths to something. If it wasn’t counterintuitive, if it wasn’t deceptive, it wouldn’t be a hard problem.
Jim: Now, you’ve mentioned deception twice. Let’s drill into this concept a little bit, because this turns out to be pretty fundamental to the rest of the story. And I think one of the easiest ways to get at least a simple-minded version of what deception is, is to talk a little bit about the robotic mouse in the maze.
Ken: Yeah. The problem with deception is that problem that when you think that you’re moving in the right direction, you’re actually not. Or the opposite would be when you actually think you’re moving in the wrong direction, it turns out you actually were on the right path. And the mouse in the maze is a kind of a metaphor for that. In this robotic mouse in the maze, we put a robot in the maze and you can’t see beyond the walls. So it’s basically, you can’t see where the end point is, but we can tell the robot how close it is to the end point. And that’s sort of like a little bit of a signal for the robot to know if it’s going in the right direction or the wrong direction. If you think about it, it’s basically a stand in for the objective.
Ken: So it’s like the end point in the maze is the objective. And we’re telling the robot how close it is. And this is a very important metaphor because this is basically how we run things in our society. We basically say, “Let’s set up a metric and let’s measure how close we are to where we want to be. And that should help us guide us to getting to that basically exit from the maze.” And the problem is, though, if you think about it, this is a very simple thought experiment and it comes out pretty bad. The problem is that, of course, you can move towards the goal and actually see your distance to the goal go down, even if you’re actually walking straight into a cul-de-sac or a dead end. And that means that like, yeah, you’ll eventually, you’ll keep on getting a higher and higher score and then just be completely stuck because you can’t keep moving forward. So you’ve been deceived. It looked like things were going well, it looked like everything was moving in the right direction, but it turns out you just ran into a brick wall and that’s basically what deception is.
Jim: Now, to probe on that just a little bit, on that example. Yes, a really dumb look at my GPS and see how far I am from the goal mouse will almost inevitably fail in a maze. You’d have to add a lot of noise to get them out of it, but there are also like quite simple algorithms that will allow a mouse to relatively efficiently navigate a maze like Trémaux’s algorithm, which essentially marks where it’s been and if you revisit the same intersection twice, you back up the way you came. That remarkably simple algorithm actually is guaranteed to solve a maze, even a pretty deceptive maze surprisingly quickly. How does the idea of these simple algorithms enter into the idea of deception?
Ken: Yes. So you have to keep in mind that it’s just, this is just a metaphor, really. The intent here is just to use this as a metaphor. And you have to consider that basically when we say like, “We don’t know the way to get through the maze.” It’s actually, the reality, as you pointed, the practical reality of like actually getting through a maze is that we do know something about mazes and we can exploit that understanding to create an algorithm that does, given enough time, have a chance to get to the end of the maze. But the thing is that we’re just using this as a metaphor for problems where we don’t know about the search space. That’s the important point here.
Ken: If we don’t know anything about the search space and it’s very high dimensional, unlike a typical maze is two-dimensional, this is going to be … You could be in 100-dimensional, 1,000-dimensional, a million-dimensional search space if we’re talking about the real world. The maze is a good metaphor when you just have this very naive kind of beacon, which says, “Okay, am I getting closer or not?” Like the GPS, because that’s effectively what we’re doing. I mean, it sounds almost ridiculous. Would you actually try to solve a maze that way in practice? No. But that is actually how we try to solve like hard real world problems is we have these very, very simplistic metrics for like, are we maximizing the profit for the year? It’s like a single measure. And then we use that to determine whether this giant multidimensional search is moving in the right direction. Just like the naive mouse in the maze.
Jim: Yep. I can give you a personal example, the reality of that perspective. One of the bits of advice you get from people for managing your career is always get a pay increase when you make it move between companies. Sort of a commonplace. And I was always a contrarion and three different times I actually took pay cuts to make a move. One time, 40%. And all of them turned out to be the correct move. So if somebody had been applying rigorously climb the hill, always get a pay increase, certainly never take a pay cut. Certainly not 40%. I would have missed out on the most interesting parts of my career.
Ken: Right. Yeah. That’s a great example. Yeah. I mean, if you want to make outlandish amounts of money, probably it’s not a good heuristic to just be like, “Okay, I just need to maximize my salary.” Because the stepping stones that lead to extreme wealth just have almost nothing to do with that. So that would be a deceptive measure.
Jim: Yep. It seems to be absolutely true in that case. The other example I’ll give, and I love it because it’s just so homey, is the Chinese finger trap. Remember, when we were kids we used to buy them for a nickel at the drug store. Probably more than that now. You stick think your finger into both sides of kind of this woven straw thing and the trick is if you try to pull your fingers out, you can’t get them out. So why don’t you talk about that a little bit and how that’s another very simple example of deception, maybe help bring the idea home for people.
Ken: Right, right. Just to intuitive metaphor. I mean, it’s basically a very, very simple search problem. You’ve stuck your fingers into this trap. And if you pull outwards, that seems like you’re moving in the right direction, because that’s basically where you want to go, is out. But it turns out that the trap actually gets tighter the farther you pull out. So if you measure your distance from your goal, it’s getting closer. So it seems like things are going in the right direction, but actually you’re going in the exact opposite direction as the one you should, because counterintuitively, that’s why it’s called a trap, actually, the way you’re supposed to push is inward, where it seems like you’re actually moving towards being trapped, but you’re actually not because the trap is designed to open up when you push inward. So it’s just deceptive by design. And it’s a very simple form of deception. But if you think about it, it’s so simple and yet people get stuck in it and fooled for a while. I mean, imagine like a real-world problem, which is like a million times more complicated than that. I mean, the deception that we’re facing is just overwhelming and pathological, if we’re guided only by objectives.
Jim: Yep. Now, you do make a point that, and this is sort of at least analogous to the exploitation exploration issue. It’s not exactly the same, but call it analogous, is that let’s say the warning to be suspicious of objectives is true for big leaps, but may not be true for relatively small leaps. Could you maybe riff on that a little bit?
Ken: Yeah. I think it’s an important concession, because it sort of distinguishes the point that the book is making from an important commentary to just like a crank, which would be somebody who would say like, “Just forget all objectives in life and stop having objectives entirely.” That is not a very wise thing to do. And we have to acknowledge that in order to have any credibility, because it’s obviously true, especially in the case where your objectives are relatively nearby or the way we put it, a stepping stone away. Like if you want to get lunch, you can make that an objective and that’s very reasonable. You don’t need to just wander around your house in the event you might find something you didn’t expect, that wouldn’t be very smart. You just go to the refrigerator and make your sandwich.
Ken: So there are a lot of things. And things even a little more complex than that. Maybe you’re a company and you’re trying to upgrade your software to the next version. That can be an objective. That’s not incredibly ambitious. That’s not an incredibly large leap from where you already are. That kind of stuff can work objectively. We’re talking about things where we just don’t know what the intervening stepping stones are to get where we want to go. Something like curing cancer, or creating AI, for that matter. There clearly we have no idea where we have to travel on the road to getting to that particular objective. And those are the places where this kind of objective paradox, as we might call it, come into play.
Jim: Yeah, very cool. I thought that was very interesting. Though, I would just quibble a little bit as a former entrepreneur and business dude and all that stuff is, yeah, we’d look more than one link out, look a few links out. But if you try to go too far, you’re just wasting your time. And one of the truisms I advise young entrepreneurs on is basically innovation in the business technology space, kind of I think is optimally two things. One is recombining what currently exists. Brian Arthur has a really good book, The Nature of Technology, gets into this in some detail. In fact, we’re going to have Brian on the show next month that much of what we think of as innovation is recombining existing elements, I think. For instance, the current popularity of electric bikes. There’s no magic in an electric bike. It’s just a unique combination of stuff that had gotten good enough and cheap enough for other reasons, which we’ll talk about in our next example and no real magic.
Jim: But I noticed that there’s a zillion electric bike companies and most of them are failing. So when I advise people to look for an interesting opportunity in business, look at something that has a novel combination of existing technologies plus one new thing. Exactly one new thing. Not less than one new thing and not more. Because if you have to solve two new things, you get into this crazy land where you’re wandering around and don’t know which way is up. If you have no new things, you ended up like the 500 electric bike companies, of which 497 will probably be bankrupt in five years. So it’s a little more nuanced than just one step. But that said, let’s go to the very interesting example, which kind of starts drilling really down to the heart of, I think, the vision that you’re trying to communicate here, which is the idea of the computer and what would happen if you wanted to develop a computer 5,000 years ago.
Ken: Yeah. So that’s a thought experiment to consider the implication of objectives. And there’s a number of stepping stones that need to be crossed to get to a computer. As you pointed out, there’s actually probably more than one path you could take. But for the sake of argument, we know that there are certain things in modern computers that you need. You need, for example, electricity, and you could say maybe vacuum tubes were one of the paths that was important. And there are all kinds of things obviously to go into computers. And so the problem is, though, let’s go back 5,000 years now and just forget about those things, those are the intervening stepping stones, and just think about the final goal, which is a computer. And then ask, why did we wait 5,000 years to do this? Why didn’t we do this 5,000 years ago? So let’s go take everybody like 5,000 years ago, all the smartest people who are around them, get them together in a special kind of Manhattan project and say, “Look guys, there’s something that’s possible that you may not have really thought of, which is a computer. Why are you wasting time working on whatever you are working on right now? Let’s get to work on this.”
Ken: And the interesting thing is like, if you did that, the problem is that all the stepping stones don’t exist yet. There is no electricity. That hasn’t been discovered yet and among a myriad of other things. So what you would do is you would basically destroy all of that mental capital, which is all these people with all this potential who in their own time could discover all kinds of things wouldn’t do that anymore. Because now they’re all going to just be thinking about computers, but they don’t have the stepping stones they need and they wouldn’t make them because it wouldn’t occur to them that you need to make those things because no one has a clue what things lead to computers.
Ken: So not only would you not get computers, but you also wouldn’t get the stepping stones and you would waste all of the effort and the resources that those people have to offer. It would be a gigantic disaster that you tried to do that. So it’s sort of a thought experiment on illustration of, in some extreme case, how damaging it can be to try to impose objectives on people who otherwise would be doing something totally different that was driven by curiosity in a different direction.
Jim: And yet, this is the cool thing, this is where my eyebrows really started to go up when I was reading the book was that, but nonetheless, we did go from 5,000 years ago to computers without a plan because people were doing it’s called local optimizations, which we’ll talk about in a few minutes, novelty and interestingness. And somehow we created enough stepping stones to get to the point let’s say 1940 or so where people said, “Okay, it’s one or two or three moves away to have a computer. And that would be kind of cool for fighting the Nazis so we could calculate trajectories for …” I think it was anti-aircraft cannons originally. And that gave the world the incentive to take two or three steps and go from 5,000 years of non-planned movements towards the computer to a final plan burst. When I got that in my head, I got, ah, this is what Ken’s talking about.
Ken: Right. That’s the fascinating thing about this, is when you consider that even though it would be crazy to go back there and tell all those people to build a computer, somehow over the intervening 5,000 years with no plan, it happened anyway. And actually, that’s like the only way it could have happened. It couldn’t have happened by just going back there and telling everybody to build a computer. It had to happen in this organic unplanned way, which is kind of why the title of the book is Why Greatness Cannot Be Planned. And what it shows is that there is another way that amazing things are achieved a much more kind of mysterious and less talked about way, but it’s not a random way as some people would characterize it. It is principled. This kind of exploration that leads to these incredible ends, it follows certain principles, which is something that the book tries to lay out. And that is something we should grapple with, like this alternative way towards really, really blue sky types of achievement, which is almost like magic. I mean, because it did happen. We do have computers today.
Jim: Absolutely. We’re going to go there. We’re going to do quite a big, deep dive into all right, this can’t possibly be a random walk. I mean, because we know there’s 10 to 100,000 images and we ain’t never going to visit any measurable percentage of them at random. But before we do that, let’s talk a little bit about some of the subjective aspects of this. You say in the book, and this is a direct quote, “Why do so many of us feel our creativity is stifled by the machine-like integration of modern society?” And then go into it a little bit longer and talk about how this is just in some ways crushing many of us and that things like our careers. And you give some very interesting examples of not being so rigid in our careers can have some really interesting outcomes. I thought one of the very interesting stories was Johnny Depp. Maybe you could tell us that story.
Ken: Yeah, I guess Johnny Depp, it was … I hope I have all this right from my research, but was originally interested in being a musician. And obviously, we’ve seen that he likes music. Obviously, he’s been in bands in recent years. But his original career path was that direction. So he wasn’t thinking about acting and it was just some kind of serendipitous connection that was made. I think it was through a girlfriend or something like that that led him to somebody saying, “Why don’t you try out for a part?” Or something like that. So he ultimately came to this opportunity and was opportunistic in order for you to take it, that was not on the radar of his original objective, which is a very common story for successful people, I think, which is interesting because it’s aligned with the theme of the book.
Jim: Yep. And in fact, another quote is, “A peer reviewed study found that nearly two thirds of adults attributed some aspect of their career choice to serendipity.” And I can say my own story in my career was, again, driven by this kind of thing. I was, frankly, a disgruntled youth when I graduated from college, did not really want to participate in the grind of the world and did a little this, little that, hitchhiked around the country, sold cars for a while to pay off my student loans. And then I got the easiest job I could find, which was … You were a college professor. You remembered us pesty college textbook paddlers. Remember those guys? I was one of those guys. It was a real easy job. Left me plenty of time for my own projects, my own thinking, writing, et cetera, and it was perfect for that kind of disenchanted youth who didn’t really want to play the game too hard, but wanted to make enough money to live modestly.
Jim: But here I was doing this in 1979 or maybe late ’78 as I just wandered around, visit 15 professors a day like we’re supposed to do. And unfortunately, once you got good at being college textbook paddler, you only had to do it about 12 weeks a year. The rest of the time you could do whatever the hell you wanted. Anyway, I started seeing these things on professors desks, like what the hell is that? He said, “That’s a computer.” And I go, “What?” And this was in the earliest days of personal computers and the things you’ve never heard of since, North Star Horizon, MITS, MSI, et cetera. And I go, “Wow, this is interesting.” And I’d done some computer stuff in college, not a lot, but enough, to know that I did not like the culture of computing in 1975 when I graduated from college, which was the IBM mainframe, punch cards, all that sort of stuff. And I go, I want nothing to do with those people.
Jim: But suddenly when I see this idea of your own computer, I go, “Holy shit. This is really interesting.” And if I hadn’t had that job at that time, suppose I was a brake shoe salesman going around to repair garages in 1978 or ’79, there’s no way I’ve likely to see a computer, but I happened to be a textbook paddler. I happened to see professors, often engineering professors, sometimes math guys, who had these little computers and I said, “Shit, that’s interesting.” Interesting, important word. So I started researching and I started hanging out at computer stores and stuff. And then the next year I said, “This is interesting. This is something really interesting and important here.” And I went and plunged down 90% of my net worth, $4,500 as I recall, for a totally loaded Apple 2 and a high end graphic capable dot matrix printer. And in fact, I think I was the only guy in Lexington, Kentucky at the time who had two, kind of two floppy drives.
Jim: So people loved to come over to my apartment to copy software. Because it used to be a little bit of memory, oh, it’s already been flopped, but if you had one, that was a ridiculous thing to make a copy of software. So anyway, there was a perfect example of right place, right time, serendipity. But this is … Now we’ll transition to, I think one of your more interesting concepts, is interesting. I had somehow made the conclusion that personal computers were fundamentally interesting and I made a jag in that direction and went on to write the world’s champion, a fellow program, and then got involved in the earliest days of the online information industry and the rest, as they say, was history. All frankly, because I saw professor X … The first one I saw was actually in the math department, the University of Kentucky, but interesting was the magic word. So in the book, you talk a fair amount about two topics are kind of related. Let’s dive into both of them, go whichever order you want, but that’s the idea of novelty and the idea of interestingness.
Ken: Yeah. And a great story, by the way. It was a really great example. Let’s think about interestingness for a second, since that’s sort of what that story really touches on. The concept of interestingness is really important in the book, partly because as you said, there’s this issue with subjectivity, I think, that makes people very uncomfortable. Particularly scientists, but also bureaucrats as well. People who run things like to be objective, makes them feel more comfortable. So it’s like, how do I know that what you are trying to do is going to work? Well, I need some kind of objective metrics so that I can actually measure and know that we’re actually moving in the right direction and so forth and so on.
Ken: And yet, like in your story, there’s so many steps that you take that actually leads to great success in the long run down the road that don’t actually involve some kind of objective metric. Actually, there’s a kind of an interesting duality of the word objective, if you think about it. An objective is something that you move towards, but also, objective is this word that we use to describe things where we can actually measure and observe them in some way that can be shared. So science talks about objectivity and we need to be able to be objective in order to understand the result of our experiments and so forth. That’s a different notion of objective, which is sort of in contrast to subjective, where it’s just like, well, how does it feel to me from my subjective point of view when I consider a contemplate going down this path versus that path.
Ken: And we just don’t trust that at all, not in any kind of consensus that we have about how things should be run. Individually, you may have a strong kind of inclination towards your subjective perception, but as a society, we really don’t trust subjectivity. And almost nobody would accept if you just told them this is your subjective impression. It’s okay, we go to your boss. So this is what I think we should do. Well, why should we do that? How is that going to help us? Well, it’s just, it’s interesting. Let’s just do it because it’s really interesting. Well, that’s probably not going to fly because it’s completely subjective. It’s like, don’t tell me it’s interesting. Tell me exactly how this is going to affect the bottom line, what is actually going to happen?
Ken: And the problem is that actually, this obsession with objectives is also an obsession with objectivity, which prevents us from utilizing when we actually have really good subjective intuitions. When you saw that that computer was interesting, whatever that means, because it’s subjective and it’s hard to measure what exactly that means, that was a very important insight, I mean, for your life, but probably also for society at large to see the potential that was there. There’s often cases where people, most people don’t see the potential in something and only a few people see it early and that’s a subjective impression a lot of the time and that’s a very important one. And humans are really, really good at that. And that’s something that the way that we run our institutions doesn’t really give us credit for, is the fact that we’re very good at having subjective intuitions about what’s interesting or not interesting and it’s just completely unrelated to all of this objective measurement stuff.
Ken: So that relates to what’s interesting. It’s like the ability to say what’s interesting is fundamental to achievement and discovery. The ability to realize what I can do or to actually realize the potential that something has depends on my ability to subjectively intuit what’s interesting right now in the here and now. So somehow there have to be pathways that people can follow where they can actually follow the interesting. So this is, and I think this is actually suggesting some social change, because of the fact that we don’t really, at least formally, sanction those kinds of pathways, except in rare exceptions. We’re so paranoid about subjectivity that we just don’t want to sanction it.
Jim: Yeah, definitely. Since then, I’ve always sort of just followed my nose to what I thought was interesting and used that as my main navigator where I decided to live, who I decided to date, eventually marry and the work that I did, the things that I’ve invented. I’ve always used that because eventually I only don’t give a damn about things that aren’t interesting.
Ken: Yeah. Yeah. Yeah. That’s what I’m saying we’re really good at, is following our nose for the interesting. We have a nose for the interesting as people. And that’s not just a fluffy statement. If you really get into it, what that really means is, look, if you spent your life learning about some subject, if you got really good at it, then you will have intuitions within that subject that are hard to formalize or put into objective terms or maybe impossible, but they’re still really, really valuable within that subject. As a computer scientist, I can have all kinds of intuitions about computers, but they may not be objectively justified, but we should still take them seriously because of the fact that I am trained in the area and have a lot of experience in the area or in artificial intelligence, for example.
Ken: And this is where we have a huge blind spot, I think, in society is that we don’t trust experts or people who have a lot of experience in something to use their expertise subjectively. We only have trust them to be objective, which is really silly when you think about it, because it doesn’t take any education to be objective. If I have to provide you a metric to show you, to prove to you that we’re moving in the right direction, anybody can follow that metric. You don’t need to have seven years of a PhD or something like that in order to know that the metric is going up. That’s like a simple measure in a single dimension. So we’re sort of denying people all of the intuition and that kind of subjective ability that built up over years and years of becoming familiar with something, something intangible, by assuming that all of that is irrelevant and should be ignored. We can’t trust you because you think something is interesting in any formal way.
Jim: As you were saying that, I was thinking about the fact that there’s kind of an interesting support for that from cognitive neuroscience, which is our conscious let’s call them system two brains in the common man topology, where you’re thinking rationally about something is remarkably low bandwidth. I think I’ve seen calculations on the order of 50 bits a second. And truthfully, we don’t do arithmetic as well as a $1 calculator and we don’t do objective very well. We look at the law lists of these posters of human cognitive flaws and all this stuff. Not only is it are we slow at it. We’re not very good at it. On the other hand, our unconscious mind is, I think, as you put it in the book, the most complex artifact we know of in the universe, competes at an unbelievable capacity in parallel, in a very strange way. Not at all, as you know, as like digital computer does. And things like our intuition are not necessarily gated by this very low bandwidth in the conscious objective brain. So that might be another argument for trusting your intuition, at least as much as your supposedly objective mind.
Ken: Yeah. That’s a good connection. Yeah, we should put more stock into this more subconscious, more intuitive part of our mind, which is responsible for so much of the discovery and advance that we see around us.
Jim: And of course, it’s actually more involved in everyday work than we think. Antonio Damasio has done a lot of great research on the fact that even the most basic decisions, like what do I have for breakfast are driven by our emotions and our unconscious processes to an amazing degree. And if people have damaged the parts of their brain that do that kind of thing, they can’t decide what to have for breakfast, no matter how smart they are. So learn to trust the unconscious processing, at least to a degree, and maybe use the objective as a cross-check. So now let’s switch and talk about something that maybe is more amenable to turning into actual software, which is novelty search.
Ken: Yeah, this segues well into novelty and all searches algorithm that I worked on with Joel Lehman, who is actually the co-author of the book as well. I think the step that we have to take to move from interestingness to novelty is to start thinking about this in terms of, could we formalize what we’re saying more algorithmically? If we’re saying that, okay, it is actually very promising and sometimes essential to discover or achieve certain kinds of blue sky types of achievements, to be able to follow paths of interestingness, you could say gradients of interestingness. Well then, is there some way to write this as an algorithm? Basically like as a recipe for how would you do this? If I could actually write this down for you as a guide. And it turns out that like, it becomes … We want to be able to do that because we want to write algorithms and I’m in the field of machine learning.
Ken: In machine learning, we want to be able to write algorithms that capture all of these facets of human intelligence, including the non-objective. And then we have powerful learning algorithms, but it becomes challenging when you get to the question of what is interesting, because that is very difficult to formalize, it turns out. I mean, it’s almost like an AI complete problem, like what is interesting. And everybody, of course, has a different view of what’s interesting, which is what’s so powerful about like the fact that we have so many people that have so many different interests pursuing so many different things simultaneously. But if we want to write an algorithm that’s going to sort of follow pads of interestingness, we need to formalize it in some way.
Ken: And it turns out that novelty is a kind of a decent proxy for interestingness. It’s not as good. So if I could actually write down an equation, which basically expresses what is all the things that are interesting, that would be better, but I can’t, and nobody can. It’s extremely difficult. We don’t know how to do that. So novelty is sort of like a proxy that’s kind of the second best option, which is to say, if you think about it, almost everything that’s novel is interesting. Sorry, almost everything that’s interesting is novel, but not everything that’s novel is interesting. So it sort of, it goes in one direction in a very positive way. So we know that like, if it is interesting, it’s almost certainly novel. Things that are not novel or almost never interesting.
Ken: The idea of a car was extremely interesting and compelling like in 1900, but right now it’s not that exciting. You’re not going to start saying, there’s a really interesting thing when you’re at a party. You could put a box on four wheels and drive around in it. I mean, that’s not interesting anymore because it’s not novel anymore. So novelty alone has a lot of stock in terms of following things that are interesting. And what’s nice about novelty is that you can formalize it pretty easily. I can sort of say, “How different is this thing that I’m contemplating from something that I’ve already encountered in the past?” So I can write down like some function that will compute novelty in that way. And then that can be used as a proxy for interestingness. So I can follow the gradients of novelty in reality in an algorithm on a computer, without a human intervening, I can actually write something that can do that.
Ken: And this then allows us to see some glimmer, I think, of what a true interestingness driven type of search would look like, which is which we could call the novelty search algorithm. And this algorithm tries intentionally to follow paths of novelty without any explicit objective. In some way, at first it’s interesting just as an algorithm, like you can do some things that are interesting, but it’s also in some ways an embodiment of what we’ve been speaking about more at a philosophical level, at a more kind of formal algorithmic levels. Can you write something down to make this a little more concrete? Well, yeah, we can write an algorithm, it’s called novelty search, which follows the paths of novelty, which is a proxy for interestingness, and has no final objective. And yet, it will actually achieve things that are in some cases cannot be achieved objectively, which helps then to highlight and validate this theory, because now we’re actually algorithmically showing that this principle comes into play rather than just arguing through hand-waving, which novelty search does. It actually does solve problems that when we try to solve them objectively, we either get worse or no answer, which can be very surprising.
Ken: Think how counterintuitive that is. I tell the computer what I want it to do. I want learn how to walk for this robot, say, for a simulated robot. I tell it, “I want that. Walk as far as possible.” And it’s actually better if I don’t tell it what I want it to do. And I just say, “Just do novel stuff.” It actually learns to walk better. That’s I think a very important demonstration of the underlying objective paradox. And now in a more formal algorithmic sense.
Jim: And as I understood it, tell me if I’m on the right track here. One of the ways novelty search works is by exhausting things that don’t work?
Ken: That’s true. It will go to places, try things and then it will remember them. So that’s an important part of the algorithm, is it sort of, we call it an archive. So keeps them in an archive. So it’s, I’ve done this. It leaves a description of what it did in the archive. And then it will reward itself for staying as far away as possible from where it’s already been or from those things it’s already done. And that’s what’s pushing it towards novelty. So it has sort of this implicit drive away from things it’s already done or already tried towards somewhere else, somewhere more blue sky.
Jim: Yeah. You gave a nice simpler example than walking of a robot trying to navigate down a hallway. Maybe you could draw that picture for us a little bit. Might be a little easier for people to get their arms around.
Ken: Right. And that’s basically like just taking more leveraged from the little metaphor of the mouse in the maze that we talked about before or the robot mouse in the maze. In this case, what we said is let’s put the robot mouse in the amazing in a real simulation that we’ll put on the computer once you simulate it. And we will put a neural network inside of that robot, which will control it. And what we’re hoping to find is like the right configuration of that neural network to help it get through the maze. That’s the problem. You’re looking for a brain basically that will get it through the maze. If you then try to just reward it for how close it gets to the goal, which is just like what we talked about a little while ago, in terms of deception. Well, yeah, you’ll you run into a deceptive problem. Actually, a lot of the time you can get up to a point where it will actually get stuck or it will take a very, very long time to find a brain that can actually run it through the maze.
Ken: But if instead, you reward it for finding some novel path that it’s never taken before. Well, then very rapidly and quickly, it will find all the paths through to the maze to everywhere, including the end. And actually ends up solving the maze, finding a brain that drives the robot through the maze much faster than if the algorithm was rewarded for actually getting closer to the end of the maze, which again, is very counterintuitive, but it’s a fact it’s much better to reward it for novelty, for doing something novel then to reward it for getting close to the end of the maze, if your goal is to get to the end of the maze. So just another demonstration of this principle that sometimes it’s just way better to follow the path of interestingness.
Jim: Interesting. Now, let’s take one more step into something like theory or at least abstraction across this idea. And you say that information, accumulation and increasing complexity are the telltale signs of any kind of search without an explicit objective. So you’re saying there that increasing complexity and information accumulation are good tells for things that are moving in the right direction or moving in interesting directions. We’re not going to use right direction, because we’re not going to talk about objectives.
Ken: Right, right. I think what I’m saying … They might be good tells, but I think the point is more that they kind of come along for the ride if you follow this kind of principle. By this kind of principle, I mean, like following trajectories of interestingness, rather than just objective types of trajectories. You’ll get increasing complexity and information accumulation, which is a very, very interesting side effect and very important. So it’s worth thinking for a second, what that actually means. What is information accumulation? It means that look, imagine this, I put a robot in a room with four walls. There’s no way out other than a door. And then I tell it to just keep on doing new things. Well, at first it’ll just crash into all the walls and then every time it does it’s new because it crushes into a different wall, but eventually to get out of the room. It’s going to have to figure out how to open a door.
Ken: And in order to do that, it’s going to have to use its sensors. Eventually it’s going to have to actually use its sensors. It’s going to learn to identify the door and learn how to open the door. Actually like it is forced. Because it is being pressured to do something new, it’s forced to accumulate information about the universe. What is a door? How do my sensors work? What are the elements of my environment that can be affected by my actions? And if you continue with this, like in the extreme, like you keep saying, “You have to do something. You have to do something new.” I mean, you’ll eventually land on Mars. But the thing is like, if you land on Mars, then you learn all kinds of things. You’ve learned about what planets are in the solar system. You’ve learned about propulsion and rocketry and all these other things. So that’s a side effect of just pushing for more interesting or more novel stuff.
Ken: And it’s partly an explanation for why like our drive towards novelty like socially, it’s also pushing us towards increasing knowledge constantly. And that is also increasing complexity because in order to integrate that increasing knowledge, we need increasingly complex systems. So you get a lot of interesting stuff. Instead of just saying, “Am I getting closer to the objective?” What you get is increasing understanding of the universe and how it works just as a side effect of not pursuing objectives.
Jim: Interesting. So I suppose in the real world, I think about back of the business dude, you also have to take a look at you’re constrained with resources. So you have to ponder when a novelty search is likely to be good enough. I mean, let me take the No Free Lunch Theorem. We know it’s by definition not a panacea. So can you give us some kind of guidance? I mean, they’re probably going to be very rough and hand-wavy at this point, but when it makes sense to use novelty search versus other kinds of more brute force kinds of searches?
Ken: Yeah. Yeah, it’s a good question. I mean, with all we’ve discussed it, it sounds like a great celebration of novelty search and sort of like the solution to everything, but it is not, of course. And the caveat that’s really important to keep in mind is that it is really about risk. I mean, in order to have great rewards, you have to take great risks. So if you’re in a situation where you can’t afford to take risks, then yeah, just doing things because they’re interesting could be pretty risky. That could actually be a bad idea. So this kind of search is really only possible when you have the resources and the cushion where you’re willing to take the risk that you don’t end up somewhere interesting because we’re taking risks.
Ken: Now, if you don’t do this and you only follow objectives, then you’re much, much less likely to find anything interesting. So this is why it’s still a good idea, to the extent that we can afford to, to follow gradients of interestingness. But if you’re going to do that, you have to bear in mind and you should be aware that you’re intentionally taking a risk. If you take that job that has a lower pay because you think that maybe that’s going to lead to something more interesting, obviously you just got a pay cut. So you’re going to have to live with that and you have to accept that that actually might not pay off. It’s not guaranteed. There’s no guarantee in novelty search that is going to pay off.
Ken: So, yeah, the novelty search approach is basically saying, if you have a stomach for exploration, it can be extremely high payoff, but you have to realize it’s not guaranteed in any way to do so, but nevertheless, it may be the only way that you’re going to get to certain kinds of discoveries. So if you’re willing to take that risk, you may actually have a very high payoff if you’re lucky.
Jim: Yeah. And I think the other thing, and I was going to think about this heuristically, is when experimentation is cheap, it could be done in parallel. That’s a pointer towards using novelty search and things like computer-driven exploration of algorithmics is a good example or in general, exploration is relatively inexpensive and can be done in parallel.
Ken: Yeah, actually, and I should also … There’s one other caveat that that reminds me, that’s important to point out, which is the big difference between like a successful novelty-driven search and a successful objective-driven search is that we don’t know where we’re going to end up in the novelty search, obviously, because there’s no objective. It has to be kept in mind. What we’re offering here is not that you will solve problem X. You have a problem. And then you just say, “Oh, well, I’ll just do whatever’s interesting.” There’s no reason to believe that’s going to lead to solving this single problem X.
Ken: What we’re saying is you have the opportunity to solve an undefined problem that we’re not really sure yet you’re going to end up solving. If you look at like the issue of the computer sitting on the desk, while you’re delivering people or you’re buying people’s books. It’s not clear what exactly you’re going to accomplish because you find that so interesting, but you did accomplish something. And that’s the thing, is that the ultimate accomplishment is undefined. So you have to be able to live with that ambiguity if you’re following a path of interestingness or novelties. We’re not really sure what we’re going to accomplish. It’s not meant to be an objective solution.
Ken: I’m not saying like, “Okay, well now I want to figure out how to get into grad school. So I’ll just wander around and do interesting things and hope that it will happen.” Well, something else really cool might happen, but that doesn’t mean you actually will get into grad school.
Jim: Yeah. If we’re going to go down this road, you have to understand what it’s about, that you have to sort of cast loose of any single objective and make the bet that interesting somehow is more interesting. We got a couple of case studies. We get kind of short here on time. This has been such an interesting conversation, which got three that we could do. One is education. The other is science, what you call that, I think innovation. And the other is AI itself. We probably have time to do two of them. Which two you think would be the most interesting?
Ken: Let’s see. I guess we can try education and AI.
Jim: All right, let’s do it. Personally, I have a great interest in the sociology of science, but we’ll put that off for another day. So you have a whole chapter in your book on how our educational systems at all levels are really bound up with objectives and there might be a better way. Why don’t you tell us your thoughts on how your way of thinking might allow us to, as you say, unshackle education?
Ken: This series of revelations that we outlined in the book … I mean, they’re revelations for me. These weren’t things I was expecting to discover starting with Picbreeder and they felt like revelations. I eventually started to see a connection between these and the educational system, which was maybe surprising to me. I wasn’t trying to say anything particularly innovative about education, but I started to see that this relates a lot to education because education is very objectively-driven. When we talk about schools failing or losing a competitive advantage to other countries or things like that, what we’re talking about basically are test scores, and that the test scores are not going up. They’re not going in the right direction.
Ken: And that’s really interesting because that’s really, that’s a very stark kind of description of an objective-driven search. If what we’re saying is that the path that we want to follow is the path of increasing test scores to the point where I guess every child in the country gets 100 on every standardized test or something like that. Well, that’s like a stark objective-driven pursuit. And what these principles that we’re discussing suggest is that these kinds of objective-driven pursuits are doomed to failure because of deception. In other words, like if I find a path that causes test scores to increase, then almost certainly I’m running into a brick wall and they will stop increasing at some point. At some point probably soon, way short of the goal of everybody getting 100, because it’s an unbelievably complex search space.
Ken: The point I’ve made is that this is a principle that applies in complex spaces. Simple spaces, you can follow the objective. No problem. That will work. But education is unbelievably complex. I mean, everybody recognizes that. That’s why it’s, again, that’s why it’s a hard problem. So it looks like what we’re really doing is just naively falling for the objective paradox as a society collectively across the entire educational system, which is basically driven by tests as an objective metric to tell us where to search next. There’s a very political issue here like with tests and things like that. And I just want to like highlight that I’m not coming at this from this political angle, like I’m coming at it from just … Because a Picbreeder, novelty search and these things, just what I see here is that following a metric like this naively is basically what you’re doing is just going to a local optimum and getting stuck.
Ken: So there’s probably a better way to think about like … And we know that none of these things ever work. That’s why this is like a perpetual problem. Every few years, there’s some new national initiative with some new test standard and it’s like, well, now we’re getting really serious this time. But I would always predict that’s not going to work, because again, it’s just back to doing the objective paradox. You’re following objectives, but they’re going to prevent you from getting where you want to go.
Ken: So we need a non-objective way because this is a blue sky type of a problem. The thing that really like catapults all of the children in the country to some amazing level of achievement is really blue sky and the stepping stones are unknown. And so to be able to navigate a space like that, what you need is a more novelty like search, which is what we’ve seen. And so it suggests very different kinds of approaches than these metric-driven kind of objective ways of doing it, which I guess to put it in a nutshell to me, because I think this does imply concrete solutions rather than just, I don’t like the way things work so let’s just think of something different. But I think there is an alternative at hand, which is that what we should really do is foster diversity intentionally in a systemic way, which means that every teacher has their own intuitions.
Ken: Again, we’re back to subjectivity and intuitions, which makes everybody scared. So we’re going all going to get scared, but they do because they’ve been teaching for years about what would work best for their kids or their class and all of those, in aggregate, all those intuitions and all those different ideas are an extremely valuable resource. And what we should do is try to collect, because novelty is all about diversity, is to try to exploit that diversity and reward it in a sense where the stepping stones become available to everybody else. So we should have a system whereby if you actually develop something, then you can … You develop an approach and it’s helping you locally with your children. That should be possible to disseminate widely and other people should be able to build off of that and try it with their local group. And then all of these different types of ideas will perpetuate, many of them novel, across the entire system.
Ken: Just like a Picbreeder where all the images anybody ever discovered are on the site and can be built on and branched from by anybody else who visits the site. And then we’ll have a Picbreeder like search in education space, and we will surely uncover a lot of innovative ideas that otherwise we will never see and will never see the light of day because everybody’s stuck on this objective metric.
Jim: Yeah, horizontal communications are results so that people can pick and choose what works. Certainly, a very important part of these kinds of parallel, social science experiments, but it’s hard. Imagine if every single teacher has their own approach. And there’s 200,000 teachers in the country, a million, I don’t know, they can’t possibly follow each other. You had a very interesting little algorithm for how to do networked propagation of best practices so that each person didn’t have to follow a million different teachers. Why don’t you lay that one out for us? I thought that was pretty clever.
Ken: That’s important. Yeah. Because there’s a straw man version of this where it’s like, okay, well, the solution is like we have a giant database of 10 million different ideas and you just go in there and just hope you find something useful. That just isn’t going to happen or it isn’t going to work. And it’s not really fair to what I’m proposing. Because that’s just makes it too easy to dismiss. I mean, what you really want is something like a system of peer review, where the amount of interactions is tractable. It’s not in the millions, but every person is basically touching like maybe less than a dozen at max other people with their ideas. So we can say that each teacher, instead of going through standardized testing, like this is the alternative, goes through, some kind of peer review on some regular basis, maybe annual basis or something like that, where they put their portfolio and their strategy together. And then it’s reviewed by peers and there can be some minimal criteria for like avoiding utter failure.
Ken: There’s still a little bit of degree of objectivity. If it’s an absolute failure, then clearly we need to do something to change how things are going in this classroom. But short of that, what the peers have an opportunity then, they also have some small set of other peers that they’re reviewing, to look at the whole case of this teacher, how they approach their year, how well it worked. And then to elevate those things that look really promising or interesting or innovative that they haven’t seen before. And that way feedback goes in both directions. The peer reviewers get to see new ideas and to rate them. And then perhaps like the very best, the highest rated things will be published to some more centralized locations where everybody can see them, but also the people being reviewed then are also getting informed by the experiences and ideas of the peer reviewers. And we have a network and across that network, new ideas and many ideas are propagating, but without everybody having to be aware of everybody.
Jim: Yeah. I thought that was an extremely clever way to handle the horizontal communications problem and find what works problem without an unrealistic every to every connection, which just wouldn’t work. So maybe think about this in terms of practical terms, in terms of that could be implemented is have a radical voucher system where anybody can be a teacher, at least for one year and have this horizontal peer review. And perhaps you have the ability of the peer reviewers to … A super majority of your peer reviewers could revoke your teaching license if you’re a total failure. That might be an interesting way to maximize the amount of diversity, yet still have the horizontal communication and still have a way to weed out the obvious incompetents.
Ken: Yeah. I mean, I don’t know if you even need to say everybody can be a teacher. We can still have some credentialism. I mean, if it makes people uncomfortable. I mean, because I think like the real barrier to doing these kinds of what some people might perceive as radical types of changes to a system are basically everybody’s really worried about guard rails, whereas what we really want to get here is innovation. As soon as you start creating an innovative system, it’s like, “Well, wait, wait, wait, where are the guarantees that things are still working? How do I know that everything’s not going to collapse and it’s going to be complete garbage.” I think we can get a little bit too wrapped up in that and it prevents us from doing anything innovative, but it’s just this whole like risk reward trade off.
Ken: I’d be okay with like saying, “Okay, well, yeah, you still have to have credentials of a teacher, experience as a teacher. but within that, we’re going to have this system, which is like of peer review and of a checks and balances still.” So there’s still some checks and balances, but it’s going to allow new ideas to percolate and it’s going to give you freedom, more freedom. And we’re all afraid of freedom. We want to be bound by objectives, but I’m telling you, if you give people more freedom, they’ll get better ideas and they will percolate around. So we have to take a little bit of risk to get a lot of reward.
Jim: Yeah. I think that’s a very interesting general template for thinking about social innovation. I think it’s very worth those folks who are working in the area of social innovation, which includes a lot of listeners to this podcast. I’d say a third of our audience is people that are interested in social innovation. This is not an easy concept to get your head around. It took me a while. Frankly, at first when I started to read a book, what the fuck is this? I was kind of expecting a book about neuroevolution, to tell you the truth. But then as I got into it, this is goddamn interesting. And as I got deeper and deeper into it, I would say it’s actually worked to change in how I will think about these kinds of social change problems. And then particularly as education one is a really neat example. That’s relatively easy to understand and is not dissimilar to a lot of problems, the kinds of problems that we need to fix in our society.
Ken: Yeah. And I mean, if any listener is interested in social innovation or education, feel free to get in touch. I would love to have some more impact in that area with the book. The book’s touched a lot of different areas, but I think one of the areas where I’m less happy about the impact it’s had, it has been education. It just hasn’t really had much impact there. And I wish that this discussion was more elevated in education coming up.
Jim: Unfortunately, I’ve done … Tempted to do some stuff in education. It’s really hard. It’s the most backward department in every university with one or two exceptions like Harvard and Columbia. It’s 50 years behind and its understanding of cognitive science, horrible bureaucracies, rigid unions. There’s a whole bunch of structural problems on why education is really hard to change it, which is frankly an argument for something like what you’re doing. A radical change to entirely different paradigm.
Ken: Yeah. Yeah. True. Yeah. I mean, and I do. I do see why it’s so hard. Yeah. Education is just a tough place to do any kind of innovative … Especially for an outsider who really, honestly doesn’t understand the system and how the bureaucracy works. So I just don’t know what to do, but I do think it’s ripe for change.
Jim: Yeah, I think an awful lot of people agree with that. Let’s go to our last topic here, which is one that has a certain amount of irony to it. As you point out, AI experts like yourself, or what are they fundamentally, but experts in search. And yet, you laid out the idea of kind of the metaheuristics. What should we be searching about in the field of AI? So why don’t you take us down that road? I thought that was as a person, who’s a bit of a participant around the edges, at least, of the AI and the AGI community, I found it very interesting. And why don’t you lay out that story for our listeners.
Ken: Sure. And hopefully, maybe this will also resonate for scientists in general a little bit, because it’s this problem and everybody’s aware of it in AI, that it’s a very benchmark-driven field. If you want to get into one of the top conferences or journals, but these days, everybody just cares about conferences in AI, then you’re going to need to improve on some benchmark. There’s something where you’re going to have to show that you’re doing better than the state-of-the-art. That obviously leads to a lot of incrementalism. You’re saying this is like 0.1% better than this. And therefore I should get in here. And reviewers tends to be mostly along for that ride and well, as long as you’ve shown that, then you have a chance at least, but if you haven’t shown that, you almost have no chance. It’s like, well, do some more experiments and come back when you’re ready.
Ken: And if you think about what that is, I mean, it’s another case, just another example of an entirely starkly objective-driven search process, which is the field itself. That’s why I use the word meta. We’re talking about how the field as a heuristic searches for the best ideas. And it’s basically using an objective metric, which is performance on some benchmarks, to decide whether progress is worth sharing with the community at large, whether it’s going to get through the publication filter. And that’s just, it is a kind of ironic because if there’s anywhere where that kind of heuristic should not dominate, it would be in a field where the people are experts in search. That’s basically AI. I mean, for people who aren’t familiar, it’s really, a lot of it is about search algorithms and search techniques.
Ken: These people who are very sophisticated and understand like why you would not want to be trapped in local optima and how important it is to avoid those things. And yet, as a field at the metal level, we act like the most naïve local optimization algorithm that’s like basically dying or asking to be trapped in local optima very soon, because all we care about is just incrementally pushing up some metric. And particularly, I mean, maybe the converse is really the right way to say. It’s not so much that we only care about pushing up the metric, but that we really, really don’t care about you if you don’t push up a metric.
Ken: Then like, yeah, you’re probably going to be told to just come back when you do. So if you, like in the sense, just have something really interesting, but it’s not better in any objective sense, you’re out. I mean, nobody’s going to hear about that stepping stone that you’re offering. And as we’ve argued throughout this conversation, I mean, that’s poisonous for innovation. We can’t share interesting things. The response might be from some stereotypical defender of the community is like, “Well, how do we know that they’re good, if it’s just subjective? It gets risky. That’s why we need these benchmarks and these metrics.” And we’re back in a circle again, because of course you can always argue that, but what I’m arguing is that eventually we have to be able to accept that some things are interesting in their own right.
Ken: And it is actually orthogonal and independent of an objective metric and yet still a very important guidepost for where we might want to explore. So our inability to embrace that or grapple with that whatsoever, because we could. Reviewers actually could engage with what’s interesting and what’s not interesting. It’s not theoretically impossible. It’s just that we’re afraid to do it. It means that we’re basically going to be acting like a naive search algorithm and constantly getting trapped in local optima unless we break out of this rut.
Jim: It’s quite interesting. It’s funny because I’m not a professional. I’m not going to get my stuff published in fancy journals. If I appeared a little conference proceeding, that’s fine by me. I tend to skip articles that are, oh 1% improvement on image net. Who gives a fuck? To tell you the truth, I look for interesting stuff. For instance, the ones that I’m always looking for and you see very little anymore is neural architectures that use longer range and stronger versions of recurrence. Everybody is in these little mini recurrences, if they use a recurrence at all. Most of the stuff is feed forward these days. And there just doesn’t seem to be enough exploration, even in the narrow field of neural computation.
Jim: So benchmarks, obviously a problem. It is highly ironic that people who would know that that’s going to lead you to a series of global maxima. Nonetheless, that’s how they’ve organized their profession these days. But the other one I think was even more interesting in some ways, though maybe not as pervasive. And that’s the theory measure. Maybe you can take us through that a little bit.
Ken: Yeah. Yeah. In the book we talked about these two kinds of measures that are used. And I focused here so far on the benchmark one. And I think that’s come to kind of dominate the field recently, just because of deep learning and it’s very empirical. But at the same time, at least historically, there was also another kind of heuristic, which was like, well, we really also like theoretical results. And that means like a proof. You can prove something is better than something else or that it guarantees something. Then that is another indication that that’s a path we should go down. But it’s just another, to me, it’s just another example of the objective fallacy, which is that just because you have some kind of objective evidence of progress, that doesn’t necessarily mean you’re not heading into a brick wall. It’s just another way of trying to be very objective. In this case, to improve. Well, how more objective can you be than improve?
Ken: And to basically give us a feeling that we have some kind of safety net around us and we’re not going in the wrong direction, but it’s just as possible to be deceptive. I mean, that’s the part that it’s sort of hard to acknowledge or even see that like, even though you’re being extremely objective by actually like speaking in the language of proof, I mean, it’s a definitively true or not true, what you’re saying, that it’s still, ultimately from a meta perspective, not necessarily a good heuristic for knowing whether we’re heading in the right direction, because it doesn’t mean it’s interesting. And sometimes you have to go down before you go up.
Ken: I mean, that’s basically the point of deception. Sometimes the mouse in the maze has to back up and go down a different route and actually go farther away from the goal before it can get to the goal. So all of these kind of objective principles where we just follow increasing guarantees or increasing score, ignore this fact that you have to back up sometimes in order to move ahead. So what is the heuristic for knowing when it’s a good backup? I mean, sometimes that is you should be able to acknowledge that this is just super interesting and opens up a whole new world of possibilities. And those can often be the best contributions.
Jim: And in fact, you propose a potential partial solution, which is, and this is your journal idea.
Ken: Right. This was to be a thought experiment. It might actually be interesting to have this journal, which is we called it the Journal of AI Discovery, I think. But if there was a journal where we say that, basically … So first of all, hope is that it’s a prestigious journal. So we’re going to get the best scientists in the world to do the reviews. It’s still, everybody’s good that’s involved in this. But it’s different from any other journal in the sense that the reviewers are not allowed to talk about the results. So the reviewers can only talk about the idea, but not the results as they’re reported. So this would actually, at first it sounds kind of crazy. The reviewers can’t talk about the results, but the results are what we always use to know whether we should publish something.
Ken: But what it does do is that it then prevents, it basically puts a short circuit in this like pathological circuit that we have where it prevents this kind of objective fallacy from coming into play because reviewers are no longer allowed to use their security blanket. Because I think after all that these kinds of objective measurements are security blankets for reviewers because it makes it so they don’t have to think. If I can just say, “Oh, well, this score is better than that score. Good.” Or, “This, you didn’t compare X to Z. You just compared it to Y. Why don’t you compare X to Z and then come back.” I don’t have to think. I don’t have to understand anything about your idea. I don’t have to engage it at all. I don’t have to say why it’s interesting or not interesting or how it relates to anything. I can just ignore all that. And just be completely objective, sound like I know what I’m talking about and then move on to the rest of my life.
Ken: But if you had a journal like this, where you can’t do that because you can’t talk about the results, then you’re in a very uncomfortable position of actually having to engage with the idea and whether it’s interesting. And I think like the best scientists certainly can do this. We just don’t want to. It’s harder. It takes longer. It’s more awkward. And also, you feel like it’s not objective so you’re worried about it, but we could do it. And if we did do it, arguably, I think much more interesting things would be published in this journal than elsewhere, because it would be entirely about engaging with what’s interesting rather than some kind of like naive, local optimization of a benchmark.
Jim: I thought that was such a clever idea and I have a background in publishing and what have you. And I actually generalized your idea and said, “Ah, I think this problem is pervasive across science. Someone should start the interesting journals company and take your idea and apply it to every domain.” If it wasn’t for the fact that paid high dollar scientific journals are kind of in their sunset period, one hopes. Who knows? I might actually hire someone to go do it, but that seems to me a sunset industry. So probably not a good business opportunity at this time.
Ken: Either way, I would enjoy it. Or make it a conference then. I don’t know. But yeah, if somebody would do that, that would be really fun.
Jim: Maybe we should talk about that. I’d be game to help get that off the ground. I know lots of top scientists in all kinds of interesting fields. You could even have the top one, which is interesting results across all fields, right?
Ken: Yeah. Right. That would probably get a lot of readers. Yeah.
Jim: Kind of the equivalent of what science and nature used to be. Right?
Ken: Yeah. Yeah. True.
Jim: Anyway, I thought that was a very, very interesting and provocative and unexpected idea. And certainly got me thinking. Well, Ken, this has been a phenomenally interesting conversation and I got to say, this book turned out to be more interesting than I initially thought it would be. Interesting. There’s that word again. And I can recommend it to people really quite sincerely as something that people that are thinking about science, thinking about social change, thinking about AI, well worth reading. So why don’t you tell us the title again?
Ken: Okay. It’s Why Greatness Cannot Be Planned with the subtitle, The Myth of the Objective from Springer.
Jim: Ken Stanley and your author was Lehman, right? What’s his first name?
Ken: Yep. Joel Lehman.
Jim: Joel Lehman. All right. Well, thanks again, Ken. This was a whole lot of fun.
Ken: This was great. Thank you. I had a great time on the show. Thanks for having me.
Jim: Yeah, look forward to having you back next month and we can talk about neural evolution.
Ken: Looking forward to that. Definitely.
Production services and audio editing by Jared Janes Consulting. Music by Tom Muller at modernspacemusic.com.