BKC's 3 Questions | Jonathan Zittrain

Are there areas where a model or a system would be capable of substituting for a human but we still wouldn't want it to? >> [music] >> Hi, I'm Jonathan Zittrain. I teach law, computer science, and public policy at Harvard. And in 1996, with Charlie Nesson, I co-founded >> [music] >> the Berkman, now Berkman Klein Center for the Internet & Society. And uh have been along for kind of a wild ride. >> [music] >> I think too much attention is given to hallucinations by large language models, of which there's no shortage. But I'd say put that aside. You know about hallucinations, and yes, they are a danger. It's why you probably don't want a large language model flying your plane or giving you legal advice unreviewed. But I'm thinking there's not [music] enough attention given to the kinds of answers that these models [music] produce not Forrest Gumpian box of chocolates, well, who can say what part of the training data of 13 trillion tokens led to this particular answer which might or might not be a hallucination, but rather answers that they're giving because as part of the fine-tuning process >> [music] >> they were basically instructed, uh yes, when you're asked about Tiananmen Square, this is the right answer. >> [music] >> And the ways in which these models are getting bolted, not just retail, although that too, uh into so many facets [music] of people's daily workflows, their information consumption, [music] their agenda setting of like, what should I care about? What what is happening in the world right now that should draw my attention? And as they turn to the models where the models just appear in front of them and offer, "Hey, I haven't talked to you in a while. Would you like some advice right now or can I help you find lunch?" Whatever it might be, I find myself wanting to give more attention to the [music] answers that are calculated and driven, offered in the voice of Claude or GPT or Bing/Sydney that have a company behind it or a government behind the company or anybody who figures they can pressure that company. There's a lot of contestation going on there. And kind of shocked at how little we are talking about the amount of trust [music] being accorded to these models knowingly or not and the way in which what they answer >> [music] >> transcends being a customer service issue or customer satisfaction issue by a company offering a product, but rather a question about what reality are we trying to form [music] for people that may or may not map to what's actually going on in the world. So, what do universities have to offer here? Um unlike the development of the internet, by far the locus of development for modern AI has been >> [music] >> the private sector, Silicon Valley. And AI hasn't had as much in its modern form this like interregnum [music] that the internet had where the people really into it saw it was going [music] to be very powerful and had opportunity to kind of shape it before the gold rush happened and before it raised [music] the interest and often ire of legacy interests, for better or worse. And AI everybody's [music] paying attention to now. And there are just natural ways in which having commercial development [music] and the competition that fuels innovation and progress there are ways in which that's good. If Gemini Pro's answers aren't as satisfying to people as GPT's, well, then more people will use GPT. [music] Like, there you go, competition. But if we're looking at societal impacts whether the stuff in the headlines about AI psychosis, people becoming over-reliant on these models, they're too satisfied or >> [music] >> too dependent, they may not be satisfied, but they're compelled or impelled. Really hard to expect the cigarette company to encourage people not to try cigarettes. And that's just one example of the ways in which leaving development entirely to commercial competition sometimes [music] that race to the top can be a race to the bottom. Academia can play a useful role if done right in producing evaluations >> [music] >> suggestions that might not be what a lab would want to hear from a commercial standpoint, but really important to know about. [music] And finally academia has been one of the key places [music] institutionally where values are debated earnestly, openly. And when I think back to that first law of digital governance, we don't know what we want. We can't agree on what we want. We got to try. We can't just leave [music] it to some folks around a conference table at OpenAI or Anthropic or anywhere to just say, "All right, well, you know, the four of us have kicked it around. >> [snorts] >> We figured out that answer to >> [music] >> should I break up with my boyfriend?" I have been thinking about AI as a kind of triad, [music] um including among experts uh in the topic, >> [music] >> uh which is to say, you've got accelerationists, who for various reasons are go, go, go. We want to develop this stuff as quickly as we can and maybe deploy it as quickly as we can, [music] too. There are the safetyists, people who say it may well be that because the thing we're developing is, by whatever metric, which is contested, intelligence, and we're asking that intelligence, among many other things, to create [music] its successor, intelligence makes more intelligence, which in turn creates a successor, and you get turtles, smart [music] turtles all the way up. That's a recipe for things not particularly going according to whatever plan humans have. And that is one of maybe, I don't know, half a dozen distinct failure [music] modes that safetyists are prepared to tell us um could be catastrophic, even existential. And then you have uh what I call the skeptics, um who often object to the name, and >> [music] >> I'm open to a new one, but the skeptics are some combination, in differing degrees >> [music] >> of thinking that AI's progress has been overhyped, whether overhyped for the purpose of it's going to cure this and help with that, the accelerationists, or overhyped >> [music] >> by the safetyists saying it's going to kill us all. Uh it's just a regular technology that has a lot of downsides, maybe [music] a few upsides, but that if not carefully steered in the public interest, could end up replicating and scaling a lot of flaws and [music] mistakes already in our society or governance structures. And the fact that there is so little agreement among reasonable people skilled in these arts about where things are going, what the current capabilities are, how we might in the now current parlance align how these systems work to how we want them to work, uh all of that keeps me up at night.