WEBVTT 1 00:00:02.350 --> 00:00:04.880 216-101 Large Conference: What happens if I try this? 2 00:00:06.380 --> 00:00:14.990 216-101 Large Conference: Okay, so now how do I arrange this? There's a little menu here, view… Side-by-side speaker, side-by-side gallery. 3 00:00:15.860 --> 00:00:20.010 216-101 Large Conference: Okay, sort gallery by, I'm not sure, first name, entry time. 4 00:00:20.400 --> 00:00:21.570 216-101 Large Conference: Okay, whatever. 5 00:00:22.770 --> 00:00:23.450 216-101 Large Conference: Huh. 6 00:00:24.420 --> 00:00:29.199 216-101 Large Conference: Yeah, and then, is there a thing that I can do? Show media on this device? 7 00:00:29.920 --> 00:00:31.230 216-101 Large Conference: What can I do? 8 00:00:33.830 --> 00:00:35.450 216-101 Large Conference: I wish it was a keyboard. 9 00:00:37.320 --> 00:00:43.530 216-101 Large Conference: View… Hide self-view. Hide non-video participants. 10 00:00:44.400 --> 00:00:47.689 216-101 Large Conference: What other menu options do we have? 11 00:00:52.210 --> 00:00:54.850 216-101 Large Conference: Show Meeting Controls. I'm gonna get rid of that. 12 00:00:55.550 --> 00:01:01.290 216-101 Large Conference: Setting that up, a brief question. Do you have thoughts on the co-opting of the phrase AI governance by 13 00:01:01.790 --> 00:01:02.970 216-101 Large Conference: Enterprises? 14 00:01:03.660 --> 00:01:06.299 216-101 Large Conference: I understand it all, but I… 15 00:01:08.350 --> 00:01:16.580 216-101 Large Conference: I work in industry, and all of the… all the time I'm talking to a client, and they're like, we need to institute an AI governance platform, and they don't really know what that… 16 00:01:16.900 --> 00:01:20.249 216-101 Large Conference: We don't have, like… Say that with any meaning? 17 00:01:21.020 --> 00:01:24.289 216-101 Large Conference: I, I mean… 18 00:01:25.210 --> 00:01:41.879 216-101 Large Conference: So sometimes I see governance use, and I'm like, kind of stretching it, you're kind of stretching the definition of governance, but sometimes it's, like, you know, technical people just doing an ML paper that they want to kind of shove a governance section into, and like… It doesn't really… sometimes I'm like… Oh, me? No, no, I had some a little earlier, so, yeah. 19 00:01:42.540 --> 00:01:43.810 216-101 Large Conference: Meeting this… 20 00:01:47.400 --> 00:01:49.650 216-101 Large Conference: That does make me kind of like, 21 00:01:49.850 --> 00:01:55.000 216-101 Large Conference: kind of, like, grossed out. It's not, like, what I'd call just co-option, it's just, like, Very good. 22 00:01:55.480 --> 00:02:04.020 216-101 Large Conference: companies doing a lot of their lobbying and writing their reports. They're like, this is how the government should work. And then they tell the policymakers, this is how the government should work. 23 00:02:05.070 --> 00:02:08.970 216-101 Large Conference: And usually, like, companies manage to say some useful things, but it's what they don't say. 24 00:02:09.080 --> 00:02:10.150 216-101 Large Conference: That's really… 25 00:02:11.490 --> 00:02:24.839 216-101 Large Conference: key, I think. It will be. So, like, I need to introduce myself in a minute. But, I, hot class is, like, 50% of my personality, and I always carry some of it around wherever I go. 26 00:02:27.480 --> 00:02:33.759 216-101 Large Conference: Actually, a new bottle that I'm just opening, but I was gonna pass it around so everyone could, you know, try a little bit. 27 00:02:34.340 --> 00:02:42.080 216-101 Large Conference: So I'm gonna… Cow there, which makes things kind of Frenchy. And… 28 00:02:42.430 --> 00:02:46.160 216-101 Large Conference: It's based on habaneros and Carolina Reapers. 29 00:02:46.620 --> 00:02:49.929 216-101 Large Conference: Sounds hot. I think it's a pretty spicy one if you have a lot, but if you have a few drops, it's hot. 30 00:02:50.980 --> 00:02:56.930 216-101 Large Conference: Gonna make a whole circle? Fine. 31 00:03:01.050 --> 00:03:01.810 216-101 Large Conference: Great. 32 00:03:02.520 --> 00:03:04.940 216-101 Large Conference: But wait, this isn't a really big group. 33 00:03:05.130 --> 00:03:17.430 216-101 Large Conference: This sounds… this seems pretty awesome, like, but everyone's here today. How many people here are Northeastern people versus, someone else? Northeastern here. Depends on how you're counting. Who's not Northeastern? 34 00:03:17.530 --> 00:03:28.429 216-101 Large Conference: You can triple count me when you want. I'm also at CBI. Who's some other… what, with CBI? Well, I'm both, at least, right now. I don't know. 35 00:03:28.760 --> 00:03:34.440 216-101 Large Conference: That's cool. Who's… who's whatever they… whatever they… they want to be, whenever we ask. 36 00:03:34.640 --> 00:03:38.949 216-101 Large Conference: Okay, yeah, that's fine. 37 00:03:39.220 --> 00:03:42.359 216-101 Large Conference: That's fantastic. That's okay, it's okay, pass it on, pass it on. 38 00:03:43.590 --> 00:03:46.500 216-101 Large Conference: So, I'm gonna hit the record button here. 39 00:03:46.830 --> 00:03:50.590 216-101 Large Conference: You're already recording. How do you think it's gonna get started? 40 00:03:51.100 --> 00:03:54.699 216-101 Large Conference: I guess we've been reporting this whole time, is that right? 41 00:03:56.260 --> 00:04:00.460 216-101 Large Conference: Where are we… where are we recording to you? Okay, well, that's fine. We can… 42 00:04:01.030 --> 00:04:04.040 216-101 Large Conference: We can… we can crop the recording, we can trim it. 43 00:04:04.800 --> 00:04:08.730 216-101 Large Conference: So, who here already knows who Steven Casper is? 44 00:04:09.100 --> 00:04:14.039 216-101 Large Conference: Oh, you all do. So I'll say it for the recording. So, 45 00:04:14.660 --> 00:04:19.410 216-101 Large Conference: So, we're lucky to have Steven, with us today in two ways. 46 00:04:19.600 --> 00:04:23.520 216-101 Large Conference: So, first of all, of course, we're very lucky to have him come talk to us. 47 00:04:24.060 --> 00:04:33.980 216-101 Large Conference: He's well known as, policy thought leader slash troublemaker. AI. 48 00:04:34.230 --> 00:04:44.069 216-101 Large Conference: And and has been interested in different ways, sometimes as a skeptic on interpretability issues, but very interested in what's going on in our corner of the world as well. 49 00:04:44.530 --> 00:04:50.589 216-101 Large Conference: And, but we're also lucky to have him here for a second reason. 50 00:04:50.720 --> 00:04:52.520 216-101 Large Conference: Because he's now… 51 00:04:52.700 --> 00:05:05.529 216-101 Large Conference: Professor Casper in the Boston area, where he is going to be a member of, the policy team of professors at… do I have it right? It's a Harvard, Kennedy School? Is that right? 52 00:05:06.760 --> 00:05:18.119 216-101 Large Conference: Where they're starting, an AI policy group, where Steven Spoon would be one of the main members there. So, so if you have any interest in… 53 00:05:18.350 --> 00:05:25.790 216-101 Large Conference: AI policy issues, AI safety things that are adjacent to policy, or other, other, other things about how 54 00:05:25.960 --> 00:05:40.209 216-101 Large Conference: AI is impacting the world, then you should wander over to the Kennedy School and, you know, bother Steven a little bit about it. He's probably already making trouble about the thing you're thinking about. 55 00:05:40.340 --> 00:05:42.200 216-101 Large Conference: So, 56 00:05:42.470 --> 00:05:49.899 216-101 Large Conference: So welcome, Steven, and we're really lucky to have you here. And this is, you said we have a pretty full house here, it's a nice group. 57 00:05:50.190 --> 00:06:06.900 216-101 Large Conference: It'll be a fuller house in the fall. You'll get busy in the fall, but if you have a chance to come and visit us sometime during the school year, we'd love to have you back also, and we'll probably have, like, a full seminar room and stuff like that. We can have things, or if there are other 58 00:06:07.010 --> 00:06:10.880 216-101 Large Conference: Like, interpretability-related, topics. 59 00:06:11.030 --> 00:06:29.580 216-101 Large Conference: We're always interested in having people here. Fantastic. Welcome, Steven. Thanks. And I might actually set up to the front, just so that people don't have to, like, change their eyes back and forth, but… You said kind of what I was just about to say, but I'm really happy I get to stay in academia and stay nearby, because I just finished it up at MIT, and I'll be at Harvard. 60 00:06:30.590 --> 00:06:47.209 216-101 Large Conference: And, one of the best things about that is being able to, you know, think, talk, work, share, go to meetings openly. And, I think, you know, Bau Lab things, MIT things, Harvard things, etc. I'm excited about, CBI things, I'm excited about Boston becoming 61 00:06:48.710 --> 00:06:58.040 216-101 Large Conference: lots of the kind of work that all of us are interested in, so that's really cool. And I'm hoping to get the chance to kind of talk more with more of you over time. That's one of the reasons I'm the most glad to kind of be here. 62 00:06:58.320 --> 00:07:07.090 216-101 Large Conference: Before we kind of do much or get started, can I, like, ask a bunch of questions? So, so who here, like, raise your hand if you have at least one mechanistic interpretability project? 63 00:07:07.690 --> 00:07:13.430 216-101 Large Conference: Is that most? Okay, who here… raise your hand if you have at least one sort of society or governance-focused project going on. 64 00:07:14.780 --> 00:07:22.430 216-101 Large Conference: Bad. And, raise your hand if you have… you're just kind of in and out on everything, aren't you? You're just everything, right? 65 00:07:22.580 --> 00:07:30.580 216-101 Large Conference: See by Northeastern, etc. And raise your hand if you have some sort of other project going on that's not MEC interpret and not go for Science Society. 66 00:07:31.210 --> 00:07:49.820 216-101 Large Conference: Okay, cool. More questions. Raise your hand if you'll be around Nemi, or if you'll be hanging around Nemi. I might be around for part of it, too, and I might, depending on how things go, I might have, like, a slot to speak for about 30 minutes. And I think at Nemi, I'm going to be talking about mechanistic interpretability. That is, 67 00:07:50.230 --> 00:08:02.189 216-101 Large Conference: dusting off and refining some thoughts of mine, to be able to share a useful 2026 message. But that's not what I've been planning on sharing today. What I'm planning on sharing today is kind of AI governance focused. 68 00:08:02.260 --> 00:08:19.029 216-101 Large Conference: Well, we'll talk about… the plan is at least to talk about what's going on in AI governance right now, and toward the end, talk about what machine learning nerds, you know, the ones who run machine learning experiments can do to more incisively or more targetedly have a positive influence on things that go on in government than is often done. 69 00:08:19.410 --> 00:08:22.930 216-101 Large Conference: And I think that's the plan, if that seems okay. Sound right? 70 00:08:23.940 --> 00:08:34.320 216-101 Large Conference: And I think that, based on how ML and inter-focused people are, I think the early part of the presentation, we're gonna go through quickly. 71 00:08:34.880 --> 00:08:43.489 216-101 Large Conference: And the middle part, we're gonna go through quickly as well, and then I'll try to spend a little bit of extra time on the last part, which is this, how ML research can influence things. 72 00:08:44.610 --> 00:08:47.590 216-101 Large Conference: And then, last thing, have anything else to say? 73 00:08:49.510 --> 00:09:07.540 216-101 Large Conference: No, I don't think I did. Oh, no, the timing, sorry. So, is the plan to kind of talk for about an hour or less, or then have some discussion afterward? Yeah, that usually is what we do. Okay, I think that seems pretty nice, but we can also do stop-go if there's any questions, just kind of, like, raise your hand or make sure I don't, like, fail to notice that you have a question or something. Does it seem okay? 74 00:09:08.380 --> 00:09:10.600 216-101 Large Conference: Alright, not bad. 75 00:09:11.420 --> 00:09:12.390 216-101 Large Conference: So… 76 00:09:13.170 --> 00:09:26.889 216-101 Large Conference: I wanted to start by talking about AI governance and what's going on today, with a bunch of graphs on the back of the screen. The text on them is all way too small to read, but don't worry, they're all just kind of ripped from the Our World and Data project, plus this is, like, the meter plot. 77 00:09:27.040 --> 00:09:33.380 216-101 Large Conference: And all of these graphs, the important thing to understand about them, that they actually have in common, is the x-axis is time. 78 00:09:33.550 --> 00:09:40.940 216-101 Large Conference: And the y-axis is some sort of measure of importance in the AI space. Sometimes the y-axis is, like, compute. 79 00:09:40.940 --> 00:09:54.160 216-101 Large Conference: or benchmark performance, or the amount of money being spent on something, the amount of energy being used by something, etc, etc, etc. And you can notice across all of these graphs, even with these, like, annoying text bubbles on the left side of the screen, that there's some hockey stick behavior going on. 80 00:09:54.480 --> 00:10:04.070 216-101 Large Conference: pretty reliably in AI, lots of these graphs start to experience… exhibit some sort of takeoff, that is noticeable around 2024, 2025. 81 00:10:05.180 --> 00:10:14.560 216-101 Large Conference: And that's pretty notable. It's a big, kind of, point in time in the history of AI. But we've seen this before, right? I'm sure everyone has looked at plenty of hockey stick graphs, and you've probably seen a lot of hockey stick graphs 82 00:10:14.560 --> 00:10:29.110 216-101 Large Conference: even, like, 10 years ago, when people were showing, kind of, like, performance on image classification benchmarks or stuff. So, I'm not here to say that just because there are some hockey stick graphs going on right now, that this is, like, necessarily the important time in history for AI. 83 00:10:29.110 --> 00:10:31.279 216-101 Large Conference: Although I wouldn't say it's contingently the important time. 84 00:10:32.870 --> 00:10:40.609 216-101 Large Conference: Just because something has a hockey stick shape doesn't mean it's necessarily that big of a deal, right? Like, here we have an example of a hockey stick. 85 00:10:41.170 --> 00:10:54.399 216-101 Large Conference: This thing was pretty flat for a long time, suspiciously, but then all of a sudden, like, boom, it starts shooting up. But that doesn't mean that, like, the world was about to be changed by this particular thing, at least not in political ways, because this was Mu Ding, search its popularity. 86 00:10:54.400 --> 00:11:04.610 216-101 Large Conference: But then, as we know, Mudane grew up and got less cute, and then people stopped caring about her as much. So, I'm not saying… like, lots of hockey sticks are really just the beginning of a sort of sigmoid. 87 00:11:04.790 --> 00:11:15.929 216-101 Large Conference: And this is why I'm not saying that just because we're seeing hockey sticks right now, that this is, like, the point in time for AI. But I do think this is maybe the point in time for AI governance. Again, not necessarily, but contingently. 88 00:11:16.720 --> 00:11:25.490 216-101 Large Conference: Like, remember in AI, lots of things that used to be a big deal. This is kind of a funny question to ask, it's like almost a boomer question to ask, but, like, who was doing ML research in the 2010s? 89 00:11:27.400 --> 00:11:30.420 216-101 Large Conference: Some of the, like, the older students are postdocs, I assume, right? 90 00:11:30.660 --> 00:11:41.610 216-101 Large Conference: And, you've probably seen a photo like this before. This is from the early days of the ImageNet Challenge, when people were really excited about being able to get, like, 50% top one accuracy on ImageNet. 91 00:11:41.640 --> 00:11:57.589 216-101 Large Conference: In 2018, we had this hopper. Have you seen the Deep Reinforcement Learning from Human Feedback paper? Where, you know, state-of-the-art, the really impressive things that machine learning was doing in 2018 involved, you know, training this, you know, little hopper robot in a simulated physics environment to do a backflip using 1,000 human labels. 92 00:11:57.590 --> 00:12:08.759 216-101 Large Conference: That was a big deal at the time. And then just one year later, we had GPT-2. And GPT-2 was kind of a big deal, because it had some puzzling linguistic capabilities. Here's an example of a poem that GPT-2 wrote. 93 00:12:08.760 --> 00:12:23.500 216-101 Large Conference: It reads, in the dark, the sun doth gleam, and in the dark, the moon doth seam, blah blah blah blah blah. And everyone was, in 2019 pretty freaked out about this, because, like, how did GPT-2 learn how to rhyme gleam and seem? These things are spelled differently. Like, how did it know 94 00:12:23.500 --> 00:12:33.070 216-101 Large Conference: or seemed to know that they were pronounced the same. To my knowledge, that's still kind of a mystery to this day, and it's just something that's just kind of written off as, like, you know, ML spookiness. 95 00:12:33.630 --> 00:12:35.369 216-101 Large Conference: But this is what used to be a big deal. 96 00:12:35.520 --> 00:12:47.630 216-101 Large Conference: And today, though, that 5 years later from 20… oh, sorry, 7 years later from 2019, AI can do a lot more. We're all very familiar with this. I used to have a whole spiel here where, you know, my example that I would bring up when talking about how impressive AI was today. 97 00:12:47.700 --> 00:12:54.129 216-101 Large Conference: was, to talk about how I can get, international medical and be at gold medals, right? Which is pretty cool. 98 00:12:54.130 --> 00:13:11.209 216-101 Large Conference: But I don't think that quite does it justice, right? And I think you know what does it justice, though, because AI can just kind of do whatever, right? You give it a task, as long as it's well-specified, and as long as it's within the option space of the AI system as it's configured, you know, agents can do 99% of things better than 99% of the time compared to a comparable human. 99 00:13:11.210 --> 00:13:12.930 216-101 Large Conference: At least for the right kind of task. 100 00:13:12.930 --> 00:13:21.330 216-101 Large Conference: So I think the best way of summing up what's going on in AI today is that it can just simultaneously be your professor, your coworker, and also your girlfriend, I guess. 101 00:13:23.090 --> 00:13:24.060 216-101 Large Conference: So… 102 00:13:24.300 --> 00:13:31.249 216-101 Large Conference: This is my setup for the whole AI is a rapidly emerging technology trope, but it is, and I think it's kind of worth appreciating that. 103 00:13:31.310 --> 00:13:48.859 216-101 Large Conference: And it's worth taking a historical perspective on it, because we seem to be at a very pivotal point in time for AI as a technology. Not just because things are hockey-sticking, but things definitely are hockey-sticking. There's this book from a while back called The Social Control of Technology by David Collingridge. And Collingridge introduced a… 104 00:13:48.860 --> 00:13:57.210 216-101 Large Conference: a concept, which has come to be known as Collingridge's Dilemma, or the governance or the management of emerging technologies. And I won't quote things, but I'll paraphrase things. 105 00:13:57.210 --> 00:14:08.249 216-101 Large Conference: College Dilemma says that in the early stages of governing an emerging technology, you have the power to change things, but you don't really know what you're doing. You don't understand the levers available to you yet, and how to, you know. 106 00:14:08.250 --> 00:14:19.229 216-101 Large Conference: change the trajectory in ways that induce very predictable outcomes. Later, though, once you understand things, new power structures and institutions will be too entrenched to make the trajectory of things very easy to change. 107 00:14:20.100 --> 00:14:28.829 216-101 Large Conference: So Colleges basically says that at the early stages of governing of technology, you can change things, but you don't know how. In the late stages, you have the exact kind of opposite problem. 108 00:14:29.910 --> 00:14:36.839 216-101 Large Conference: And does anyone want to guess, by the way? This isn't a load-bearing for the talk, but it's just a fun tidbit. Like, when this book was written. 109 00:14:37.820 --> 00:14:52.339 216-101 Large Conference: Oh, I was gonna ask you that. I do know the answer. You wanna guess? Is this the actual cover? This is the cover that I can find on Google. I'm just gonna, like, guess from the shape of the cover, that's the 70s. The 70s. 110 00:14:52.540 --> 00:15:01.490 216-101 Large Conference: Whoa, so, so, so good! Wait, did you know this by heart, David? No, I just looked it up on Wikipedia. 111 00:15:01.760 --> 00:15:02.690 216-101 Large Conference: So it… 112 00:15:03.190 --> 00:15:13.759 216-101 Large Conference: It says that the book was published in 1980, which would have meant that it was written in the 1970s. Yeah, you get a… I don't know, you can take an extra hit of the hot sauce. 113 00:15:13.880 --> 00:15:28.879 216-101 Large Conference: But, those are a really good guess, actually. Yeah, we didn't really need to, like, get… throw that many more guesses out there. But yeah, this is a wonderful book written about AI, evidently. You know, like, how many years ago is this? 40, 46, 40 years ago? 46 plus years ago? 114 00:15:29.710 --> 00:15:30.820 216-101 Large Conference: And… 115 00:15:31.350 --> 00:15:41.299 216-101 Large Conference: In light of Collingridge's dilemma, you know, I want to look at a few things today. One is to look at the stages of governing emerging technology. Then, what challenges arise when things are just crammed into a few years. 116 00:15:41.300 --> 00:15:57.820 216-101 Large Conference: Then how regulators are going about things, and what the Overton window is right now in AI governance. And this is actually going to be the most dense part of today's presentation. It will be kind of fast when we go through this stuff, though. And then finally, we're going to look at how things are going to blow up in the next few years, like, figuratively, probably. 117 00:15:58.080 --> 00:16:13.220 216-101 Large Conference: And what the next stages of AI politics are gonna look like, and how to prepare for that. And then finally, at the very end, very, very last thing we're going to talk about, is some approaches to research that machine learning people can use to, I think, have a pretty good hit rate in influencing 118 00:16:14.140 --> 00:16:17.130 216-101 Large Conference: what's gonna happen in the next kind of stages of governing AI? 119 00:16:17.530 --> 00:16:23.409 216-101 Large Conference: And, toward the end, we're also going to have a little bit of a conversation or a debate today about which one of these six people 120 00:16:23.470 --> 00:16:41.800 216-101 Large Conference: could be the one to take actions that could blow things up and usher in the next kind of era, or chapter or political meta. Who's in the upper right corner there? The upper right corner? That's a rando. The Guy Fawkes mask means just a random person. So all the other five… the other five are faces you know. 121 00:16:42.090 --> 00:16:48.399 216-101 Large Conference: Things could also be changed, you know, randomos can change the world sometimes, too, which we'll talk a little bit about. Sound okay? 122 00:16:48.690 --> 00:17:01.600 216-101 Large Conference: Alright, so, let's think about stages of governing emerging technology. We have this Collingridge perspective, where Collingridge says that there's this early stage where we can fix things, and there's this late stage when we don't… when it's harder to fix things. 123 00:17:01.600 --> 00:17:10.160 216-101 Large Conference: And we have this kind of a dilemma. Does anyone want to, like, throw out there some ideas critiquing this way of thinking about stages of governing a technology? 124 00:17:10.420 --> 00:17:16.460 216-101 Large Conference: Right? Like, how is the calling-rich model of this process potentially kind of, like, reductive or naive or wrong? 125 00:17:17.069 --> 00:17:24.629 216-101 Large Conference: Yeah, what do you think? We know how assumes, like, a centralized knowledge, when in reality, at the beginning, a lot of people have different opinions. 126 00:17:24.630 --> 00:17:38.669 216-101 Large Conference: Yeah, so, like, this whole WE thing, you know, who the hell is we, right? Who's the agent here? The picture from any different agent you might think of is probably very different, and it's not like, you know, the institutions making decisions are monolithic. I think that's a really good point. 127 00:17:38.930 --> 00:17:53.499 216-101 Large Conference: That was great, yeah. I was thinking if we apply this to dot-com, or how internet evolved, since this book came way before, right, 70s, so could we learn from that, like, is that how the internet evolved, like. 128 00:17:53.500 --> 00:17:59.260 216-101 Large Conference: Like, we could, but we didn't know how, and then regulations came on top of it that you have. 129 00:17:59.440 --> 00:18:09.209 216-101 Large Conference: what do you have these days? So, like, I'm just thinking about that right now, like, how things evolved from when we had internet to now, like, and applying the course in which 130 00:18:09.790 --> 00:18:26.510 216-101 Large Conference: model of this, like, do that. Yeah, and I don't know, do you think… do you think this kind of well models what happened with the internet, or social media, or data on the internet? To an extent, I think, like, yeah, I… I mean, I was pretty… you know, so somebody, I think, yeah. I agree with you there. What are you thinking? 131 00:18:27.120 --> 00:18:37.570 216-101 Large Conference: It's hard to say that you could actually fix things before they become a problem, because you don't have any, like, proof or buy-in that something will have, like, will be the thing that 132 00:18:38.130 --> 00:18:42.500 216-101 Large Conference: Becomes the most problematic aspect of a piece of technology, like… 133 00:18:42.800 --> 00:18:49.729 216-101 Large Conference: This is just saying how technology emerges. It doesn't… it isn't saying, like, this is how a technology 134 00:18:49.930 --> 00:19:03.500 216-101 Large Conference: becomes harmful, and so it's sort of conflating the two things with technology and the thing that needs being fixed. Yeah, there's this notion of fixed things, I guess they're saying, that is like, you know, it's not just a single thing to do, and it's not a thing that everybody agrees on. 135 00:19:03.740 --> 00:19:19.539 216-101 Large Conference: That seems kind of troublesome, I agree with you there. Maybe one or two more? Yeah? Sure. I mean, I guess I think the fact that there are two stages separately to begin with, there's nothing to say that they can't coexist with each other, because you could have, like, you know. 136 00:19:19.610 --> 00:19:31.210 216-101 Large Conference: tech, and then, like, a… I guess, like, a hierarchical thing of, like, there are different techs to consider in, like, one bigger entity, like, within AI, there's, like, a lot of things in it, and so, like, you may understand one, but not the other, but… 137 00:19:31.330 --> 00:19:41.040 216-101 Large Conference: Who said these stages really were discrete. They're obviously not. And who said that, like, there's just kind of one transition for, like, the technology itself, and not, like, a bunch of microscopic ones? 138 00:19:41.430 --> 00:19:49.320 216-101 Large Conference: One more, maybe? I don't fully agree with we can't fix it anymore, because if there is a problem, and it's a major one, we will definitely fix it. 139 00:19:49.700 --> 00:19:58.259 216-101 Large Conference: Yeah, the can't seems to imply that, you know, lately, this can't can probably, in practice, be overcome by enough political and institutional will. 140 00:19:58.290 --> 00:20:17.260 216-101 Large Conference: It's just… it's a matter of getting that much will, right? Dealted gasoline. What's that? Yeah, yeah, that's a good point. That's a great point. Yeah, we can just do things. So there's one more critique of Collingridge, that Collingridge's, framework that I want to… that I want to add to this heap. 141 00:20:17.660 --> 00:20:23.819 216-101 Large Conference: And… You know, Collingridge wrote about this in the 70s and released the book in 1980. However. 142 00:20:24.350 --> 00:20:38.809 216-101 Large Conference: I want this critique to come from, like, 2,000 years ago. This is Plutarch. They're wrong to think of the politics as, like, a military… like an ocean voyage or military campaign, something to be done with some particular end in view, something which leaves off as soon as that end is researched. Not a public chore to be gone over with. 143 00:20:39.020 --> 00:20:54.439 216-101 Large Conference: It's a way of life. Which I think is a really good way of thinking about things, too, because there's not really, like, an end state. You don't get done with governing some sort of technology. There's not really some sort of technology in history that poses complex problems for society where, like, governance is fixed and static and, like, at an end point. 144 00:20:54.510 --> 00:21:02.210 216-101 Large Conference: Right? Things kind of constantly evolve, because the nature of the technology is to be incorporated into an ecosystem. It's not just a very simple thing. 145 00:21:02.300 --> 00:21:21.430 216-101 Large Conference: And for this reason, there's also a lot of kind of modeling or thinking about stages of governing technology, like AI, in terms of analogy to different types of fields, like the internet and social media. I love that that was the first thing that immediately came to mind for you. We also, you know, all of us, probably all the time, periodically have discussions about 146 00:21:21.460 --> 00:21:28.279 216-101 Large Conference: comparing AI to nuclear, or airplanes, or electricity, or cell phones, or food, or automobiles, or, the internet itself. 147 00:21:29.010 --> 00:21:30.740 216-101 Large Conference: Right. And… 148 00:21:31.530 --> 00:21:36.439 216-101 Large Conference: Raise your hand if you've had a conversation comparing AI to one of these things, like, in the past week. 149 00:21:36.490 --> 00:21:49.670 216-101 Large Conference: See, I knew it, okay, yeah, yeah. And whether we think about things in terms of stages, like Collingridge did, or in terms of analogies like we can here, and we have a lot of conversations about, though, I think there is, like, one very clear thing that does seem to emerge. 150 00:21:49.700 --> 00:22:01.359 216-101 Large Conference: From these types of ways of modeling things, which is that there's always some sort of notion when we're learning how to govern emerging technology, about how we're kind of building the plane while we're flying it. 151 00:22:01.750 --> 00:22:20.740 216-101 Large Conference: Where we need to learn a lot of lessons as fast as we can on the fly as we implement solutions, but the process of learning the lessons and implementing the solutions are just not temporally, like, separated and distinct. And this kind of process and this kind of nature implies that we need to learn things the hard way, and we need to probably… well, the way these things tend to work is that 152 00:22:20.860 --> 00:22:25.620 216-101 Large Conference: Messy solutions get implemented and modified over time. 153 00:22:26.500 --> 00:22:46.609 216-101 Large Conference: So, there's this AI governance thing, and in whatever kind of scalling-rich stage we are, or in whatever, you know, stage you might compare where we are now to historical preceding with some sort of other technology, we're in a really interesting time for AI governance in 2026, because there's a lot of action, and there's a lot of uncertainty at the same time, and we're all going about this governance thing. And there are lots of motivations for doing so, too. 154 00:22:46.610 --> 00:22:50.650 216-101 Large Conference: Other than… and a lot of motivations for studying it, other than it's also just inevitable. 155 00:22:50.650 --> 00:22:55.210 216-101 Large Conference: You know, one of these motivations for governing AI might be that, like, you know, it could kill everyone, possibly. 156 00:22:55.210 --> 00:23:09.559 216-101 Large Conference: Depending on who you ask. If you ask a survey of thousands of machine learning researchers, like this one paper did, it reported that between 38 and 51% of respondents gave AI at least a 10% chance of killing everyone on Earth by doing something worse. So that might be some motivation for governing AI. 157 00:23:09.570 --> 00:23:12.920 216-101 Large Conference: Perhaps even worse, AI could hurt the economy. 158 00:23:13.110 --> 00:23:20.480 216-101 Large Conference: There's, like, now a Wikipedia page for AI Bubble, which I think is kind of, like, interesting, delightful, it's an interesting meme. 159 00:23:20.480 --> 00:23:40.300 216-101 Large Conference: And I don't know if it's a bubble, and it'll pop, and the economy will suffer, and people will lose their jobs, or if it's not a bubble and the AI can automate a bunch of labor and people will lose their jobs, but I think it's safe to expect some sort of disruption here. It's a good motivation for kind of, you know, making the onset of the technology better governed and have more, like, gradual and gentle impact on society. 160 00:23:40.620 --> 00:23:55.060 216-101 Large Conference: Another motivation for governance is distributing benefits of the technology. It's, something like about a little over a billion people in the world really are regular users of generative AI. Most people in the world just don't have the means or access or, you know, need for it. 161 00:23:55.070 --> 00:24:08.279 216-101 Large Conference: And, it's, you know, it's like the poorest 85% of the world, as you might imagine. And, you know, if AI really, truly is beneficial, then we probably want to try to actually distribute those things, and governance can be a useful tool. 162 00:24:08.310 --> 00:24:11.260 216-101 Large Conference: And there's also an accelerationist motivation for governing AI. 163 00:24:11.320 --> 00:24:18.680 216-101 Large Conference: You know, here's a graph. The words are a little bit hard to read from a distance, but maybe that's actually a good thing. But you will notice this kind of trend. 164 00:24:19.040 --> 00:24:25.239 216-101 Large Conference: Look at these lines. We have this kind of hockey stick shape in this thing that we're looking at over time. 165 00:24:25.410 --> 00:24:39.020 216-101 Large Conference: The x-axis here, it's about 1985 when something changes. So this thing, like, is taking off, it all of a sudden stops around 1985 or 1986. And for those of you who cannot read the words here, do you want to guess what this is a graph of? 166 00:24:39.860 --> 00:24:44.579 216-101 Large Conference: What type of technological adoption is being, plotted here? 167 00:24:45.940 --> 00:24:47.979 216-101 Large Conference: Remember, this line is 1986. 168 00:24:48.530 --> 00:24:50.509 216-101 Large Conference: I think it's 86, maybe 87. 169 00:24:51.290 --> 00:24:53.320 216-101 Large Conference: Yeah, oh, by using a hand. 170 00:24:53.380 --> 00:25:08.390 216-101 Large Conference: Maybe we can all see it. Can you all read it? It's, it's nuclear energy adoption. You know, for some reason, things were really, you know, taking off, accelerating a lot, and nuclear was, you know, poised to benefit a lot of people, maybe, possibly, probably. 171 00:25:08.390 --> 00:25:13.729 216-101 Large Conference: But then just something happened in 1986, and then adoption really leveled off, because there's, like, a little… 172 00:25:13.730 --> 00:25:23.569 216-101 Large Conference: Chernobyl or something that happened. So, AI governance is just gonna happen. So I want to, like, use the middle part of today's presentation to, like, go over a picture of what's going on. 173 00:25:23.570 --> 00:25:35.520 216-101 Large Conference: in the AI governance space. I want to look at a bunch of questions about, like, how governments are building capacity, how they're building understanding, how they're trying to influence supply chains, how they're trying to influence development and deployment, accountability, and how they're trying to handle those things. 174 00:25:35.730 --> 00:25:43.040 216-101 Large Conference: This is just kind of a sort of coarse way of breaking this up, but my process for designing what we're about to go through was maximally maximized. 175 00:25:43.290 --> 00:25:47.340 216-101 Large Conference: My goal was to… 176 00:25:47.630 --> 00:26:02.829 216-101 Large Conference: go to all of my notes, go to some curricula and resources, like kind of textbooks, ask some other people, and think as hard as I could about everything I could possibly include, and then include everything I could possibly include in the next kind of whirlwind tour that we're about to go into. So my goal is not 177 00:26:02.880 --> 00:26:16.429 216-101 Large Conference: to, you know, present things with a useful plot that is, you know, makes everything, like, understood all of a sudden. It's to throw a bunch of stuff up at the wall, so that you can selectively pay attention to what's relevant to you, and you can refer to these slides as notes afterward. 178 00:26:16.430 --> 00:26:24.330 216-101 Large Conference: I will have to share these slides in PDF form as notes, maybe right after this, and maybe they could be sent out over LabSlack or something? Could that work? Yeah. 179 00:26:24.380 --> 00:26:26.810 216-101 Large Conference: Okay, very nice. So… 180 00:26:28.240 --> 00:26:37.930 216-101 Large Conference: We're gonna look at a bunch of things. Oh, man, I forgot the order of my slides. Oh, damn. Okay, I was gonna do a thing where I'm like, okay, still do the thing with me. Raise your hand if you've seen at least one of the Fast and Furious ones. 181 00:26:38.460 --> 00:26:43.079 216-101 Large Conference: Raise your hand if you've seen at least 6 of them. 182 00:26:43.670 --> 00:26:55.640 216-101 Large Conference: Like, just to reiterate, we are about to go to a part of this presentation that is going to adopt the fast and furious methodology, like I was talking about. We're going to go fast, everyone's going to be furious. 183 00:26:56.420 --> 00:27:02.050 216-101 Large Conference: There will be no plot. And it might even get ludicrous toward the end. 184 00:27:02.310 --> 00:27:03.660 216-101 Large Conference: That sounds okay. 185 00:27:04.500 --> 00:27:15.720 216-101 Large Conference: Sorry I didn't, like… sorry I messed up my transitions, but we are about to be fast and furious. We're about to talk through all of these things with a bunch of examples, and we're probably going to take less than a minute per slide, if that's okay. 186 00:27:16.780 --> 00:27:23.829 216-101 Large Conference: just remember, like, selectively pay attention and note things down, and these slides will be available. But the reason we're gonna go through this is then to, like. 187 00:27:24.490 --> 00:27:27.130 216-101 Large Conference: Once we characterize what's going on in AI right now. 188 00:27:27.280 --> 00:27:31.609 216-101 Large Conference: Think about what's happening in AI next, and then think about how machine learning research can influence it. 189 00:27:33.490 --> 00:27:42.189 216-101 Large Conference: So first, we're going to talk about how governments are trying to build capacity for studying AI and informing future decision modeling. So one way that this is happening 190 00:27:42.230 --> 00:27:56.189 216-101 Large Conference: is through the creation of national AI institutes. For example, there's the UK AI Security Institute, you've probably heard about it before, right? I actually used to work here for 7 months last year in a residency, which was pretty cool. And, anyone ever is considering doing something like this, or 191 00:27:56.190 --> 00:28:13.309 216-101 Large Conference: working with some of the Mech Interp people or the other people at AC, let me know, because I can give you some thoughts or advice about working there. But AI Security Institute is kind of the world's biggest AI security institute of its kind, and it does a lot of research, it does a lot of informing the government, and as we know, it runs a lot of evaluations. 192 00:28:13.590 --> 00:28:28.420 216-101 Large Conference: The U.S. has a counterpart for this, it's USKC, as probably lots of you know. And USKC is kind of like if AC was Hoffel. If it was, like, very, very much less funded and much more bureaucratically restrained, but it's a similar type of model. 193 00:28:28.480 --> 00:28:36.760 216-101 Large Conference: for, building an AI governance institution. And there are other ACs around the world, too. You know, like in Korea, Japan. 194 00:28:37.650 --> 00:28:45.699 216-101 Large Conference: Canada, there might be one in Germany soon, etc. Australia, et cetera, et cetera. I'll point out one more, too, of particular note, which is Singapore AC. 195 00:28:46.130 --> 00:28:59.619 216-101 Large Conference: which kind of plays a unique role as being a convening point for the world. It, has a kind of unique niche in the international ecosystem as a result. But all of these ACs and the other ones that I just mentioned are examples of kind of industry partner, 196 00:28:59.620 --> 00:29:16.630 216-101 Large Conference: at governance institutes. A model of industry partner governance is kind of being forged by each of them. But that's not how it's going on exclusively in the world. We also have the EU AI Office, which is a regulator, and is forging kind of a different model for how governance can work. And then there's the case of China. 197 00:29:16.830 --> 00:29:34.569 216-101 Large Conference: Which is just kind of impossible to characterize. Like, basically every country in the world who, or every jurisdiction in the world who has tried to govern AI using an institute has done so with one institute that kind of has a clear job to do, except for kind of Canada, but definitely China. 198 00:29:34.680 --> 00:29:44.099 216-101 Large Conference: And what China's done instead is, like, have a dozen different things. Some of these organizations kind of blur the line between, like, government-sponsored think tanks and, like, government-partnered academic labs. 199 00:29:44.100 --> 00:29:57.679 216-101 Large Conference: and industry-government kind of partnerships, too. So, there's this very interesting kind of, like, nebulous governance apparatus slash cloud going on in China, and it's really complex, and other people are much better, you know, qualified to talk about the details of it than I am. 200 00:29:57.680 --> 00:30:05.779 216-101 Large Conference: But it's another really interesting kind of model of how AI governance can happen by, you know, a very complex system. So this is how governments are building capacity around the world. 201 00:30:06.010 --> 00:30:06.910 216-101 Large Conference: Seem okay? 202 00:30:07.350 --> 00:30:09.390 216-101 Large Conference: Should we do the Fast and Furious thing and switch gears? 203 00:30:09.540 --> 00:30:15.409 216-101 Large Conference: You make the sound? What's that? Will you make the sound? 204 00:30:15.570 --> 00:30:32.560 216-101 Large Conference: Next, I want to have a few slides on how governments are trying to maintain awareness and understanding of what's going on in the AI space. And a big part about this is just having transparency requirements for AI companies, and sometimes about AI systems. And there are different ways of doing this, and I'm going to pull a couple of examples from the EU AI Act codes of practice. 205 00:30:32.660 --> 00:30:39.179 216-101 Large Conference: One way of doing this is to have transparency around systems, like this little excerpt from the Codes of Practice talks about. 206 00:30:39.180 --> 00:30:58.419 216-101 Large Conference: Requiring signatories to report to the office information about their model, information that is specified below, outside of the excerpt. And you also sometimes have transparency requirements that target organizations themselves, rather than just the systems they build, like this excerpt, which talks about how signatories commit to adopting a state-of-the-art safety and security framework and providing that to the office. 207 00:30:58.420 --> 00:31:00.719 216-101 Large Conference: For the office to analyze and scrutinize. 208 00:31:00.720 --> 00:31:11.549 216-101 Large Conference: And this is just kind of very basic stuff, but it's very foundational stuff, because, you know, understanding of what's going on in the AI ecosystem is really the foundation upon which all other kind of governance ventures are really built upon. 209 00:31:12.690 --> 00:31:21.180 216-101 Large Conference: Along with this theme of… that was funny. I just have to acknowledge that. You can't ignore whatever just happened. 210 00:31:21.560 --> 00:31:33.779 216-101 Large Conference: But in addition to transparency requirements, there are also third-party evaluations and sometimes transparency around those things, which are designed to produce information that can then be scrutinized and we can learn things from. 211 00:31:33.790 --> 00:31:45.819 216-101 Large Conference: And there are many, many examples of this, because, as we probably know, third-party audits are very central to how many agendas for governing AI are constructed. But I pulled one example, a recent one. 212 00:31:46.020 --> 00:31:48.349 216-101 Large Conference: the Great American AI Act. 213 00:31:48.500 --> 00:32:07.940 216-101 Large Conference: in the U.S. Congress, which has been proposed by two representatives, Obernolti and Trahan, and it is one example of a bill that creates a responsibility for a government office to evaluate and give license to independent verification organizations, which can then do evaluations of AI companies. 214 00:32:08.360 --> 00:32:12.799 216-101 Large Conference: This is an example of trying to set up a very kind of formalized third-party evaluation ecosystem. 215 00:32:14.070 --> 00:32:27.790 216-101 Large Conference: Another approach to evidence generating is incident reporting and whistleblower protections. These things also, you know, produce information that we can action on. Here's an example of a New York bill that has incident reporting requirements. Here's an example of a California bill, which has whistleblower protection requirements. 216 00:32:28.170 --> 00:32:35.020 216-101 Large Conference: And these are also, you can think of as being pretty foundational to producing more actionable evidence in the AIC ecosystem. 217 00:32:35.330 --> 00:32:46.100 216-101 Large Conference: Also, regarding transparency and, you know, better understanding and evidence in the space, there's watermarking and content provenance. This is good for understanding the data ecosystem, and sometimes even helping you track open weight models. 218 00:32:46.170 --> 00:32:56.300 216-101 Large Conference: Very interestingly, and I don't know what this means, and I don't know if it's a coincidence or not, but the two jurisdictions in the world who have done the most, or taken the most initiative on watermarking and content provenance. 219 00:32:56.440 --> 00:32:57.889 216-101 Large Conference: Actually, anyone want to guess? 220 00:32:58.750 --> 00:33:02.249 216-101 Large Conference: Anyone know? How broad are we talking? Can I say, like, the EU? 221 00:33:02.680 --> 00:33:11.639 216-101 Large Conference: You can say the EU, but I think the EU, unless I'm missing something, it's not one of the two I'm about to flash up. China? China's one of them. 222 00:33:12.290 --> 00:33:20.620 216-101 Large Conference: And… California? California is the second one! That was really good. Dude, like, you, you deserve a hit. 223 00:33:20.870 --> 00:33:21.620 216-101 Large Conference: Excuse me. 224 00:33:22.950 --> 00:33:32.789 216-101 Large Conference: But yeah, China and California have taken a lot of initiative on, kind of, creating watermarking and content provenance requirements. 225 00:33:33.720 --> 00:33:40.910 216-101 Large Conference: So that's a quick tour of, things focused on evidence-generating policy and understanding the tech… understanding the ecosystem. 226 00:33:41.270 --> 00:33:45.470 216-101 Large Conference: Shift gears again? Things going okay? Everyone curious? 227 00:33:47.110 --> 00:33:52.749 216-101 Large Conference: Governments can also regulate supply chains. There are lots of ways of doing this. Sometimes this is through hardware. 228 00:33:52.750 --> 00:34:08.400 216-101 Large Conference: like, this policy, which was announced, on fact, by Trump Truth Social, which was pretty interesting, but where he announced to the world this new policy of the U.S. for allowing some types of H200 sales to China, and this is pretty foundational to the supply chain, obviously has lots of ripple effects. 229 00:34:08.400 --> 00:34:16.149 216-101 Large Conference: Not just hardware, though. Data is another really big input for AI systems and their lifecycle. And there aren't a lot of examples 230 00:34:16.150 --> 00:34:25.230 216-101 Large Conference: Or maybe I'm just too lazy to find good ones, of like, you know, legislation that has really clearly tried to take a heavy-handed approach at influencing AI supply chains by, 231 00:34:25.360 --> 00:34:36.470 216-101 Large Conference: Creating restrictions or requirements around data that can be used. But, copyright law has a big effect here, and tightening up copyright law made it hard to train on copyrighted content. 232 00:34:36.469 --> 00:34:51.100 216-101 Large Conference: would be one example of something that would be a very big deal. I think talent and the immigration policies and the, rule about, like, mythos not being allowed to be used by any non-US nationals, that… 233 00:34:51.530 --> 00:35:06.689 216-101 Large Conference: You slide dog, past the hot sauce. That's exactly what we're about to talk about next. Talent? I thought you were saying there wasn't one. Immigration policy is, is another kind of supply chain. It is AI policy now. What the U.S. Department of War is doing with… 234 00:35:07.350 --> 00:35:25.609 216-101 Large Conference: Designating Anthropic as supply chain risk is a big part of this. Like you just mentioned, too, government procurement is an example of this, too, because governments are big customers, and they can kind of throw their weight around and signal a lot of things by who they choose to partner with, or who they choose to purchase things from. And we can look at this announcement from the Department of War, the Department of Defense. 235 00:35:25.620 --> 00:35:30.840 216-101 Large Conference: which talks about different AI companies that it's collaborating with, and like, yes, you know, it doesn't take a lot of, 236 00:35:30.920 --> 00:35:37.099 216-101 Large Conference: discernment to be able to notice that this, interestingly, doesn't include Anthropic. 237 00:35:37.220 --> 00:35:43.990 216-101 Large Conference: It also doesn't include Meta, which I'm not sure what that really means, but maybe Meta just, you know, wasn't on it. 238 00:35:44.440 --> 00:35:50.560 216-101 Large Conference: And didn't have as much, you know, burning desire to have defense contracts as these other companies did. 239 00:35:52.600 --> 00:35:53.400 216-101 Large Conference: Got it? 240 00:35:55.210 --> 00:36:02.899 216-101 Large Conference: So that's how governments can influence supply chains, but really, like, there's so many inputs to models along the whole life cycle, so anything that affects the data, the labor. 241 00:36:03.300 --> 00:36:10.429 216-101 Large Conference: the talent, the, the hardware, etc, and can pretty… have some pretty basal influence over the AI ecosystem. 242 00:36:10.550 --> 00:36:11.480 216-101 Large Conference: Look at me. 243 00:36:11.790 --> 00:36:12.779 216-101 Large Conference: Shift gears? 244 00:36:13.310 --> 00:36:31.609 216-101 Large Conference: All right, development and deployment. Sometimes governments try to influence how these things happen. One way of doing this is just, like, requiring some types of technical protocols. An example from last week, there was a bill announced by a Texas representative, Moran, and a California representative in the U.S. House, Lou. 245 00:36:32.410 --> 00:36:52.379 216-101 Large Conference: About, called the Kill Switch Act. And this Kill Switch Act would require that companies have a protocol and have a maintained and tested ability to be able to shut down things that they were doing. So, stop inference of a color for technology, and then it says more in details about what exactly this means. But, this is an example of a government, potential government attempt. 246 00:36:52.400 --> 00:36:57.019 216-101 Large Conference: To require that companies have and follow certain protocols around certain types of actions. 247 00:36:58.400 --> 00:37:02.609 216-101 Large Conference: You also have standards around system development and risk management. 248 00:37:02.690 --> 00:37:22.039 216-101 Large Conference: Where governments will just come in and say, like, you have to do X or Y. We can turn again to the codes of practice for an example of this, which requires that companies have… signatories of the codes have state-of-the-art model evaluations in the modalities relevant to systemic risk. What does state-of-the-art model evaluations really mean? Good question. No one knows. We'll talk more about this later. Hashtag foreshadowing. 249 00:37:24.140 --> 00:37:37.519 216-101 Large Conference: In addition to just having standards around system development and risk management, you can also have system registration. China really likes this. It's a really simple thing to do, but for some reason, only China has taken, like, tons of initiative on it. Not sure what that means, but it's interesting nonetheless. 250 00:37:37.650 --> 00:37:53.860 216-101 Large Conference: Use case requirements and prohibitions. I have a few different examples of a few different things of different levels of severity you can do here, and they're all from the AI companion regulation space, because this is a convenient place to turn to. Here's an example of a U.S. bill, which is one among many bills on companions. 251 00:37:53.860 --> 00:38:00.299 216-101 Large Conference: That would require notification to users at periodic intervals that they're not interacting with a real human, edging psychosis. 252 00:38:00.340 --> 00:38:10.640 216-101 Large Conference: So this is a very soft requirement around how your model has to behave when it's being used, where… it's actually not even a model requirement, it's just a system requirement, where there needs to be a notification. 253 00:38:10.980 --> 00:38:14.189 216-101 Large Conference: You can have things that are significantly more, 254 00:38:14.670 --> 00:38:21.889 216-101 Large Conference: rigorous, though, or significantly more strenuous, like this Pennsylvania bill, which requires a model behavior. 255 00:38:22.010 --> 00:38:35.959 216-101 Large Conference: Where a companion may not claim that it is a mental health professional. And this is a really interesting Pennsylvania bill to use as an example here, because there's an active lawsuit going over it, because there was an instance in which a character AI chatbot claimed that it was a mental health professional. 256 00:38:36.090 --> 00:38:40.549 216-101 Large Conference: And, some harm ensued. I can't remember the details, but this is actively being litigated right now. 257 00:38:41.260 --> 00:39:00.100 216-101 Large Conference: And you can go even harder. Like, this Chinese regulation that the Cyberspace Administration of China was drafting. This did not get passed, but it was a draft, where it's just like, yeah, anthropomorphic interactive AI Services, aka AI companions, you can't give them to minors, you can't let children use them. 258 00:39:00.220 --> 00:39:01.869 216-101 Large Conference: This eventually got softened. 259 00:39:02.100 --> 00:39:10.360 216-101 Large Conference: And children can, in China, use AI companions. This is an example of, you know, something that is just a conditional kind of ban, or use case ban. 260 00:39:12.020 --> 00:39:29.109 216-101 Large Conference: When we're talking about bans, Article 5 of the EU AI Act Codes of Practices may be the best example of a set of bans in the world, which just bans 11 things. And they're not, like, terribly restrictive, like, these bans are, you know, usually of just the kind of things that, like, dictators or Peter Thiel would want to do with AI. 261 00:39:29.410 --> 00:39:46.220 216-101 Large Conference: you know, like, study how to manipulate people's decisions, exploit their vulnerabilities, evaluate and classify people based on social behavior, personal traits, etc, etc, etc. But this is a great example to kind of point to if you ever, you know, are looking to talk about examples of just banning things as forms of AI regulation. 262 00:39:46.540 --> 00:40:00.550 216-101 Large Conference: And then finally, there's also sometimes regulations on outcomes. Not about how your system acts or behaves, or what notifications are happening, but just the impact that your system can have on society. And one place you see a lot of this is in laws around 263 00:40:01.000 --> 00:40:16.799 216-101 Large Conference: discrimination, where there is a discrimination type of output, or a behavior, or outcome that is prohibited. Like this Colorado bill, which requires that developers use reasonably care to prevent any known or reasonably foreseeable risks of things that are legally classified as algorithmic discrimination. 264 00:40:18.500 --> 00:40:19.510 216-101 Large Conference: Sound okay? 265 00:40:19.830 --> 00:40:23.170 216-101 Large Conference: Shift gears, we're getting close to the end, and thanks for bearing with me. 266 00:40:23.290 --> 00:40:38.259 216-101 Large Conference: Even though you're all furious by now. Hopefully you're all furious. Stuff on accountability, right? This is a big topic in, like, ecosystemic hygiene. There's liability and immunity. Sometimes governments pass bills that allow companies, if they do something that harms someone. 267 00:40:38.470 --> 00:40:55.829 216-101 Large Conference: for that person to sue that company. So here's an example of a proposed bill in the federal government by Senator Josh Hawley, which would establish a federal tort related to AI companies non-consensually training on your data. So if it's not in the terms of service, or if they're just kind of scraping stuff randomly off the internet, they can't do that. 268 00:40:55.960 --> 00:41:01.969 216-101 Large Conference: And train on your stuff, and if they do, without your permission, and if they do, you can sue them and get, like, $1,500 of compensation, for instance. 269 00:41:02.260 --> 00:41:04.150 216-101 Large Conference: Sometimes… 270 00:41:04.220 --> 00:41:19.389 216-101 Large Conference: grounds to sue people are not merely created. Other times, governments can have strategies where they give attorneys general, or whatever the equivalence of an attorneys general is, the statutory sanction to bring some lawsuits against some companies on behalf of the public. 271 00:41:19.390 --> 00:41:28.509 216-101 Large Conference: So here's an example of an Arkansas law, which gives its attorney general statutory sanction to bring a public nuisance lawsuit against AI companies who develop image generation technology. 272 00:41:28.580 --> 00:41:33.990 216-101 Large Conference: If that provider or developer did not, have reasonable safeguards in place. 273 00:41:34.530 --> 00:41:39.999 216-101 Large Conference: What does reasonable safeguards mean? Nobody knows. Again, foreshadowing. We'll get to this later. 274 00:41:40.180 --> 00:41:42.980 216-101 Large Conference: And on the other side of the point of liability, there's also immunity. 275 00:41:43.050 --> 00:41:46.620 216-101 Large Conference: There was a recent bill that was kind of not passed in Illinois. 276 00:41:46.670 --> 00:41:54.800 216-101 Large Conference: Partially because it, kind of infamously traded very small transparency requirements for very broad immunity from being sued. 277 00:41:54.830 --> 00:42:07.160 216-101 Large Conference: Inside of Illinois. So if AI companies had minimal transparency, even lacking transparency about the domain in which an incident happened… an incident happened, they couldn't be sued for some of these instances… incidents as long as they weren't negligent. 278 00:42:07.440 --> 00:42:08.869 216-101 Large Conference: So you get both sides of the money here. 279 00:42:10.270 --> 00:42:11.580 216-101 Large Conference: Insurance? 280 00:42:12.040 --> 00:42:28.230 216-101 Large Conference: Insurance markets and how they're constructed do a lot to shape incentives, and I actually don't know of any sort of bill to point to here in the AI space on insurance that's very comprehensive. So instead, I'll just point to a bunch of academic papers, because something's happening in 2026, and I don't really understand what or why. 281 00:42:28.310 --> 00:42:34.219 216-101 Large Conference: But a bunch of academics, including these and some other more recent papers, have just kind of… things have popped up. 282 00:42:34.900 --> 00:42:48.680 216-101 Large Conference: about insurance, and if that's any sort of indication of what's to come, maybe this means in 2027, we'll have a lot of discussions around governing and… around structuring insurance markets around AI, including large-scale risk from AI. 283 00:42:49.160 --> 00:42:51.479 216-101 Large Conference: But I'm not sure, this is just speculation. 284 00:42:52.120 --> 00:42:58.869 216-101 Large Conference: And then the last thing in the Whirlwind tour that I want to talk about is, intervention strategies the government's out. 285 00:42:59.110 --> 00:43:08.419 216-101 Large Conference: Sometimes governments have national security and emergency powers. For example, this is what Trump used to block the release of Fable. That was just an existing power that was leveraged. 286 00:43:08.860 --> 00:43:15.870 216-101 Large Conference: There's sometimes talk about nationalizing AI systems or AI services, so the government has at least some sort of stake in controlling them. 287 00:43:15.990 --> 00:43:22.889 216-101 Large Conference: And this is now just in the Overton window, and it previously really wasn't, thanks to this Bernie bill, which would establish a sovereign wealth fund. 288 00:43:22.950 --> 00:43:39.479 216-101 Large Conference: I actually want to just take a sec to reflect on this, because I read this one. I read this one closely, and I think it's the most, like, unique and interesting AI draft law that I've read all year. And, I don't know, if you want to take a swing at it, it's like 30 pages of a lot of white space. You should totally do it. It was pretty interesting. 289 00:43:39.820 --> 00:43:42.010 216-101 Large Conference: And, this bill would do a couple things. 290 00:43:42.630 --> 00:43:45.900 216-101 Large Conference: It would create a federal commission of 7 people. 291 00:43:46.070 --> 00:43:58.070 216-101 Large Conference: to control, oversee a certain process, and the first thing that would happen is the federal government does a one-time 50% tax on all of the stock and capital of large AI companies. So the government is a 50% owner. 292 00:43:58.550 --> 00:44:03.220 216-101 Large Conference: Then it would create a sovereign wealth fund from these, 293 00:44:03.570 --> 00:44:14.400 216-101 Large Conference: these things… from what is taxed, that is able to disperse a maximum of 5% of its value to the American people every year, overseen by this seven-member commission. And the third and most important thing that it would do. 294 00:44:14.730 --> 00:44:26.359 216-101 Large Conference: The reason why this is on this slide is that this seven-member commission inside of the federal government have a very immense power to be able to appoint up to 50% board membership on all of the companies that are subject to this tax. 295 00:44:26.480 --> 00:44:29.240 216-101 Large Conference: So that would be… of the FD. 296 00:44:30.370 --> 00:44:36.789 216-101 Large Conference: And, if anyone ever wants to, like, talk more about the details of these, I have some thoughts on the specific mechanics of how it tries to implement this. 297 00:44:37.430 --> 00:44:39.740 216-101 Large Conference: Actually, some ways it could be improved, too. 298 00:44:41.030 --> 00:45:00.740 216-101 Large Conference: And, then there's societal resilience, the last main kind of category of thing I want to talk about, which is actually maybe the broadest, too, because AI can have a lot of impacts, and maintaining societal resilience is a really complicated thing. So, I'm not going to give an example here, but if you want to read more about it, I'm going to recommend you look at the last year's International AI Safety Report, and go to Section 3.5 on building societal resilience. 299 00:45:01.100 --> 00:45:04.109 216-101 Large Conference: So, this concludes our Whirlwind tour. 300 00:45:05.070 --> 00:45:17.859 216-101 Large Conference: And I want to move on to just a quick snapshot. This is another kind of, like, faux pas of a slide, because it's just a bunch of information at once. But, like, what's going… what's really hot right now in mid-2026? Because, as we know, 2026 is pretty mid. 301 00:45:18.620 --> 00:45:26.030 216-101 Large Conference: So, there's lots of discussions around the EU AI Act codes of practice, some of which actually kick in, I believe, tomorrow, August 1st, right? 302 00:45:26.200 --> 00:45:37.100 216-101 Large Conference: So that's big in the AI governance world right now. There's perennial discussions that are going on around US and China competition. The latest stage of these discussions were kicked off by Kimi K3. 303 00:45:37.900 --> 00:45:53.999 216-101 Large Conference: AI models being used in bombings is a perennial topic. It's been going on for a couple years now. Not very large-scale ones, but interesting ones nonetheless. Federal preemption of state laws is a big thing that the federal government is considering. Probably people aren't holding their breath for it to pass this year, but it's still a possibility. 304 00:45:54.830 --> 00:46:03.119 216-101 Large Conference: Transparency is always a perennial topic. The Obernolti-Trahan bills, I had one example of the earlier note that Obernolti and Trihan bit of. 305 00:46:03.150 --> 00:46:19.860 216-101 Large Conference: sorry, an early example of a bill that Obne Elsie and Trahan had proposed, but right now, California Senator Oban Elche and Massachusetts Senator Trahan have really shown, like, taken a lot of initiative within the U.S. House of Representatives to propose federal frameworks for governing AI and preempting some state activity. 306 00:46:20.320 --> 00:46:24.159 216-101 Large Conference: And, these are kind of the locus of federal discussion right now and this summer. 307 00:46:25.230 --> 00:46:34.930 216-101 Large Conference: AI developer liability for downstream harms is a perennial topic. Mental health, suicide, child safety, and non-consensual AI nudification are two big domains in which we talk a lot about liability. 308 00:46:36.160 --> 00:46:39.499 216-101 Large Conference: Data centers, everyone hates them. Well, the public hates them a lot. 309 00:46:39.740 --> 00:46:53.689 216-101 Large Conference: Sovereign Wealth Fund, which Bernie introduced to the Overton window, Trump's national security powers and what's going on between the Department of War and Anthropic. OpenAI Hacking Hugging Face. I hope we talk about this example for a long time, it's pretty interesting. And then, oh, that was it. 310 00:46:54.160 --> 00:47:01.369 216-101 Large Conference: So this is, this is my kind of, laundry list, like, unstructured, plot list laundry list of some big things that are going on in 2026 right now. 311 00:47:01.580 --> 00:47:04.640 216-101 Large Conference: And now that I've put you through all of this, and we're about to… 312 00:47:04.790 --> 00:47:12.220 216-101 Large Conference: Can I end this chapter of the presentation and move on to more interesting things. I just want to check if everyone's kind of feeling okay. 313 00:47:12.980 --> 00:47:20.049 216-101 Large Conference: And we're about to have more fun, I promise. But, I wanted to get through this kind of whirlwind discussion of just what's going on. 314 00:47:20.230 --> 00:47:21.319 216-101 Large Conference: to set up. 315 00:47:21.570 --> 00:47:23.139 216-101 Large Conference: More important stuff coming next. 316 00:47:23.350 --> 00:47:24.999 216-101 Large Conference: Okay, anyone have any questions? 317 00:47:25.870 --> 00:47:29.759 216-101 Large Conference: Are there any interesting examples of international coordination? 318 00:47:35.550 --> 00:47:40.179 216-101 Large Conference: Not really. So there are a network of AC meetings. 319 00:47:40.330 --> 00:47:42.370 216-101 Large Conference: And that happens sometimes. 320 00:47:42.630 --> 00:47:47.830 216-101 Large Conference: The trouble with the network of ACs is that no one knows what the actual… who the actual members are. 321 00:47:48.140 --> 00:47:59.649 216-101 Large Conference: Which is a little bit… you know, you'd think if a network exists, there would be the fine list of numbers, but the network of ACs is trying to, like, you know, pretend that the U.S. is kind of at the table, even though it refuses to be, so that's… 322 00:48:00.020 --> 00:48:06.290 216-101 Large Conference: Sometimes there are joint testing exercises that go on between AI safety institutes and security institutes across the world. 323 00:48:06.720 --> 00:48:11.799 216-101 Large Conference: But when it comes, like, beyond that, there's really just talk about international coordination. 324 00:48:12.050 --> 00:48:13.120 216-101 Large Conference: And… 325 00:48:13.980 --> 00:48:24.900 216-101 Large Conference: We might, though, you know, like, this is a good example, though, of something to kind of think about how it might change in the next few years, and what types of research might help us inform or understand that process as it happens. 326 00:48:25.040 --> 00:48:39.370 216-101 Large Conference: Because there's, you know, this much happening, and there's this much talk about international coordination, and if the right things happen, or I guess the wrong things happen, big incidents or things happen in the next few years in AI, this might be seen more and more as a necessity. It just might be, 327 00:48:39.570 --> 00:48:47.369 216-101 Large Conference: you know, more and more of a kick to get governments into, you know, collaborating on standards or evaluations or anything like that, so… 328 00:48:47.560 --> 00:48:54.520 216-101 Large Conference: We'll talk a little bit more later today about how ideas like this can be useful Brainstorming father. 329 00:48:55.740 --> 00:48:56.760 216-101 Large Conference: Good question. 330 00:48:57.430 --> 00:49:07.670 216-101 Large Conference: Was there… yeah. I guess similar to, like, international coordination, like, within the agencies, considering they're, like, a bunch of checkpoints. 331 00:49:07.670 --> 00:49:21.550 216-101 Large Conference: like, how does the coordination within these ones work? And also, like, I kind of wanted to know a little bit more about insurance, and like, like, how, let's say, watermarking and common problems may come… 332 00:49:21.920 --> 00:49:33.929 216-101 Large Conference: Or, like, be related to insurance, stuff like that? Oh, that's a good point. Okay, sorry. So, second question was insurance, first question was coordination among who, yes? Like, within the agency, among all of these, you know, 333 00:49:34.050 --> 00:49:44.659 216-101 Large Conference: approaches, or, like, all these, like, points to think about. Like, just within an agency, like, because, you know, for example, China has, like, multiple agencies working on… Oh, yeah, yeah. Like, within… yeah. 334 00:49:44.660 --> 00:49:59.040 216-101 Large Conference: There's just, like, one institute. Okay, I think I have, like, not too many enlightening things to say on, like, coordination problems, other than that they exist and they're very acute. Like, I had a certain perspective on this when I was doing my residency at the UKAC for 7 months. 335 00:49:59.420 --> 00:50:03.040 216-101 Large Conference: You know, UKAC is a very… 336 00:50:03.600 --> 00:50:18.700 216-101 Large Conference: interesting kind of body that is very, you know, bureaucratically suppressed, but also has a lot of creativity inside of it, and is just trying to do a lot of things at once. And, I don't know, like, central leadership at it was pretty good, but wasn't able to, you know, knock down every wall. 337 00:50:18.850 --> 00:50:20.550 216-101 Large Conference: But maybe more generally. 338 00:50:21.470 --> 00:50:27.759 216-101 Large Conference: I just think, you know, coordination is a challenge. We're gonna talk a bit in a few minutes about patchworks. 339 00:50:27.890 --> 00:50:45.679 216-101 Large Conference: As well. And, as you might get an impression from what we talked about earlier, like, there is so much patchworking within, you know, national borders and between countries in the world, there's so many different things being tried, and it's just… that's one of the reasons why the subtitle of this presentation was Why Things Are a Mess and Why They're Gonna Be A Mess Eater. 340 00:50:45.820 --> 00:51:04.319 216-101 Large Conference: On the second point on watermarking data progress, like, my main comment here is that these things are really useful for just, you know, digital forensics. When something appears where it shouldn't, or when something appears and you don't know what to trust, you know kind of where it came from, and better yet, if you could verify where it came from, kind of formal. 341 00:51:04.420 --> 00:51:12.420 216-101 Large Conference: You know, this is really, really nice. And I think the way that this would play into proceedings that happen around torts or insurance claims. 342 00:51:12.660 --> 00:51:15.250 216-101 Large Conference: Is by being able to verify, you know. 343 00:51:15.430 --> 00:51:17.390 216-101 Large Conference: Where the problematic stuff came from. 344 00:51:17.510 --> 00:51:23.040 216-101 Large Conference: Right, and then that can very straightforwardly inform some really important decisions that might need to get made. 345 00:51:23.300 --> 00:51:29.460 216-101 Large Conference: That's how I would imagine, like, the main surface between this and, you know, accountability. 346 00:51:29.620 --> 00:51:31.180 216-101 Large Conference: procedures taking place? 347 00:51:31.920 --> 00:51:39.039 216-101 Large Conference: But maybe I'm missing something. Is this closely related to anything you're working on? Yeah, but we can talk about it offline after. 348 00:51:40.240 --> 00:51:42.680 216-101 Large Conference: Should we get to the better part? 349 00:51:43.130 --> 00:51:44.030 216-101 Large Conference: Alright. 350 00:51:44.870 --> 00:51:46.970 216-101 Large Conference: Thanks for eating your veggies with me. 351 00:51:47.100 --> 00:51:47.950 216-101 Large Conference: But… 352 00:51:48.850 --> 00:52:01.940 216-101 Large Conference: there's a lot going on in AI governance today, right? There's a big, messy picture, and if you don't want to follow it all, that's fine. Don't worry about it, because things are going to change anyway, right? It's, like the weather in Colorado or whatever. And… 353 00:52:02.050 --> 00:52:08.039 216-101 Large Conference: There's this, like… Given what's going on now, I think it's worth looking back a few years. 354 00:52:08.540 --> 00:52:16.179 216-101 Large Conference: at what we used to think AI governance was going to look like, and kind of laughing at ourselves a little bit for, like, how poorly we were able to anticipate certain things. 355 00:52:16.350 --> 00:52:24.540 216-101 Large Conference: Like, I think it's a fun exercise to go, you know, read the abstracts of a few old papers from, like, 2020 and 21 on this, you know, governance of AI kind of thing. 356 00:52:24.590 --> 00:52:33.929 216-101 Large Conference: And these papers weren't done, like, they were really useful and foundational impressions at the time, at least as much as they could be, but they still kind of failed to do a thing that was basically impossible. 357 00:52:33.930 --> 00:52:51.220 216-101 Large Conference: Which was to, you know, understand what AI governance was really going to look like and how things were really going to go a few years later. So if papers from 2020 and 2021 were pretty bad at predicting, kind of, the things we'd be talking about today and what's going on today, well, are we bad today at predicting what things are going to be like 5 years from now? 358 00:52:51.220 --> 00:53:01.379 216-101 Large Conference: And the answer is almost certainly yes, right? If AI is a big deal, and it seems like it's a big deal, things are gonna change. So, how are things going to change? They could happen gradually. 359 00:53:01.870 --> 00:53:03.739 216-101 Large Conference: But I don't think they will. 360 00:53:03.930 --> 00:53:17.789 216-101 Large Conference: And I think a really important takeaway from understanding the way that AI governance has worked and proceeded, and conversations around it have evolved in the last few years, is to observe just how supremely incident-driven the discussion is. 361 00:53:17.910 --> 00:53:34.769 216-101 Large Conference: For example, last year, there were a couple of big, high-profile incidents. Like, remember last summer when the headlines came out about the Adam Raine suicide and child safety and AI significantly and psychosis? And this year, we have an avalanche of bills about AI child safety, many of which are under consideration, many of which have been passed. 362 00:53:34.880 --> 00:53:47.589 216-101 Large Conference: And then, at the end of last year and the beginning of this year, there was a period in time in which there was mass nudification on, or undressing or sexualization of images non-consensually on XAI. And nudification has been a problem for a while, but this served as a unique type of… 363 00:53:47.600 --> 00:53:58.100 216-101 Large Conference: point in time in which this really surged into public consciousness and got a lot of attention. And this year, we have seen an avalanche of not just white bills around notification, but also lawsuits. 364 00:53:58.100 --> 00:54:08.580 216-101 Large Conference: Many of whom are against XAI. I think Elon Musk is getting, like, sued from 5 different directions at once, or something like that right now, for what happened from December 29th to January 8th. 365 00:54:09.500 --> 00:54:25.419 216-101 Large Conference: And this slide is even a little bit outdated, because there's a great example from the last week, right? Remember the OpenAI hacking Hugging Face? Now it looks like Anthropic had Claude accidentally hack some companies, too. There's this whole snafu around AI systems hacking other companies. 366 00:54:25.420 --> 00:54:34.750 216-101 Large Conference: And that's a problem, and it has seized a lot of public attention. And what do you know? Within a day of this happening, some representatives 367 00:54:35.020 --> 00:54:41.410 216-101 Large Conference: Moran and Liu, whose bill I talked about a few slides ago, the Kill Switch Act, released this Kill Switch Act. 368 00:54:41.530 --> 00:54:46.039 216-101 Large Conference: Right? And… So it's an example of, like. 369 00:54:46.800 --> 00:54:57.030 216-101 Large Conference: This particular announcement was opportunistic, but it's an example of a response that came, in this case, particularly swiftly, after a big incident happened, when there was a lot of attention on an issue like this. 370 00:54:57.700 --> 00:55:08.939 216-101 Large Conference: And this is not just unique to AI, right? Crisis legislation or incident-driven initiatives in different fields are a common thing, like the Patriot Act. 371 00:55:09.110 --> 00:55:23.350 216-101 Large Conference: was previously… had parts of it rejected by Congress for years, but was kind of cobbled together and passed very quickly, like 40 days after 9-11. Or, vaccine policy around the world, and how quickly vaccines could get to market in different companies. 372 00:55:23.350 --> 00:55:32.060 216-101 Large Conference: after the COVID-19 pandemic started. This was an example of, like, maybe a more positive, example of how crisis legislation can get good things done. 373 00:55:32.930 --> 00:55:43.810 216-101 Large Conference: So, maybe we should think about how things might change in the next few years involving AI in terms of different types of incidents, or different types of attention-grabbing crises that might happen. 374 00:55:44.040 --> 00:55:47.839 216-101 Large Conference: So, I want to walk everyone through… 375 00:55:47.980 --> 00:55:59.410 216-101 Large Conference: 6 possible scenarios for big things that could happen in the next few years that could totally change the meta, or totally change the paradigm involving AI policy and governance, and make things that were previously impossible possible. 376 00:56:00.380 --> 00:56:08.119 216-101 Large Conference: And this is not an exhaustive list of 6 things, but I think… I think that I'm on… I'm onto something. I think there's a good possibility that one of these things could be a big thing. 377 00:56:08.410 --> 00:56:11.379 216-101 Large Conference: We'll have a little discussion afterward, and maybe there are some other examples, too. 378 00:56:11.870 --> 00:56:26.990 216-101 Large Conference: But one way things could change is something happens in Taiwan, right? Maybe a big earthquake, or maybe, possibly more likely, some sort of hostile action from China. And the person who kind of is at the trigger or holds the keys on this one more than anyone else would be Xi Jinping. 379 00:56:27.720 --> 00:56:33.339 216-101 Large Conference: Another way things could change is some sort of terrorism event. Probably, if it happens soon, it'll be bio, cyber, or CAM. 380 00:56:33.640 --> 00:56:47.709 216-101 Large Conference: And the person who would hold the keys here would be some rando, right? Some terrorist cell, state-sponsored, not state-sponsored, for whatever motivation or whatever reason, someone working on a chemical or bio or cyber attack, they could do a lot of damage. 381 00:56:48.330 --> 00:56:58.250 216-101 Large Conference: Another way things could happen is with a major non-terrorism incident, where an AI system is deployed and wrecks a bunch of things, or an AI system is deployed and gets… does something kind of autonomously and wrecks a bunch of things. 382 00:56:58.350 --> 00:57:13.160 216-101 Large Conference: you know, there's a lot of, outcry about this OpenAI hacking Hugging Face kind of incident, but no one got hurt, right? No grid system went down, for example, but if that kind of thing happened, but people got hurt, that's the kind of thing I am thinking of in terms of here. 383 00:57:13.970 --> 00:57:20.390 216-101 Large Conference: A bunch of different companies could cause this, but my money's on OpenAI. 384 00:57:20.460 --> 00:57:37.560 216-101 Large Conference: Relatedly, maybe there's an AI Enron. Maybe some sort of company is found to be, you know, inside of a big scandal, where they do tremendously irresponsible things at great cost, like financial or otherwise, to the public. And let's be honest, that one's probably going to be XAI if it happens. 385 00:57:38.340 --> 00:57:48.349 216-101 Large Conference: There could be an economic crisis, right? Bubble pops, economy down, people lose jobs, or bubble is not a bubble, and AI takes over jobs, people lose jobs, tech lash, that kind of thing. 386 00:57:48.400 --> 00:57:59.510 216-101 Large Conference: If I had to put a face on this, it would be Jensen Huang, who seems to be, you know, more than anyone else, the… inside of NVIDIA, the ringleader of the kind of circular financing that's going on in AI right now. 387 00:57:59.640 --> 00:58:13.589 216-101 Large Conference: And the last possible scenario, this one's not exactly a crisis, depending on who you ask, but maybe Democrats take over the federal government in 2029, at which point things that were previously politically impossible could become possible in the U.S. I'll put Bernie's face up here. 388 00:58:14.100 --> 00:58:28.600 216-101 Large Conference: with his mittens, kind of doing the Gangman-style thing, because I think Bernie Sanders, more than anyone else, has taken a role in shaping the Overton window from the left when it comes to AI and AI governance right now. 389 00:58:29.280 --> 00:58:32.710 216-101 Large Conference: So… Here's the assignment for the next 5 minutes. 390 00:58:32.830 --> 00:58:41.210 216-101 Large Conference: We're gonna discuss… don't take this too seriously, too, right? Like, this is just a frivolous… like, don't even consider it forecasting. This is frivolous, reckless speculation. 391 00:58:41.370 --> 00:58:45.620 216-101 Large Conference: But, like, maybe cluster into… self-cluster into groups of three? 392 00:58:46.200 --> 00:58:47.460 216-101 Large Conference: Or 3 or 4. 393 00:58:47.800 --> 00:58:59.560 216-101 Large Conference: And discuss a little bit about what scenario you think might be the most likely to be something that is, you know, a much bigger incident or happening than anything in AI that's happened before that could really, really, you know. 394 00:58:59.700 --> 00:59:01.660 216-101 Large Conference: Jarringly shake up the political meta. 395 00:59:01.830 --> 00:59:15.190 216-101 Large Conference: Is that okay? So just talk amongst each other about which one or two of these things you think might be the most likely to make a huge splash. And the one that might do it first. Sound okay? I'll start a 5-minute timer. 396 00:59:16.880 --> 00:59:19.170 216-101 Large Conference: They are integrated rocket. 397 00:59:19.170 --> 00:59:39.050 216-101 Large Conference: Oh, and there's a big scandal that, like… Great. 398 00:59:39.050 --> 01:00:03.649 216-101 Large Conference: And then it just, like, turned out it was… All of their, all of their everything was fake, or something like that, like, all of Enron strategy. Okay, so, like, yes, what about, what about a… 399 01:00:03.650 --> 01:00:17.699 216-101 Large Conference: AIC or AI higher… I feel like situational awareness is, like, very low-key. Yeah, super low-key. 400 01:00:17.750 --> 01:00:29.280 216-101 Large Conference: I heard you wanted to be rich, but sales. I like to be rich anyway. 401 01:00:29.280 --> 01:00:52.210 216-101 Large Conference: Have the immediate precedent for number 3. 402 01:00:52.210 --> 01:01:04.140 216-101 Large Conference: There might be a certain… 403 01:01:04.140 --> 01:01:28.400 216-101 Large Conference: Like, I think people were worried about, like, implementation, and now they're like, oh shit, this is, like, certainly one more step. 404 01:01:28.400 --> 01:01:42.160 216-101 Large Conference: Opening up. 405 01:01:42.260 --> 01:01:54.969 216-101 Large Conference: But they are dealing with more likely increases. 406 01:01:55.000 --> 01:02:18.429 216-101 Large Conference: Huggy Face, which stores multiple, multi-trillion. 407 01:02:18.430 --> 01:02:43.390 216-101 Large Conference: Because they are… they're building pretty good. 408 01:02:43.390 --> 01:03:07.610 216-101 Large Conference: Okay, Huawei, go build Indonesian, like, go build a better Indonesian engine for us, and now they're actually coming up, and so, like, without that sort of… 409 01:03:07.610 --> 01:03:32.260 216-101 Large Conference: Yeah, yeah, yeah. 410 01:03:32.380 --> 01:03:54.960 216-101 Large Conference: But it's not impossible. 411 01:03:55.020 --> 01:04:19.880 216-101 Large Conference: They're getting a lot of money. 412 01:04:19.880 --> 01:04:44.459 216-101 Large Conference: It might not be bad for, like… 413 01:04:44.460 --> 01:04:52.719 216-101 Large Conference: Clap once if you can hear me. 414 01:04:54.410 --> 01:05:04.520 216-101 Large Conference: Nice. Okay, that actually kind of worked. I might keep using that or something. So, the reason I wanted to think about this a lot, and something we're about to talk about more in the very kind of end slide. 415 01:05:04.910 --> 01:05:20.579 216-101 Large Conference: is that it's really, I think, valuable to do scenario planning like this, and to try to think of, like, what types of incidents might happen in the future, what types of ways the conversation might be shaken up, and how it relates to the work that you do. So you can do work on predicting and preventing things. 416 01:05:20.580 --> 01:05:32.820 216-101 Large Conference: that, stands a chance of either helping prevent something bad happen, which is Plan A, or, when something bad does happen, Plan B, you know, guiding us to implementing a good solution fast, as opposed to a bad solution or taking longer. 417 01:05:33.220 --> 01:05:39.230 216-101 Large Conference: So, let's, just share really quickly, and don't take this seriously at all. This is, this is… 418 01:05:39.320 --> 01:05:57.340 216-101 Large Conference: reckless speculation, this is not a serious thing, but I want everyone to kind of vote for one of these things. You know, think of the one in your head right now that you're going to vote for, that you think is the most likely to make the biggest splash first, right? So I'll call these out in order, and then everyone raise your hand once for one of them that you think, you know, that you put your tips on, or you bet might shape things up. 419 01:05:57.370 --> 01:06:00.120 216-101 Large Conference: Who thinks some sort of Taiwan scenario could do it? 420 01:06:00.430 --> 01:06:08.189 216-101 Large Conference: David's there. What's your name? Hadass. Hadeth? Okay, very nice. What about a terrorism scenario? This is probably the one I'd vote for. 421 01:06:08.690 --> 01:06:10.659 216-101 Large Conference: A major non-terrorism incident? 422 01:06:11.270 --> 01:06:14.290 216-101 Large Conference: The most hands so far, and AI Enron? 423 01:06:14.960 --> 01:06:17.810 216-101 Large Conference: No one. An economic crisis. 424 01:06:18.250 --> 01:06:20.000 216-101 Large Conference: and, Democrats. 425 01:06:20.630 --> 01:06:27.629 216-101 Large Conference: Alright, no Democrats and no Enrons, which is interesting, I don't know what that means. I have gotten some hands for those things before in the past. 426 01:06:27.860 --> 01:06:34.650 216-101 Large Conference: But, I think it's worth kind of thinking about what might happen, so that we can, you know, try to prevent or help us clean up after things in the future. 427 01:06:34.850 --> 01:06:49.600 216-101 Large Conference: So, before we go into the last few slides, I want a quick summary here about bracing for the next few years. I'll pop up the Collingridge book to do so. Because governing AI today right now, is a case study in Collingridge dilemmas, and we're kind of in the middle of some hockey sticks, or at the tip of some hockey sticks right now. 428 01:06:49.600 --> 01:07:00.619 216-101 Large Conference: Which suggests that, you know, our understanding is limited amidst ongoing entrenchment, and I think you could argue that right now is one of the most important times in AI… the history of governing AI. 429 01:07:00.630 --> 01:07:07.140 216-101 Large Conference: to generate lots of information that we can… is actionable for us. Try to, like, resolve mysteries as fast as we can at this moment. 430 01:07:07.460 --> 01:07:13.659 216-101 Large Conference: Governments are exploring very diverse approaches to governing AI, and discordant regulatory regimes are probably inevitable. 431 01:07:14.170 --> 01:07:17.770 216-101 Large Conference: And the AI politics, maybe most importantly, are incident-driven. 432 01:07:17.880 --> 01:07:23.809 216-101 Large Conference: Turbulence is coming. Lots of regulations already have been written in blood, and some will probably continue to be. 433 01:07:24.040 --> 01:07:32.489 216-101 Large Conference: And I know that's a little bit of a dark point to end on, so… here's me dang again. And then finally, 434 01:07:32.640 --> 01:07:39.179 216-101 Large Conference: now that we've thought about, like, what types of opportunities might exist to positively influence the trajectory of AI in the future. 435 01:07:39.380 --> 01:07:42.090 216-101 Large Conference: I want to just go through a few more slides. 436 01:07:42.200 --> 01:07:59.020 216-101 Large Conference: on how machine learning people… one way, one strategy by which machine learning people can try to, like, have a solid and tangible impact on things. Who here does ML? Who here has, like, coded in the last week? Yeah, good. This is for you. This is what we've been leading up to. 437 01:08:00.050 --> 01:08:12.799 216-101 Large Conference: I think that ML researchers should increasingly think of themselves as policy makers, not like in an imaginary kind of dollhouse way, but like as actual, literal makers of policy, or at least interpretations of policy. 438 01:08:13.160 --> 01:08:25.850 216-101 Large Conference: Because in this stage that we're at, in the Collingwood dilemma, and in the governance of an emerging technology, it's really, really common for lawmakers to punt problems that they don't know how to solve, or things that they don't know how to specify. 439 01:08:25.850 --> 01:08:37.440 216-101 Large Conference: to the machine learning community's evolving understanding of certain types of risks and risk management practices. And there are a ton of examples of this, and I'm just going to go through a few, but these are obviously by no means exhaustive. 440 01:08:37.580 --> 01:08:49.749 216-101 Large Conference: California, SB53 is a bill that maybe some of you have heard of or talked about or read. And the bill kind of itself hinges on a definition of catastrophic risk, which is defined in part as a risk that is perceivable. 441 01:08:49.800 --> 01:08:57.720 216-101 Large Conference: Well, what does foreseeable mean? Literally nobody knows. This doesn't have a meaning unless slash until it gets decided in the court sometime. 442 01:08:57.960 --> 01:09:01.180 216-101 Large Conference: More California bills talk about state-of-the-art techniques. 443 01:09:01.390 --> 01:09:20.209 216-101 Large Conference: Or reasonable measures, whatever those things mean. In New York, we see similar language about appropriate safeguards or reasonable efforts against certain types of harm mitigation. In Illinois, we have reference to appropriate safeguards. Colorado talks about reasonable care to protect consumers from known or reasonably foreseeable risks. Like, what does any of this mean? 444 01:09:20.210 --> 01:09:34.460 216-101 Large Conference: Here's an Arkansas bill talking about reasonable safeguards, here's a Montana bill talking about reasonable, here's a bill in China, or I'm sorry, a regulation in China talking about reasonable and effective organizational and technical measures. And kind of as the final… the final boss of it all, there's the EU. 445 01:09:34.460 --> 01:09:41.189 216-101 Large Conference: the EU AI Act codes of Practice specifically, and I don't even have an example of a quote here or an excerpt that I want to give you. This is just a count date. 446 01:09:41.250 --> 01:09:46.989 216-101 Large Conference: inside of this 30-page document, how many times each of these dead giveaway phrases appears. 447 01:09:47.050 --> 01:10:02.240 216-101 Large Conference: in which a government is saying something, intentionally, vaguely, as a way of, you know, hunting the actual meaning of these things to the machine learning community's understanding of things. If you count this up, this is over 100 mentions of these very vague terms. 448 01:10:03.330 --> 01:10:15.369 216-101 Large Conference: And, there's also tort law, too. Inside of tort law, like, tort law is not statutory, but it's the type of law that governs court cases around when, you know, one party sues another party because that second party allegedly harmed the first party. 449 01:10:15.370 --> 01:10:26.369 216-101 Large Conference: But we have notions like duty of care and proximal cause inside of tour law, which are questions that need to be answered in order to determine if one party owes another compensation for damages. 450 01:10:26.770 --> 01:10:43.590 216-101 Large Conference: These things, for hundreds of years in common law traditions, have resisted attempts to give them formal definition. And the way that the legal field just works is for these kinds of things to always be interpreted, contextually and plastically. 451 01:10:44.280 --> 01:11:01.509 216-101 Large Conference: So you probably know the point that I'm making before I kind of say it explicitly, but I want to make this point in a little bit of a roundabout way by talking about some experience of mine in the past year that has done a lot to really change how I do research and shape how I think about, like, Hatch having impact in machine learning research that I do. 452 01:11:01.580 --> 01:11:08.340 216-101 Large Conference: In places where it matters, like in the courtrooms, and in… Poly… and then, like. 453 01:11:08.470 --> 01:11:10.770 216-101 Large Conference: Policy maker or legislative chambers. 454 01:11:11.640 --> 01:11:24.100 216-101 Large Conference: So I want to talk about a paper titled, Video Deepfake Abuse, How Company Choices Predictably Shape Misuse Patterns. I'm this author, but it was led by an undergrad named Max, and there's some other very nice people that we worked on it with. And this paper, in a nutshell. 455 01:11:24.100 --> 01:11:35.959 216-101 Large Conference: is talking about, like, the choices that companies make and how this sometimes foreseeably, mitigatably leads to harm. So, we made some observations about the deepfake ecosystem, the non-consensual deepfake ecosystem specifically. 456 01:11:35.970 --> 01:11:43.020 216-101 Large Conference: About how there's this supply chain behind harms, but that model developers are a very critical part of that supply chain, and one of the narrowest parts of it. 457 01:11:44.080 --> 01:11:55.899 216-101 Large Conference: We also find that it is, in fact, a very small number of models from a very small number of model developers that are, by and large, the most popular for creating non-consensual NSFW content online. 458 01:11:56.900 --> 01:12:16.119 216-101 Large Conference: We also find that empirically, like, not just the research on the safeguards, but the empirical findings about safeguards suggest that they work, or they're at least effective in some mitigation. There are very big differences between how popular models that have safeguards versus don't have safeguards are for creating this type of content. And as a cherry on top, it's a very, like, non-transparent field. 459 01:12:16.170 --> 01:12:17.510 216-101 Large Conference: there is… 460 01:12:17.950 --> 01:12:24.130 216-101 Large Conference: Very little transparency from developers about risks at all, let alone the measures they take to mitigate some of these risks. 461 01:12:24.440 --> 01:12:28.690 216-101 Large Conference: So, we wrote this paper on these things, and we titled the paper kind of strategically. 462 01:12:29.120 --> 01:12:34.819 216-101 Large Conference: And in retrospect, I think there are some useful things about this project that, I'm trying to learn from. 463 01:12:35.010 --> 01:12:40.480 216-101 Large Conference: I've learned from, in addition to, like, different failures I've learned from in the past, there are some… a few nice things about this one that I'm trying to learn. 464 01:12:40.820 --> 01:12:42.909 216-101 Large Conference: This paper had a very concrete motivation. 465 01:12:43.030 --> 01:12:53.270 216-101 Large Conference: which was to inform future lawsuits. We wrote this in 2025 because we were predicting some lawsuits in 2026. We had a very incisive title, and this title was written for lawyers. This was written for, 466 01:12:53.270 --> 01:13:03.579 216-101 Large Conference: You know, to signal to lawyers that this is a place to look if you want evidence about, like, company choices and how, bad choices can predictably and preventably lead to people being harmed sometimes. 467 01:13:03.790 --> 01:13:12.099 216-101 Large Conference: This paper, just partially by luck, turned out to be really Prussian with respect to the 2026 zeitgeist. Like, we just got so lucky. We put this out on December 15th. 468 01:13:12.430 --> 01:13:23.040 216-101 Large Conference: And then on December 29th, that's when all the news stories and the, you know, the political fallout hit from the mass sexualization scandal that was happening on Brock, just a couple weeks later. 469 01:13:23.770 --> 01:13:40.420 216-101 Large Conference: This paper was intentionally written to be accessible to more than just machine learning people, and overall, it has come to be a useful tool that we can… have been able to use to talk to a lot of people in government and courtrooms. We've been able to talk with officials from different states and different countries. 470 01:13:40.530 --> 01:13:50.410 216-101 Large Conference: And we've also reached out to lawyers, like, incessantly, and been overly, kind of… we've worked very hard to help lawyers with things. 471 01:13:50.470 --> 01:14:06.829 216-101 Large Conference: We didn't have to do this. This was kind of, like, post-project stuff that never really, like, saw any sort of public light. But, we've done a lot of work this year to meet with lawyers, to teach them about stuff, to make slides for them, to proofread and edit over some things that they would send us sometimes, and also to… 472 01:14:06.830 --> 01:14:18.989 216-101 Large Conference: make an annotated bibliography. Like, I spent days of my life creating a 25-page annotated bibliography about everything that I think could ever be relevant to cite in some of these lawsuits, and sent it to a bunch of lawyers, and they loved it. 473 01:14:19.470 --> 01:14:21.540 216-101 Large Conference: And… I thought that was interesting. 474 01:14:22.040 --> 01:14:39.880 216-101 Large Conference: I think this was an example of a paper that was really, like, absolutely not done when it was finished, because we had so much work to do afterward. So, so here are some of my reflections. This is my thinking circa summer 2026 on, like, how to have impact sometimes in machine learning research in a way that's, like, fairly reliable when it comes to influencing what's going on in the world. 475 01:14:40.730 --> 01:14:58.540 216-101 Large Conference: One piece of advice I have is to read bills, like, lots of bills, at least bills related to your field, like via safeguards or evaluations or things like that. Lately, I read more bills than I read… oh, sorry, I read about as many bills as I read ML papers, and I think I benefit a lot more from reading the bills, and I get many more ideas from reading them as I do just ML papers. 476 01:14:59.450 --> 01:15:07.539 216-101 Large Conference: I think it's also really useful to pay attention to policy trends and debates. They're constantly changing, but understanding them now is really helpful to be able to predict what's happening in the future. 477 01:15:07.910 --> 01:15:12.810 216-101 Large Conference: Thinking a few months ahead is really important, kind of like the exercise we just did. A few months, few years, I guess. 478 01:15:13.030 --> 01:15:20.830 216-101 Large Conference: Focusing on real-world problems. Like, many of us are kind of motivated by medium or longer-term risks from AI, but the, 479 01:15:21.160 --> 01:15:35.090 216-101 Large Conference: case law and the frameworks, regulatory frameworks that we establish now for handling near-term AI problems are, in addition to just being important themselves, gonna set the stage for what we do with future AI problems, which could potentially be, like, much larger. 480 01:15:35.620 --> 01:15:51.490 216-101 Large Conference: It's important to know your audience, it's important to have clear titles that communicates well to the audience, it's useful to write to general audiences sometimes, and it's useful to do a lot of last-mile delivery work. This is what I was talking about when I talked about, you know, how much work we did to just reach out to lawyers and talk to them about this paper. 481 01:15:51.630 --> 01:15:57.910 216-101 Large Conference: I'm starting to think that if a paper is good, if a paper, you know, has the potential to really matter. 482 01:15:58.720 --> 01:16:12.280 216-101 Large Conference: then it is not crazy, and it should perhaps be expected, that you should spend just as much time communicating your work as you did actually writing and producing that work. So in, like, delivery and supply chain logistics, there's this notion of, like, last mile delivery. 483 01:16:12.350 --> 01:16:21.169 216-101 Large Conference: Which, refers to the interesting logistics problem of getting something from the last distribution center it reaches, like a package, to where it needs to go, like a doorstep. 484 01:16:21.510 --> 01:16:35.329 216-101 Large Conference: And the last mile delivery problem is, like, logistically very hard, it has a lot of ad hoc effort that goes into it, it's very expensive per mile compared to the rest of the process, and it's kind of different for every, every, every audience, or every, you know. 485 01:16:35.470 --> 01:16:36.929 216-101 Large Conference: Recipient of a delivery. 486 01:16:37.990 --> 01:16:45.660 216-101 Large Conference: But it's really, really important, because if your research just ends up on archive, and it doesn't get read by the people who need to read it, then, 487 01:16:46.320 --> 01:16:50.199 216-101 Large Conference: It's, in a very reasonable sense, you could just say, then why do the research at all? 488 01:16:51.030 --> 01:16:53.609 216-101 Large Conference: So, here's a quick little, like. 489 01:16:54.140 --> 01:17:02.139 216-101 Large Conference: recipe that I think is useful for doing ML research that stands a really good chance of having a clear impact on things that matter. 490 01:17:03.580 --> 01:17:07.730 216-101 Large Conference: They'll pick a consequential issue that policymakers are thinking a lot about right now. 491 01:17:09.210 --> 01:17:11.479 216-101 Large Conference: We've talked about a lot of them so far today. 492 01:17:11.730 --> 01:17:18.819 216-101 Large Conference: Then you pick a vague term or an open question related to it in some sort of a legislative framework, or I should also add, in some sort of, 493 01:17:19.150 --> 01:17:20.900 216-101 Large Conference: Present or future court case. 494 01:17:22.370 --> 01:17:32.249 216-101 Large Conference: Then you write some sort of paper that is designed to, as incisively as possible, as targetedly as possible, answer that question, or provide clarity where clarity could be needed. 495 01:17:32.310 --> 01:17:49.219 216-101 Large Conference: And then you set up lots of meetings, like, a lot of meetings. If you finish a paper and it's a good paper, it might be reasonable to send out, like, 50 cold emails and, you know, tap, like, 10 hot contacts that you have in order to try to, you know, set up meetings with some lawyer or some staffer or some lobbyist who might be able to, be useful. 496 01:17:50.480 --> 01:17:59.230 216-101 Large Conference: And I have one last thing to say, and this is actually, like, kind of not a good climactic ending. We're about to go out with a whipper and then just have an open discussion for a little bit. 497 01:17:59.400 --> 01:18:13.939 216-101 Large Conference: But I'll say one more thing about, like, doing this type of research, because if the idea of going out in the world and prospecting, you know, things in legislation and lawsuits to go influence with your research seems a little bit Slytherin, good. 498 01:18:14.040 --> 01:18:16.769 216-101 Large Conference: Right? I think that's the right reaction to have to this kind of stuff. 499 01:18:17.880 --> 01:18:35.380 216-101 Large Conference: And when you do stuff like this, which we could call lab coat lobbying, I think it requires a commitment to the kind of complete and objective truth, not just a convenient perspective. I think there are lots of, kind of, practical and ethical reasons why this is important, especially in the long run, and it's never really good to, like, try to only tell part of the story. 500 01:18:35.980 --> 01:18:49.120 216-101 Large Conference: But if you're still a little bit worried about lab coat lobbying and if it's a Slytherin thing to do, I still think that's the right reaction. But a few things to remember, and one is that all science is political, even when the science itself, the subject matter, isn't political, the pursuit of that is. 501 01:18:49.190 --> 01:19:03.160 216-101 Large Conference: And that's okay. We just should be honest about that. Like, most of us probably have, or probably all of us, have some goals behind the work that we want to do involving positively shaping the trajectory of AI, and that is a political agenda, and that is okay. We should just be open and honest about that. 502 01:19:03.600 --> 01:19:05.729 216-101 Large Conference: And finally, lab coat lobbying. 503 01:19:05.780 --> 01:19:21.629 216-101 Large Conference: you know, despite being sometimes a Slytherin thing to do, is so much less Slytherin than actual lobbying that happens, right? When academics who don't have any conflicts of interest, and they're sometimes, like, underpaid and overworked are doing work like this, that is something that is, you know. 504 01:19:21.630 --> 01:19:26.490 216-101 Large Conference: Usually a much better, more reliable source of information relative to the counterfactual sources of information. 505 01:19:26.530 --> 01:19:41.210 216-101 Large Conference: which are, you know, sometimes well-paid, well-streamlined industry lobbying machines. So there's a necessary role that academia needs to play in this kind of stuff. And that's one of the reasons why I'm glad that we're all in academia, or in CBI, or things kind of related to that. 506 01:19:41.310 --> 01:19:48.569 216-101 Large Conference: And, I think this is a very important place and a very important time to be doing, you know, the work that we're doing, so… 507 01:19:48.980 --> 01:19:54.090 216-101 Large Conference: That's it. I told you we were going out on a winter. But, I… 508 01:19:55.480 --> 01:20:02.169 216-101 Large Conference: I hope that, as you think about, like, if there are ways for you to take your research and, 509 01:20:03.120 --> 01:20:21.849 216-101 Large Conference: make it able to target it better at having specific changes in policy debates or courtroom debates. Because as you think about how to do that, it's enjoyable, and it is fulfilling, and it is interesting, and it is a good bridge into, you know, working on some of these problems, and that, you know, you do so in a way that really helps the conversation. 510 01:20:22.030 --> 01:20:29.459 216-101 Large Conference: Like, the commitment to the complete and objective truth. And now I think we can just kind of hang out and discuss more. So, phase one. 511 01:20:34.400 --> 01:20:49.670 216-101 Large Conference: Is this a good Q&A time, or I can sit, we can… we can… so if some people need to leave, you might leave, or we could do a small group, but maybe we can just go with the flow. Do you have a question? I got a question that's relevant to everything that you were just saying in the last 5 minutes. Why am I so excited to ask it first. 512 01:20:51.100 --> 01:20:52.760 216-101 Large Conference: I don't know much about case law. 513 01:20:54.080 --> 01:21:08.440 216-101 Large Conference: What's the ratio of, in case law, citing prior prescient work in order to evaluate something vague, versus doing an ad hoc evaluation of the thing that's right in front of you, since it's so contextual? 514 01:21:09.600 --> 01:21:11.449 216-101 Large Conference: Do you have an example in mind? 515 01:21:13.270 --> 01:21:18.790 216-101 Large Conference: like… So, this paper that you guys wrote about video deepfakes. 516 01:21:19.420 --> 01:21:21.750 216-101 Large Conference: Is it more or less… 517 01:21:22.570 --> 01:21:30.599 216-101 Large Conference: I guess, relevant to the case that you guys wrote that before all of the litigation, versus if you guys were to be hired by the prosecutors 518 01:21:31.410 --> 01:21:40.359 216-101 Large Conference: that are, like, attacking Grok to be like, hey, you write a case study on how exactly this company has done exactly this type of rock. Yeah, I see. 519 01:21:40.550 --> 01:21:50.580 216-101 Large Conference: And there's room for both, right? And there are paper examples of both that I think are… I have watched become useful. I've watched them become cited in some lawsuits in the last few years. 520 01:21:52.370 --> 01:22:00.910 216-101 Large Conference: You know, our paper was pretty broad, but it had some implicat- and it wasn't even… central focus wasn't even on closed models, it was on open models, but it had some implications here. 521 01:22:01.310 --> 01:22:11.429 216-101 Large Conference: And, it was an example of a type of paper that kind of talked about some general themes. But there was, like, a handful, like, 5 or so. I can send anyone my annotated bibliography if you want, it's open. 522 01:22:11.620 --> 01:22:16.990 216-101 Large Conference: Because I was not… we did this for free. This is our stuff that we shared. They didn't contract us for this, but… 523 01:22:17.120 --> 01:22:33.810 216-101 Large Conference: There are a few papers that kind of came out in January, February, March of this year, which did a great job of post-morteming the Grok incident, and you know, putting some academic rigor behind kind of what everyone already knows about how, yeah, there were some pretty suspect choices being made here. 524 01:22:34.240 --> 01:22:36.860 216-101 Large Conference: And I think both are really valuable, right? 525 01:22:37.290 --> 01:22:48.230 216-101 Large Conference: Sometimes, like, if you're flexible and you have the opportunity, right place, right time, and you can just do a cleanup project after a big incident, that can be sometimes really, really great, especially because, you know. 526 01:22:48.840 --> 01:22:56.220 216-101 Large Conference: Lawsuits take a long time, you know? That's an unfortunate thing, but it's also… there's a silver lining to this, because of how… 527 01:22:56.340 --> 01:23:05.440 216-101 Large Conference: You know, by the time a big lawsuit starts versus ends, you can sometimes work on, like, 3 or 4 more projects, you know, in serial, that could influence it. 528 01:23:05.590 --> 01:23:24.649 216-101 Large Conference: But, you know, there's… nothing can replace work that predicts things before it happens, because that's some of the best way to, like, you know, establish that something was indeed foreseeable, right? For example, I'm trying to work on a paper right now, predicting some of the details of the scenario that might lead up to the first big AI-enabled terrorism incident, right? 529 01:23:24.790 --> 01:23:33.130 216-101 Large Conference: this is… this is work designed to be in front of a very big incident. We don't know how it's gonna happen, but we're trying to predict the ways that it could happen, and if we're right. 530 01:23:33.350 --> 01:23:38.089 216-101 Large Conference: then… about some of the predictions that we're making, and I think we'll be right about some of them. 531 01:23:38.340 --> 01:23:47.680 216-101 Large Conference: Then it will be… That much more convincing to say after this happened that, yes, this scenario was foreseeable. 532 01:23:48.630 --> 01:23:51.079 216-101 Large Conference: Would you advocate for carpet bombing in that sense? 533 01:23:51.440 --> 01:24:03.819 216-101 Large Conference: Like, literal, or figurative? No, like, like, publishing a bunch of papers about things that could happen, so that one of them will happen. I can on my head to that. Wait, sorry, sorry, sorry, sorry, sorry, sorry. 534 01:24:03.950 --> 01:24:04.920 216-101 Large Conference: No. 535 01:24:05.430 --> 01:24:17.579 216-101 Large Conference: So, if you put out a bunch of research to effectively count PHAT, that might be a little bit, you know, in odds with the principle of kind of doing labs or lobbying in a way that's maximum. 536 01:24:17.680 --> 01:24:29.800 216-101 Large Conference: Sorry, we're just throwing predictions out there and picking what sticks. You know, seems like not the smartest thing to do in the long term. The form of carpet bombing, and the thing that I'm used to carpet bombing being referred to more. 537 01:24:29.870 --> 01:24:42.299 216-101 Large Conference: is when a bunch of people, you know, get together or, like, you know, start working on a benchmark together from a bunch of different places and perspectives, just descending on a topic, right? And the thing I would support is a lot more research right now on terrorism scenarios, for example. 538 01:24:42.420 --> 01:25:00.080 216-101 Large Conference: Yep. Yeah, that's the… that's what I… that's what I meant by carpet bombing. Not, like, the… a super slithering strategy. I'm sort of talking about the same thing, like, if we collectively think that terrorism's probably going to be the next thing, we write a bunch of papers about all of the different possible types of terrorism to occur. Yeah, that sounds good. And… 539 01:25:01.050 --> 01:25:02.440 216-101 Large Conference: Yeah, yeah, with… 540 01:25:02.600 --> 01:25:17.329 216-101 Large Conference: And there's a way of doing that. There's a way of doing exactly that, that is, like, you know, just throwing stuff out there that's, like, low quality, or that is, you know, really trying to scenario plan and, like, be thorough, right? The second version is obviously the better one, yes, yeah, but yeah. Sorry, I talked about to death, but I like your thoughts. 541 01:25:17.560 --> 01:25:18.510 216-101 Large Conference: Other thoughts? 542 01:25:19.080 --> 01:25:23.380 216-101 Large Conference: You, you talked… a lot of this was about formal governance? 543 01:25:23.500 --> 01:25:29.270 216-101 Large Conference: And I wonder, like, to what extent, you know, you can imagine something like governance through 544 01:25:29.580 --> 01:25:35.139 216-101 Large Conference: Scott Bescent meetings, or you could imagine, like, DeanVoL private governance style stuff. 545 01:25:35.480 --> 01:25:41.310 216-101 Large Conference: To what extent do you think, like, we should be focusing on formal governance? And, like, does… 546 01:25:42.300 --> 01:25:58.230 216-101 Large Conference: you know, do these two models change the kind of research that you think people should be doing? Yeah, my focus here really was on formal governance, and maybe that's a slightly unnecessary thing for some of the most important parts of this presentation, right? Like, the important parts of this presentation to machine learning people 547 01:25:58.480 --> 01:26:05.590 216-101 Large Conference: Are the parts about, like, you know, how to have useful influence, or some of the recipes you can use to try to have useful influence, and do more professionally. 548 01:26:06.890 --> 01:26:10.449 216-101 Large Conference: And, you know, your work doesn't need to. 549 01:26:11.440 --> 01:26:17.449 216-101 Large Conference: Necessarily, like, downstream have an effect on a lawsuit or an actual bill in order to be really impactful. 550 01:26:17.700 --> 01:26:25.849 216-101 Large Conference: Because, you know, as the… as I… I think the amount of vague policy-making out there in the… 551 01:26:25.970 --> 01:26:27.780 216-101 Large Conference: Space is a testament to. 552 01:26:29.120 --> 01:26:34.489 216-101 Large Conference: things are so norm-based in governance right now, right? Like, norms… everything is kind of a norm. 553 01:26:34.760 --> 01:26:45.830 216-101 Large Conference: Right? And, like, norms are even kind of put into statutory, law sometimes. Oh, sorry, I didn't know I wasn't sharing anymore. That's fine, though. When we see, you know, all of the vague things that we saw. 554 01:26:46.030 --> 01:26:56.120 216-101 Large Conference: So, I think it's probably just as legitimate, and maybe just as useful, even though I didn't emphasize it today, to go prospect norms to influence, rather than prospecting, 555 01:26:56.700 --> 01:27:03.779 216-101 Large Conference: laws and lawsuits to influence. David, we tried to do that a few years ago. Black box access is insufficient for rigorous AI audits. 556 01:27:03.890 --> 01:27:23.049 216-101 Large Conference: And that wasn't not relevant to… that project that the two of us and many others were working on, but that project is not irrelevant to lots of laws. But I think our idea back then was to influence the discussion, just the discussion itself, around auditing, to point out how there's a lot more you can do with inside the box and outside… sorry, white box and outside the box access. Right. 557 01:27:24.170 --> 01:27:25.190 216-101 Large Conference: That's one project. 558 01:27:26.110 --> 01:27:28.809 216-101 Large Conference: Thanks. What do you think? 559 01:27:29.000 --> 01:27:35.769 216-101 Large Conference: So, with respect to auditing, I had a few questions. So, like, with policy itself, so, like. 560 01:27:36.310 --> 01:27:50.889 216-101 Large Conference: For a company of, like, a third-party audit, you have these big companies, private companies, EVY, Accenture, doing audits on these companies. So, like, let's imagine a world where people have these models deployed, and you have a certain incident. 561 01:27:50.940 --> 01:28:01.019 216-101 Large Conference: The next incident. But these companies, especially like we saw with OpenAir and Anthropic, they had certain incidents. They came out and told us, yeah, we had an incident. 562 01:28:01.020 --> 01:28:13.290 216-101 Large Conference: there's no third-party audit, there's no federal government involved, or even a private audit. So, what do you think, like, the future is in this direction of how audits could and should be happening? 563 01:28:14.330 --> 01:28:15.750 216-101 Large Conference: If anything comes in, I think. 564 01:28:16.480 --> 01:28:21.260 216-101 Large Conference: Yeah, we can reflect on the past week or so, because it is… 565 01:28:21.820 --> 01:28:28.030 216-101 Large Conference: a little bit of a gray area, but I think the understanding is that under currently applicable laws. 566 01:28:28.200 --> 01:28:32.510 216-101 Large Conference: Anthropic and Open Man did not have to come out and say that what happened, happened. 567 01:28:32.690 --> 01:28:37.620 216-101 Large Conference: Right, they could have… especially the what Anthropic didn't have to, the thing, like, what they did yesterday. 568 01:28:38.210 --> 01:28:44.119 216-101 Large Conference: And from that perspective, you could view the type of transparency in reporting that is transpiring in the past few days. 569 01:28:44.310 --> 01:28:46.800 216-101 Large Conference: To be a success of, you know. 570 01:28:47.440 --> 01:28:59.729 216-101 Large Conference: Sometimes good people doing good things, or people exercising good judgment sometimes, and the success of norms, right? Like, there is a norm being forged right now when these companies are kind of taking these steps to be transparent about things. 571 01:29:00.210 --> 01:29:05.360 216-101 Large Conference: And… That's governance, right? Like, norms are a vehicle of governance. 572 01:29:05.600 --> 01:29:16.619 216-101 Large Conference: Should we trust these things forever? Probably not. I think there's a… there is a trend, or a trope in the AI space that, you know, sometimes the first time things happen, it goes very well. 573 01:29:16.870 --> 01:29:19.380 216-101 Large Conference: But subsequent times, things don't go very well. 574 01:29:19.510 --> 01:29:27.170 216-101 Large Conference: imagine, you know, Dolly 2 being a very safe model, which was followed by stable diffusion models, which were very easy to misuse. 575 01:29:27.350 --> 01:29:30.390 216-101 Large Conference: Or imagine ChatGBT being followed by Worm GBT. 576 01:29:30.520 --> 01:29:35.970 216-101 Large Conference: Or imagine, you know, these, you know, companies like OpenAI and Anthropic getting audits. 577 01:29:36.320 --> 01:29:40.880 216-101 Large Conference: Followed by other companies who don't get audits regularly, like Meta and Crock. 578 01:29:41.630 --> 01:29:43.260 216-101 Large Conference: Or at least third-party ones. 579 01:29:43.420 --> 01:29:57.730 216-101 Large Conference: And this might be another example of this, you know, like, the first time some of these incidents happen, you know, the frontier companies who are the first to run into incidents like these, you know, have a lot of scrutiny on them, have a lot of safety culture within them, and can report on stuff like this sometimes. 580 01:29:57.990 --> 01:30:03.459 216-101 Large Conference: Which is great, you know? They're bailing government out, like, electing to do things that they aren't technically required to do. 581 01:30:03.560 --> 01:30:07.769 216-101 Large Conference: But it's probably not just gonna happen. It's not always gonna happen reliably. 582 01:30:14.780 --> 01:30:16.020 216-101 Large Conference: Yeah, what do you think? 583 01:30:16.700 --> 01:30:25.030 216-101 Large Conference: I think with your paper, you're… it, like… With the, epics. 584 01:30:25.250 --> 01:30:28.480 216-101 Large Conference: You're able to use open source models to sort of show 585 01:30:28.730 --> 01:30:32.340 216-101 Large Conference: That, like, these methods improve this, like, specific result. 586 01:30:32.470 --> 01:30:37.060 216-101 Large Conference: But I think… Do you think that papers can still be impactful when they're… 587 01:30:37.240 --> 01:30:43.590 216-101 Large Conference: Suggesting potential safeguards or risk management strategies that can't necessarily be verified. 588 01:30:43.720 --> 01:30:46.830 216-101 Large Conference: Because companies use, like. 589 01:30:47.280 --> 01:30:54.529 216-101 Large Conference: Obfuscation, and they would never actually admit to doing something one way or the other, so would they actually be legally 590 01:30:54.770 --> 01:31:05.870 216-101 Large Conference: helpful, because if, like, they could have done this, and this wouldn't have happened, and that's very cheap and easy to do, where companies could 591 01:31:06.180 --> 01:31:08.279 216-101 Large Conference: And also, it's cheap and easy to do. 592 01:31:08.420 --> 01:31:16.680 216-101 Large Conference: separately, another argument of how… how is that… how did you make that claim, like, provable? As well as proving that 593 01:31:16.850 --> 01:31:19.619 216-101 Large Conference: That's something that these companies are not doing. 594 01:31:21.090 --> 01:31:39.799 216-101 Large Conference: So, yeah. We did run into some versions of these problems with our video deepfake newspaper, because, you know, developers report so little, and so little is verifiable just from model weights. We're doing a lot of breadcrumbs, and a lot of inference, right? And, you know, the… what was possible on a paper like that, you know, the 595 01:31:39.980 --> 01:31:52.250 216-101 Large Conference: the dots that were possible to connect were limited, right? The paper could have been a lot better if there was a lot more transparency reporting. And I think this is probably going to be more of a trouble for, you know, frontier stuff and closed model stuff in particular, right? 596 01:31:52.680 --> 01:31:57.679 216-101 Large Conference: And… So part of my answer, though, here is… 597 01:31:58.610 --> 01:32:06.049 216-101 Large Conference: Following the breadcrumbs as best you can and pointing out the types of trends that are suspiciously lacking that don't allow you to connect all the dots sometimes is a useful thing to do. 598 01:32:07.530 --> 01:32:08.440 216-101 Large Conference: But… 599 01:32:11.520 --> 01:32:19.709 216-101 Large Conference: There's still a fundamental problem here, where a lack of transparency, you know, does its job sometimes, one of its jobs being to deflect scrutiny. 600 01:32:19.860 --> 01:32:22.889 216-101 Large Conference: And deflect and reduce the chances of liability. 601 01:32:23.110 --> 01:32:24.170 216-101 Large Conference: And 602 01:32:26.520 --> 01:32:37.299 216-101 Large Conference: There is a… there is a role for researchers to adopt in pointing out when this is the case, and kind of, you know, singling out companies that are worse than others in cases like this sometimes. 603 01:32:37.430 --> 01:32:42.719 216-101 Large Conference: In order to lay the groundwork for, you know, either regulations that can fix this. 604 01:32:42.930 --> 01:32:46.189 216-101 Large Conference: Or, to make it clear to… 605 01:32:46.450 --> 01:32:54.750 216-101 Large Conference: Litigants, and sometimes attorneys general, what type of information they should try to, unsurface. 606 01:32:55.410 --> 01:32:56.540 216-101 Large Conference: So… 607 01:32:57.400 --> 01:33:06.540 216-101 Large Conference: One really important and really unique and sometimes understated thing in AI governance is just the power of attorneys general to get information. 608 01:33:06.540 --> 01:33:19.819 216-101 Large Conference: that is not publicly available. And, that information won't always become publicly available, but it can at least be externally scrutinized, like, within an Attorney General's office. And there are different, powers that attorneys Generals can exercise at, 609 01:33:19.950 --> 01:33:30.689 216-101 Large Conference: that have kind of different thresholds for when they can be used, right? Attorneys general can get together and send letters to companies asking for things to be made public or things to be sent to them privately all the time. And the barrier to doing that is, like, zero. 610 01:33:30.980 --> 01:33:43.810 216-101 Large Conference: There are higher barrier things where… I can't remember the term for this, but without starting a lawsuit, attorneys general can use investigative powers in order to request information be sent privately to them. 611 01:33:43.970 --> 01:33:59.080 216-101 Large Conference: And then when lawsuits are actually started, there's a higher threshold to making these happen, there's the discovery process of a lawsuit, by which lots and lots of things can be requested and subpoenaed, like mountains of documents sometimes. And again, those things don't end up being public, but they can be externally scrutinized. 612 01:33:59.260 --> 01:34:00.260 216-101 Large Conference: And… 613 01:34:00.430 --> 01:34:07.700 216-101 Large Conference: If you are finding yourself working in a domain in which you're consistently blocked by not being able to 614 01:34:08.100 --> 01:34:11.040 216-101 Large Conference: Get information that you and the world probably needs. 615 01:34:11.210 --> 01:34:27.820 216-101 Large Conference: It is not a bad idea to think about if there is, doing some research on attorneys general offices who might be interested in what you're working on, and might be interested in listening to your recommendations for what type of information they should try to get. 616 01:34:28.420 --> 01:34:31.320 216-101 Large Conference: I'm not saying this is always gonna, like, work out. 617 01:34:31.470 --> 01:34:39.780 216-101 Large Conference: But if you have interests that are aligned with an AG, they might love to talk to you as a computer scientist to, like, help refine their ideas sometimes. 618 01:34:42.230 --> 01:34:43.549 216-101 Large Conference: Yeah. 619 01:34:43.560 --> 01:35:03.489 216-101 Large Conference: Yeah, back to the topic of, like, informal governance. So I want to go back to one thing that was on your supply chain slide, just as, like, a particular… just as, like, an arbitrary example to illustrate, like, the broader thing I'm interested in asking you about, but, the export ban on Blackwells to China. I see this as, like. 620 01:35:03.760 --> 01:35:09.650 216-101 Large Conference: A particular case of, like, the phenomenon of, like, the private agenda of, like, 621 01:35:10.420 --> 01:35:13.800 216-101 Large Conference: private company, in this case NVIDIA, coming into conflict with, like. 622 01:35:14.060 --> 01:35:21.139 216-101 Large Conference: the agenda of the national interests, at least as articulated by the present administration and all of that stuff. So, 623 01:35:21.750 --> 01:35:24.020 216-101 Large Conference: There's two questions I have related to that. 624 01:35:24.240 --> 01:35:32.209 216-101 Large Conference: I don't know anything about, like… I understood, like, just like everyone did, like, the general contradiction between these two agendas, like. 625 01:35:32.430 --> 01:35:38.570 216-101 Large Conference: why did it go that way and not the other, right? Like, what sources of information and, like. 626 01:35:38.710 --> 01:35:57.949 216-101 Large Conference: like, for cases like, like the passing of bills and stuff, you're describing these mechanisms like, well, like, lawyers and lawmakers, like, consult the literature that you produce, and that, like, plays some sort of causal role in, like, the discourse and, like, downstream to, like, actual decisions getting made and bills being passed. What are the equivalent mechanisms 627 01:35:57.950 --> 01:36:06.560 216-101 Large Conference: For, like, administrative, like, for executive decision making. And that kind of leads to another point I'm kind of curious about, which is, like. 628 01:36:06.560 --> 01:36:20.399 216-101 Large Conference: Well, in the event of, like, a true, immediate crisis, like, any of those, like, cases you mentioned, not any, but, like, a lot of them, I can see just as well that, like, very consequential decisions are going to be passed down, by the federal government. 629 01:36:20.430 --> 01:36:24.589 216-101 Large Conference: Through the mechanism of, like, an executive order or something like that, which, like. 630 01:36:24.760 --> 01:36:27.350 216-101 Large Conference: again, I just don't really have an understanding of, like. 631 01:36:27.720 --> 01:36:37.570 216-101 Large Conference: These… the equivalent mechanisms to what you're describing, in terms of, like, how information gets bubbled up, and, like, incorporated into the decision-making process, and, like. 632 01:36:37.950 --> 01:36:42.339 216-101 Large Conference: what's going on there, just very broadly, I'm curious. Yeah, 633 01:36:42.530 --> 01:36:46.590 216-101 Large Conference: These two questions sound similar to me. Maybe I can answer them both at the same time? 634 01:36:46.730 --> 01:36:48.980 216-101 Large Conference: And the answer doesn't come from my experience. 635 01:36:49.100 --> 01:36:59.629 216-101 Large Conference: The answer mostly comes from what people have told me when I've asked them very similar questions about, like, you know, how to… how to find the rooms where it happens, how to know… how to figure out… 636 01:36:59.900 --> 01:37:00.910 216-101 Large Conference: who… 637 01:37:01.680 --> 01:37:06.819 216-101 Large Conference: is influential with respect to different decisions. It's, like, all sorts of different people in different positions that are at different times. 638 01:37:07.540 --> 01:37:14.870 216-101 Large Conference: But, the core thing to realize, the core thing to internalize, the core thing that people keep telling me every time I ask them something similar. 639 01:37:15.000 --> 01:37:17.270 216-101 Large Conference: Is that everything's relationship-based. 640 01:37:18.520 --> 01:37:27.930 216-101 Large Conference: Right? If you know the people who are in the room where it happens, you can talk to them. And it's just hard to know them sometimes, because, though, like, you know, the rooms where it happens, it's not public when it happens. 641 01:37:28.050 --> 01:37:39.670 216-101 Large Conference: Or, you know, the people who are kind of, like, in the informal, network behind drafting something, or creating some initiative, or informing something, or, you know, talking to executives about things, you know, are usually pretty quiet externally about what they're doing, for lots of reasons. 642 01:37:40.540 --> 01:37:45.829 216-101 Large Conference: So, one of… the things that I'm trying to get a lot better at. 643 01:37:46.540 --> 01:37:49.110 216-101 Large Conference: in the next few years, is… 644 01:37:49.940 --> 01:37:54.080 216-101 Large Conference: The version of the last mile delivery problem that involves knowing 645 01:37:54.240 --> 01:38:05.279 216-101 Large Conference: like, having the relationships to know who to talk to about specific things, right? There's no shame in sending out 50 cold emails, that's always fine. Meeting with a bunch of people, and that's probably a good way to build some networks and inroads. 646 01:38:05.560 --> 01:38:13.830 216-101 Large Conference: But… I think the only and the best way to really have a good idea of… 647 01:38:13.950 --> 01:38:22.450 216-101 Large Conference: You know, who, like, specifically the person, sometimes, or the set of people, specific named people, who are your core intended audience. Sometimes the only way of really knowing that is through. 648 01:38:22.670 --> 01:38:25.220 216-101 Large Conference: Knowing them, or knowing the right people to ask. 649 01:38:25.530 --> 01:38:31.990 216-101 Large Conference: So… If you're… If you're thinking about stuff like this, it is useful to… You know… 650 01:38:32.200 --> 01:38:34.250 216-101 Large Conference: At least build some sort of, you know. 651 01:38:34.840 --> 01:38:38.099 216-101 Large Conference: lightweight ML version, ML person version of the network. 652 01:38:38.350 --> 01:38:47.359 216-101 Large Conference: Among, you know, sometimes lawyers, or AG people, or staffers, or other people, or lobbyists who work on, things that are relevant to you. 653 01:38:50.260 --> 01:39:00.050 216-101 Large Conference: I'm trying to get better at this a lot myself, like, it's not easy. Because you're never taught this skill in machine learning, right? You're taught to write papers. You're never taught to actually make sure they matter. 654 01:39:00.440 --> 01:39:01.370 216-101 Large Conference: We're just tall. 655 01:39:01.840 --> 01:39:03.420 216-101 Large Conference: Let me grab one of that one. 656 01:39:05.350 --> 01:39:10.499 216-101 Large Conference: I think you're good at making your research matter. 657 01:39:11.620 --> 01:39:19.880 216-101 Large Conference: So, I have a question for your students. So I think that… so I like how the different students have been picking up on the formal governance question. 658 01:39:19.990 --> 01:39:28.490 216-101 Large Conference: Just a little bit of the flavor of the focus of the talk. But we're at this interesting juncture where there's a lot of debate in the last even, like, week. 659 01:39:28.780 --> 01:39:43.730 216-101 Large Conference: about the open model versus proprietary model, where the center of gravity of the technical, advancement is going to happen in our field? Is it going to continue to be driven 660 01:39:44.270 --> 01:39:49.439 216-101 Large Conference: by the closed model labs, or is it so… just from a technical point of view, I think this is interesting. 661 01:39:49.660 --> 01:39:56.270 216-101 Large Conference: To ask the question, if in the next few years, we're gonna see the open approach. 662 01:39:56.500 --> 01:40:06.629 216-101 Large Conference: with it being a technical driver, as opposed to a follower, which is… which it's been. But I think it's a reasonable question to ask. If that's one of the 663 01:40:07.250 --> 01:40:11.020 216-101 Large Conference: changes as happens if the center of gravity of innovation 664 01:40:11.420 --> 01:40:15.109 216-101 Large Conference: in AI, moves to more of an open model. 665 01:40:15.860 --> 01:40:19.389 216-101 Large Conference: Which… You know, could take a few forms. 666 01:40:19.690 --> 01:40:22.819 216-101 Large Conference: Doesn't that have implications for… 667 01:40:23.480 --> 01:40:28.410 216-101 Large Conference: like, I assume, like, the goal of impact is… 668 01:40:28.580 --> 01:40:36.559 216-101 Large Conference: Is to, think about what type of world we're building, right? And try to help 669 01:40:37.370 --> 01:40:38.950 216-101 Large Conference: Make sure it's what we want. 670 01:40:39.350 --> 01:40:40.130 216-101 Large Conference: Right? 671 01:40:40.390 --> 01:40:49.489 216-101 Large Conference: And and I think that if… when you've got the innovation happening at these very concentrated power centers, then it makes sense 672 01:40:49.740 --> 01:40:58.090 216-101 Large Conference: A lot of the measures that you're talking about, you know, making their way through bills and regulations, you know, you can hold a few companies to account. 673 01:40:58.300 --> 01:41:11.820 216-101 Large Conference: And try to shape their behavior. But if innovation is happening in a different way, if it's sort of diffuse, if it's open models, if it's just, you know, broad economic forces, it's just, like, lots of people working in the living rooms, or… 674 01:41:11.970 --> 01:41:12.990 216-101 Large Conference: Whatever. 675 01:41:13.260 --> 01:41:20.470 216-101 Large Conference: Is that… does the same… does the same approach make sense, or is there a different way that you think about 676 01:41:20.600 --> 01:41:25.680 216-101 Large Conference: You know, how to… How to help, you know, create a… 677 01:41:26.030 --> 01:41:28.489 216-101 Large Conference: A world that we want to live in. 678 01:41:29.010 --> 01:41:37.199 216-101 Large Conference: Does that make sense as a question? Yeah, I think it's a pretty important question to ask when your research is… especially when your research is more foundational. 679 01:41:37.920 --> 01:41:42.170 216-101 Large Conference: You're trying to solve more basic problems, as opposed to, like, you know, 680 01:41:42.550 --> 01:41:48.239 216-101 Large Conference: I don't know, make a lawsuit happen, or, like, give someone a tool for one specific thing to solve one specific problem, right? 681 01:41:48.350 --> 01:41:54.250 216-101 Large Conference: And, you know, this kind of describes the, you know, interpretability research and model mechanics research pretty well, I think. 682 01:41:54.750 --> 01:42:01.090 216-101 Large Conference: And I don't have very mature, you know, very well-developed thoughts around this, because it's not what I think about most of the time. 683 01:42:01.240 --> 01:42:06.700 216-101 Large Conference: I think last MecInter project I worked on was 2022? So, it's been a little bit. 684 01:42:06.890 --> 01:42:15.200 216-101 Large Conference: But… I think that… What should I say? I might say two quick things. 685 01:42:15.960 --> 01:42:17.330 216-101 Large Conference: One is that… 686 01:42:18.130 --> 01:42:28.629 216-101 Large Conference: you know, there are different styles of doing research. You could think of it as, like, a Greek hero versus a Roman Legion style, maybe. I'm kind of thinking out loud, but, like, maybe the Greek hero style of doing research is where you try to be a maverick. 687 01:42:28.840 --> 01:42:35.180 216-101 Large Conference: try to do unique things, forge your own path, you're working on something unique. You're not part of a, 688 01:42:35.900 --> 01:42:42.189 216-101 Large Conference: You don't necessarily try to be part of a bigger field, or, like, a broader effort that takes a lot of people to work on. 689 01:42:42.350 --> 01:42:50.169 216-101 Large Conference: As opposed to the Roman Legion style, where you're, you know, you have a network of collaborators where you're working to, you know, solve a… working within a paradigm or solve a problem together. 690 01:42:50.920 --> 01:42:57.360 216-101 Large Conference: And I think that if you're a Greek hero style of researcher, you need to be really careful about having good taste. 691 01:42:58.110 --> 01:43:12.189 216-101 Large Conference: And, really good at marketing your stuff industry. So, if you're a Roman Legion-style researcher, I think you need to solve a similar problem, but in a different way, and be really good at working on 692 01:43:12.270 --> 01:43:21.170 216-101 Large Conference: The frontier problems within your discipline, rather than just kind of, you know, putting things together or making new songs out of old chords. 693 01:43:21.460 --> 01:43:30.430 216-101 Large Conference: Or, building out a research field where the paradigm was established by people who were more prescient than you last year. So… 694 01:43:30.930 --> 01:43:37.769 216-101 Large Conference: when it comes to trying to, like, really build initiatives around big problems, I think that there's a type of… 695 01:43:38.790 --> 01:43:40.540 216-101 Large Conference: Maverickiness. 696 01:43:40.730 --> 01:43:46.280 216-101 Large Conference: That I think is useful. Maybe, maybe a good… a slightly spicy take on this is if you work in a field where 697 01:43:46.680 --> 01:43:57.579 216-101 Large Conference: or where there's a large group of people working on a certain paradigm. I think it's useful to have your special… your pet things, your pet peeves about that paradigm, where you think everyone else is wrong and you're the only one right. 698 01:43:57.800 --> 01:44:00.490 216-101 Large Conference: So that you, you know, you work on things and you, 699 01:44:02.360 --> 01:44:09.970 216-101 Large Conference: help the thinking evolve within that paradigm, rather than just kind of, you know, building things out. This is a creative point, just another one of these points about creative destruction. 700 01:44:10.770 --> 01:44:11.820 216-101 Large Conference: And… 701 01:44:12.090 --> 01:44:14.859 216-101 Large Conference: I think… I think that's the thought that comes to mind, but I'm not sure if it's a great… 702 01:44:14.970 --> 01:44:19.550 216-101 Large Conference: And, I don't know, can I pass it back to you or the room? Is anyone thinking of something more interesting right now? 703 01:44:27.600 --> 01:44:32.220 216-101 Large Conference: You had asked about open versus closed models. Open versus closed, yeah. 704 01:44:32.710 --> 01:44:49.280 216-101 Large Conference: What are the thoughts here? I work a lot on open model safeguards. This is the… most of the ML stuff that I do. Anyone can talk to me more about this. One take about what's going on in open models right now is that if you follow the trend lines, it looks like we might get a Claude Mithos-level open model by Christmas. 705 01:44:49.420 --> 01:44:56.000 216-101 Large Conference: And, if I want to open models like that, you know, if we have, like, a Worm GPC-style mythos-level capability model. 706 01:44:56.080 --> 01:45:14.670 216-101 Large Conference: It gets used for a lot of hacking. We're in for some incidents, and we're in for some fallout and conversations and disruptive opportunities for political entrepreneurship. And what do you think the right thing to do? So, I feel like, you know, you could say, oh, there ought to be a law, you know, we ought to pass some regulation, you know. 707 01:45:14.800 --> 01:45:22.730 216-101 Large Conference: Ukac should be on it, whatever, right? But… but if it's… if it's an open model, if it's a thing I can fit on my thumb drive here, right? 708 01:45:22.770 --> 01:45:34.939 216-101 Large Conference: you know, if it runs on my, RTX 6000, that maybe cost me, you know, $8,000 to buy, or whatever, right, but I can plug it in. 709 01:45:35.110 --> 01:45:40.280 216-101 Large Conference: Then, you know. 710 01:45:40.420 --> 01:45:47.299 216-101 Large Conference: Does, does, does pa… you know, does passing another regulation in the SEC saying that in your quarterly reports you need to… 711 01:45:47.420 --> 01:46:00.640 216-101 Large Conference: disclose your risk register or something like that? Does that make any difference to the 16-year-old who just downloaded the thing on their thumb drive? I don't… like, I'm not… does that make any sense? Like, I feel like there's… there's this, like, what's… what's the… like, how do we shape 712 01:46:01.250 --> 01:46:04.470 216-101 Large Conference: How do we shape behavior, and how do we shape the future? 713 01:46:04.610 --> 01:46:11.370 216-101 Large Conference: If it's diffuse, is there a different model that we should be thinking about? Personal opinion of mine, quick note, is that, like. 714 01:46:11.370 --> 01:46:31.149 216-101 Large Conference: you know, I've tried to be adamantly neutral on open models, like, I don't really care about openness and transitory, or no, and I mean, it… I don't know, I find myself pretty… feeling kind of pro-open. No, no, I think open is great. I'm just… I'm not asking a value judgment of open versus closed. Let's say it happens. 715 01:46:31.150 --> 01:46:39.870 216-101 Large Conference: Because open models, like, I find it hard to imagine a positive future without them. And also, they're kind of just inevitable. Even if a government tries to ban them… That's right. Probably not going to work, probably going to go back. 716 01:46:40.510 --> 01:46:55.810 216-101 Large Conference: So, the best question to ask is about how we can kind of make the ecosystem safer. On the technical side, I think there are some big and neglected problems that are still left to be solved, involving improving tempo resistance, making it go from this difficult and expensive to this difficult and expensive. 717 01:46:55.950 --> 01:47:01.790 216-101 Large Conference: To maliciously fine-tune an open model to be a harmful asset for something like biohazards or cyber offense. 718 01:47:02.150 --> 01:47:09.190 216-101 Large Conference: And once we solve that, and kind of we have a good toolbox, and that toolbox is, you know, something who's… 719 01:47:09.870 --> 01:47:21.700 216-101 Large Conference: you know, we're ignoring that toolbox as a faux pas in the space, and you can be sued for ignoring best practices. Once we're at that point, then I think the game with open models just turns into a never-ending problem of, like, maintaining ecosystemic hygiene. 720 01:47:21.790 --> 01:47:32.059 216-101 Large Conference: I'm playing whack-a-mole forever. Like, I'm a whack-a-mole apologist. We usually talk about whack-a-mole as being a bad thing to do, but it's kind of what we need to do. It's how society solves lots of problems like CSAM on the internet, or crime, right? 721 01:47:32.210 --> 01:47:36.370 216-101 Large Conference: And, there are gonna be harmful open models that get circulated in the world. 722 01:47:36.530 --> 01:47:43.160 216-101 Large Conference: But the ecosystem is not only as safe as its least safe models. A bunch of things involving access and ease of use matter here. 723 01:47:43.290 --> 01:47:48.090 216-101 Large Conference: So, being prepared to forever, you know, 724 01:47:48.650 --> 01:47:58.960 216-101 Large Conference: have partnerships between, like, online platforms where things circulate and law enforcement and stuff to, take down bad open models online, seems kind of useful. 725 01:47:58.960 --> 01:48:15.040 216-101 Large Conference: We're really not as good at this as we kind of honestly should be right now, because you can still go to, like, Hugging Face or Civitai, and you can find tens of thousands of models that are… have names that are suspiciously indicative of possibly specializing in creating NSFW content of young people. 726 01:48:16.040 --> 01:48:19.220 216-101 Large Conference: You know, like, you can find thousands of models online. 727 01:48:19.590 --> 01:48:23.859 216-101 Large Conference: that are designed to create porn that have teen inside the name. That's kind of suspicious, right? 728 01:48:24.160 --> 01:48:32.639 216-101 Large Conference: That's one thing that we talked about in the video, Deepak in Newspaper. But that's a kind of problem that I think we should figure out how to, you know, have habits. 729 01:48:32.930 --> 01:48:35.110 216-101 Large Conference: for mitigating. 730 01:48:35.320 --> 01:48:44.920 216-101 Large Conference: And, you know, bad stuff is going to get out. Open models are going to get creative, people are going to use them for bad things. And we should prepare to make that harder for them and hold them more accountable forever. 731 01:48:45.680 --> 01:49:05.170 216-101 Large Conference: I'll interject one more thing, a little story. So, Steven and I spent a day on Capitol Hill. Was it a year ago, a year and a half ago, something like that? It was, January 2025. Right, yeah. It was when… it was… wasn't it right after DeepSeek? It was right after DeepSeek. And I thought, oh, shoot, we should, talk a little bit about that. 732 01:49:05.260 --> 01:49:15.969 216-101 Large Conference: But I was… I was actually… it left an impression on me, this… this… this Capitol Hill visit. I thought it was really interesting. I was actually pretty impressed by the Capitol Hill staffers. 733 01:49:16.140 --> 01:49:24.400 216-101 Large Conference: You know, you think, oh, everybody in DC is such an idiot or whatever, right? But actually, you know, there's a lot of people there. 734 01:49:25.060 --> 01:49:36.160 216-101 Large Conference: you know, I got the impression that, you know, it's true, there's a fraction of them that are idiots, right? But there's… there's people who are really thinking hard about these things, and I was… I was struck by two 735 01:49:36.250 --> 01:49:46.020 216-101 Large Conference: Once, and one of them is a topic you just mentioned. You know, I forget which staffer it was. It was an older guy who said, you know, don't you think that this whole concern about 736 01:49:46.170 --> 01:49:59.240 216-101 Large Conference: regulating OpenAI or whatever, it's gonna be obsolete, because it's all gonna be OpenAI… it's gonna be, like, you know, AI is in a box, it's gonna be all these open models, right? And that's really what we're gonna deal with in a few years, and that's going to be the regulation. 737 01:49:59.330 --> 01:50:12.779 216-101 Large Conference: problem. So I forget, do you remember which staffer it was who… who told us that that was his concern? I don't remember the name, but… Yeah, but you remember that conversation, right? Yeah. Right? 738 01:50:12.990 --> 01:50:18.760 216-101 Large Conference: And then… and then the, and then the other person who left an impression on me 739 01:50:18.980 --> 01:50:21.450 216-101 Large Conference: Was, oh, what's his name? 740 01:50:21.770 --> 01:50:39.799 216-101 Large Conference: We actually… the legislature, we… Raskin? Yeah, Raskin. And when we talked to Jeremy Raskin, and he had this one question, and I don't know why he was obsessed by this question, but I, you know, I wonder what you think the policy implication of it, and why Raskin was so… 741 01:50:40.010 --> 01:50:41.350 216-101 Large Conference: Interested in this. 742 01:50:41.530 --> 01:50:44.819 216-101 Large Conference: He said, Are these things conscious? 743 01:50:46.560 --> 01:50:51.570 216-101 Large Conference: And he really wanted us to, you know, get into it and talk about it with him. And of course, you know, as scientists, we're like. 744 01:50:51.720 --> 01:50:59.720 216-101 Large Conference: You know, this is an open research question, right? But… but this is… this was… this is, like, the thing that he wanted. 745 01:50:59.930 --> 01:51:01.160 216-101 Large Conference: That's it. 746 01:51:01.900 --> 01:51:10.570 216-101 Large Conference: give an honest, face-to-face opinion about, you knew that people wouldn't write papers about it. It's like, just tell me, right? You know, tell me for real. 747 01:51:10.880 --> 01:51:15.399 216-101 Large Conference: And so I wonder what your perspective… like, why was that Raskin's first question? 748 01:51:20.670 --> 01:51:25.330 216-101 Large Conference: It's gotta just be curiosity. I think it's just curiosity, right? Because… 749 01:51:25.500 --> 01:51:36.029 216-101 Large Conference: And there are some bills out there, like, mostly at state levels, where, you know, some legislatures have taken action on this, saying that, like, yeah, AI systems can never have rights within our borders, ever. 750 01:51:36.690 --> 01:51:40.740 216-101 Large Conference: So maybe there's some legislative relevance there, but I think, I think it's a… 751 01:51:41.010 --> 01:51:44.840 216-101 Large Conference: an immense amount of curiosity, and I think this speaks to your point about, like, how 752 01:51:45.060 --> 01:51:50.459 216-101 Large Conference: You're impressed with how, with it, some of these staffers have been. 753 01:51:51.030 --> 01:52:03.860 216-101 Large Conference: Yeah, just for context, is this Jamie Raskin? Yes. Yeah, yeah, yeah. He has also introduced animal welfare legislation. That's what I was saying. Okay, model welfare, animal welfare. It's worrying about, like, accidentally doing slavery. 754 01:52:04.130 --> 01:52:05.060 216-101 Large Conference: Yeah. 755 01:52:06.410 --> 01:52:08.439 216-101 Large Conference: Interesting, right? I just learned something. 756 01:52:09.950 --> 01:52:18.890 216-101 Large Conference: Yeah, just actually to put another note on David's question, I think one of the things I've been thinking about recently is, like. 757 01:52:19.040 --> 01:52:24.119 216-101 Large Conference: state capacities versus, sort of, diffuse capacities, so we have, like. 758 01:52:24.460 --> 01:52:30.550 216-101 Large Conference: Nuclear arms, which are sort of very different from small firearms in the way we, like, have to 759 01:52:30.860 --> 01:52:36.749 216-101 Large Conference: Navigate those, because one of those can proliferate through, like, a relatively small supply chain. 760 01:52:37.480 --> 01:52:43.290 216-101 Large Conference: You know, firearms are fairly easy to procure and trade, and, like, whereas, like, nuclear arms, not so much. 761 01:52:43.460 --> 01:52:48.960 216-101 Large Conference: And I also think, like, there's a similar situation going on with, sort of, illicit drugs versus, like. 762 01:52:49.370 --> 01:52:55.370 216-101 Large Conference: You know, alcohol and alcohol prohibition back in the day, like Bonsai being, you know… Very… 763 01:52:56.490 --> 01:53:07.179 216-101 Large Conference: highly proliferating technology, like, how to make alcohol. Anyway, I don't have another thought. That was just, like, analogies. Analogies to the opening, yeah. 764 01:53:08.630 --> 01:53:11.449 216-101 Large Conference: I kind of like the prohibition analogy, I might use that sometimes. 765 01:53:12.720 --> 01:53:32.019 216-101 Large Conference: Distillation at home is still legal. That's why they call it distillation. Such a good paper title. You can't stop distillation. What AI regulation can learn from prohibition. 766 01:53:32.240 --> 01:53:42.579 216-101 Large Conference: I've seen… I've seen the… I've seen the jokes on social media, but I think you're actually serious. I think you're right. There's only, like, 30% jokes, I think. 767 01:53:42.700 --> 01:53:49.859 216-101 Large Conference: Steven, do you have time to hang out with our group this afternoon a little bit, or are you headed off? You have to be back at Harvard in, like, an hour and a half. 768 01:53:49.960 --> 01:53:52.460 216-101 Large Conference: Okay, that's my, that's my move. So, I've got time. 769 01:53:52.600 --> 01:54:12.479 216-101 Large Conference: Okay, so let's… let's, let's wrap up, and, you know, maybe if… Steven, if you have a chance to come see our space, and maybe just chat informally in the hallway for a little bit before you head off. That sounds great. We can… it can be, like, an office half hour. Yes, something like that. Office, office, office 45 minutes. Thanks so much for letting me come over. 770 01:54:14.860 --> 01:54:19.050 216-101 Large Conference: I am so glad to be around, and I'm so glad for synergies.