WEBVTT 1 00:00:13.250 --> 00:00:14.420 Emma Bortz: Hello! 2 00:00:15.500 --> 00:00:15.940 Zikai Wang: Huh. 3 00:00:32.049 --> 00:00:32.680 Emma Bortz: Hi! 4 00:00:32.689 --> 00:00:33.449 Arjun Khurana: Bull? 5 00:00:33.960 --> 00:00:35.060 Emma Bortz: How's it going? 6 00:00:35.380 --> 00:00:37.010 Arjun Khurana: Going alright, how you doing? 7 00:00:37.150 --> 00:00:39.529 Emma Bortz: Good! Very excited for this. 8 00:00:39.860 --> 00:00:40.860 Arjun Khurana: Me too. 9 00:00:42.220 --> 00:00:44.319 Arjun Khurana: I think this is gonna be a fun one. This was a… 10 00:00:44.560 --> 00:00:46.559 Arjun Khurana: Props to David, good choice of paper. 11 00:00:46.770 --> 00:00:50.480 Emma Bortz: Yeah, I think it's his favorite paper. 12 00:00:50.970 --> 00:00:52.490 Emma Bortz: It always comes up. 13 00:00:52.790 --> 00:00:54.700 Arjun Khurana: Yeah, makes sense. 14 00:00:57.170 --> 00:00:58.429 Arjun Khurana: How've you been doing? 15 00:00:58.940 --> 00:01:02.650 Emma Bortz: Good, yeah. 16 00:01:02.910 --> 00:01:04.690 Emma Bortz: Lots of stuff going on. 17 00:01:04.890 --> 00:01:06.200 Arjun Khurana: Lots of stuff going on. 18 00:01:07.280 --> 00:01:08.719 Arjun Khurana: Almost too much stuff going on. 19 00:01:08.720 --> 00:01:09.320 Emma Bortz: Yeah. 20 00:01:09.620 --> 00:01:16.240 Emma Bortz: Hopefully we get… some attendance Yay! 21 00:01:16.240 --> 00:01:17.010 Arjun Khurana: Just like it. 22 00:01:26.170 --> 00:01:26.860 Emma Bortz: I've gotta go down. 23 00:01:26.860 --> 00:01:28.009 Arjun Khurana: review to see everyone. 24 00:01:28.770 --> 00:01:29.620 Emma Bortz: I know. 25 00:01:39.210 --> 00:01:40.410 Emma Bortz: Welcome. 26 00:01:53.910 --> 00:01:55.050 Emma Bortz: Hello. 27 00:01:57.020 --> 00:02:01.270 Arjun Khurana: Emma, would you happen to know, off the top of your head… yes, sorry, Arjun remains silent. 28 00:02:01.270 --> 00:02:02.140 Emma Bortz: Would you… 29 00:02:02.140 --> 00:02:09.120 Arjun Khurana: I guess this is maybe not just a Northeastern Zoom question, this is a general Zoom question. If I send a link, will it persist to people that join? 30 00:02:09.850 --> 00:02:10.870 Emma Bortz: Yes. 31 00:02:11.260 --> 00:02:12.360 Arjun Khurana: Okay, great. 32 00:02:12.360 --> 00:02:20.079 Emma Bortz: I think… I hope so. I can keep… I can be your resending, puppet, if not. 33 00:02:20.080 --> 00:02:25.370 Arjun Khurana: Then I am sending my initial straw poll for, did you read the paper? 34 00:02:31.380 --> 00:02:36.579 Arjun Khurana: To those who were in the Summer Mech Interpreting Group, welcome back. This is Subsuming That. 35 00:02:36.690 --> 00:02:40.940 Arjun Khurana: Thanks to… the lovely everyone at Endiff. 36 00:02:43.710 --> 00:02:48.070 Arjun Khurana: Let me pull up this… At the same time. 37 00:02:50.450 --> 00:02:52.539 Arjun Khurana: Really got too many windows open now. 38 00:02:53.080 --> 00:02:56.619 Arjun Khurana: This is one of the problems with having a ZoomIn browser. 39 00:03:03.940 --> 00:03:08.559 Emma Bortz: Does everyone see that link, or does anyone not see the link in the chat? 40 00:03:09.340 --> 00:03:12.259 Arjun Khurana: Specifically, people that might have joined after I sent the link. 41 00:03:12.260 --> 00:03:12.820 Emma Bortz: Yes. 42 00:03:12.950 --> 00:03:15.150 Eric Todd: I joined late and don't see it. 43 00:03:15.150 --> 00:03:16.120 Emma Bortz: Okay, so it doesn't. 44 00:03:16.120 --> 00:03:16.800 Arjun Khurana: Hey! 45 00:03:17.960 --> 00:03:21.619 Emma Bortz: I think there's a setting David can… 46 00:03:22.400 --> 00:03:29.919 Emma Bortz: I think there's a setting David can, add to the meeting where they will persist, hopefully. 47 00:03:30.980 --> 00:03:32.200 Arjun Khurana: We can ask him later. 48 00:03:32.200 --> 00:03:32.820 Emma Bortz: Yes. 49 00:03:42.860 --> 00:03:44.830 Emma Bortz: Zoom does have pins. 50 00:03:48.080 --> 00:03:52.350 Emma Bortz: Let's see… It's like, wait, does it? 51 00:03:52.610 --> 00:03:54.709 Emma Bortz: Oh, maybe not for messages. 52 00:04:00.930 --> 00:04:09.079 Arjun Khurana: Okay, well, I mean, in the meantime… Here is… If I can find it. 53 00:04:17.200 --> 00:04:18.829 Arjun Khurana: Here's what we've got so far. 54 00:04:21.279 --> 00:04:23.499 Arjun Khurana: So it looks like everybody at least took a skim. 55 00:04:24.020 --> 00:04:26.010 Arjun Khurana: Most everybody at least took a skim. 56 00:04:26.830 --> 00:04:31.610 Arjun Khurana: If you did not, I would encourage you to… 57 00:04:31.850 --> 00:04:37.499 Arjun Khurana: read it afterwards. It is a very short paper, and if you know anything about machine learning. 58 00:04:37.610 --> 00:04:39.630 Arjun Khurana: Now, I think it is… 59 00:04:40.350 --> 00:04:46.449 Arjun Khurana: Pretty intuitive, because so much of what we do now is built on what was introduced in this paper. 60 00:04:48.140 --> 00:04:52.530 Arjun Khurana: And, I don't know, I'd never read it before, I had fun reading it, so maybe you will too. 61 00:04:54.960 --> 00:05:01.849 Arjun Khurana: That being said, I think we have critical mass to actually get started. 62 00:05:05.520 --> 00:05:09.829 Arjun Khurana: Also, for those who have never met me, hi, I'm Arjun, I'm helping host this. 63 00:05:12.140 --> 00:05:19.999 Arjun Khurana: And today we are reading 1986, Romo Hart, Hinton and Williams, Learning Representations by Backpropagating Errors. 64 00:05:20.460 --> 00:05:26.460 Arjun Khurana: Before we even go into the paper, I've got some little historical context to look at. 65 00:05:28.130 --> 00:05:39.439 Arjun Khurana: There's 3 authors, Rummelhart, Hinton, and Williams. Rummelhart is a cognitive psychologist at UCSD. Before this, he was doing stuff with, like, comprehension of reading and comprehension of letters and words. 66 00:05:39.830 --> 00:05:43.700 Arjun Khurana: Which is seemingly unrelated to what's going on here. 67 00:05:43.940 --> 00:05:53.800 Arjun Khurana: But in spite of that, I think he did some cool work on this. Jeff Hinton is the guy that you all would have heard of, big ML guy. Right before this, he was working on Boltzmann machines. 68 00:05:54.010 --> 00:06:08.530 Arjun Khurana: Which we will get into in a second. And then the last author is Williams. He was a mathematician, and then he left to work for defense, and then he came back and worked on this at UCSD, and then went on to work on reinforcement learning algorithms. 69 00:06:08.760 --> 00:06:14.600 Arjun Khurana: This is a 1986 paper, so the historical context is… 70 00:06:15.220 --> 00:06:19.920 Arjun Khurana: Not terribly long, but somewhat relevant, just because it's somewhat unfamiliar to us. 71 00:06:20.140 --> 00:06:25.649 Arjun Khurana: In 1958, the Perceptron was invented. 72 00:06:25.790 --> 00:06:40.619 Arjun Khurana: The perceptron does come with a convergence theorem that you will, if you train your perceptron right, with a one-layer perceptron. Does everybody, by the way… I should make sure. Does everybody know what a perceptron is? Give me a thumbs up, or a thumbs down, or the like. 73 00:06:42.050 --> 00:06:43.830 Arjun Khurana: Great. Okay. 74 00:06:44.530 --> 00:06:56.320 Arjun Khurana: then we will proceed past that. Perceptrons were invented in 1958. If you don't know what it is, look it up. It's pretty simple. One-layer perceptron networks had a convergence theorem at the time that they could 75 00:06:57.360 --> 00:07:02.069 Arjun Khurana: If you train a one-layer perceptron on linearly separable data long enough. 76 00:07:02.340 --> 00:07:09.469 Arjun Khurana: then your perceptron will converge to a good answer, sort of like support vector machines. The problem is. 77 00:07:09.730 --> 00:07:12.540 Arjun Khurana: Most interesting problems are not linearly separable. 78 00:07:13.450 --> 00:07:18.189 Arjun Khurana: So, we don't know what to do with it in those cases at this point. 1969, 79 00:07:18.640 --> 00:07:22.380 Arjun Khurana: There's a specific, famous paper by Minsky and Papert 80 00:07:22.950 --> 00:07:26.370 Arjun Khurana: I don't think I've ever said Papert aloud, so I hope that's how it's pronounced. 81 00:07:26.500 --> 00:07:30.830 Arjun Khurana: They released this proof that a one-layer perceptron can't calculate the XOR operation. 82 00:07:31.110 --> 00:07:38.510 Arjun Khurana: They show that you can do it if you start including multiple layers, but at this point, we don't know how to train multiple layers, because we haven't invented backpropagation. 83 00:07:39.330 --> 00:07:47.710 Arjun Khurana: after this, in the 70s and 80s, people stop doing statistical AI and distributional semantic stuff for a little bit. They start getting really into symbolic AI. 84 00:07:47.980 --> 00:07:53.929 Arjun Khurana: We'll talk about this later in this reading group with Chomsky and Fodor and everything that they're arguing about. 85 00:07:55.750 --> 00:08:05.770 Arjun Khurana: Then, early 80s is when we get back to statistics. 1983, the… is the invention of simulated annealing with a proof of convergence in 1984. 86 00:08:06.070 --> 00:08:13.880 Arjun Khurana: Then in 1985, Hinton and his bud invent the Boltzmann machine, which is… another… 87 00:08:14.950 --> 00:08:23.150 Arjun Khurana: let's call it a training mechanism. Simulated annealing and Boltzmann machines are both loosely training mechanisms. They're means by which you can optimize the weights in your perceptron. 88 00:08:23.250 --> 00:08:29.380 Arjun Khurana: And they work on multiple layers, but they work so, so, so slowly that it is basically useless at the time. 89 00:08:30.120 --> 00:08:36.729 Arjun Khurana: Then, in 1985, just before the release of this paper, a bunch of different researchers, Jan Lacoon being one of them. 90 00:08:36.980 --> 00:08:43.149 Arjun Khurana: all independently discover backpropagation. And they go, oh, this could be really useful, there's a way that we can do the chain rule. 91 00:08:43.309 --> 00:08:50.170 Arjun Khurana: But nobody yet has trained it on real, useful problems, or thought of how… 92 00:08:50.430 --> 00:08:53.879 Arjun Khurana: It'll end up being representational, let's say. 93 00:08:54.400 --> 00:08:59.129 Arjun Khurana: So, this paper is really important for two big reasons. 94 00:08:59.730 --> 00:09:10.270 Arjun Khurana: The one big reason is they take backprop, and they apply it to a multi-layer perceptron in a real-life setting, and it gets the right answer. So that is an interesting proof of concept for the thing that 95 00:09:10.530 --> 00:09:14.949 Arjun Khurana: As we all probably know now, ended up becoming super, super important, and we use it all the time. 96 00:09:16.170 --> 00:09:25.969 Arjun Khurana: The second important thing, and the thing that I would like to spend the most time talking about today, is the representationality that results from using backpropagation on a multi-layer perceptron network. 97 00:09:26.640 --> 00:09:32.120 Arjun Khurana: It is at this point, before we get into the paper, that… 98 00:09:32.280 --> 00:09:37.680 Arjun Khurana: I want to do a little bit of commentary on what this reading group actually is, and what it's all about. 99 00:09:38.300 --> 00:09:40.100 Arjun Khurana: If this were a normal reading group. 100 00:09:40.360 --> 00:09:46.920 Arjun Khurana: We would be focusing on this part, the training real things, Of the back propagation. 101 00:09:47.280 --> 00:09:53.790 Arjun Khurana: This is not quite a normal reading group. From the title, you could probably assume that this reading group is about metalearning and cognition. 102 00:09:57.190 --> 00:09:59.790 Arjun Khurana: These are two very… 103 00:10:01.110 --> 00:10:08.249 Arjun Khurana: Complicated, closely related topics that we should be thinking about all the time, and don't think about all the time. 104 00:10:09.570 --> 00:10:10.290 Arjun Khurana: if… 105 00:10:10.790 --> 00:10:21.899 Arjun Khurana: anybody in this call could put together an indisputable definition of either cognition or metalearning, I would be very, very impressed. I think a big part of this reading group is going to be determining 106 00:10:22.150 --> 00:10:25.719 Arjun Khurana: What those two words actually mean, and how we should think about them. 107 00:10:26.190 --> 00:10:35.490 Arjun Khurana: But, at a very, very general level, what we're going to be talking about in this reading group, we will be going into some of the technical aspects of these papers, and some of the technical aspects of interpretability. 108 00:10:35.930 --> 00:10:40.390 Arjun Khurana: But… More so, the focus is going to be on 109 00:10:41.450 --> 00:10:48.259 Arjun Khurana: What does it mean for a model to learn? What does it mean for a model to understand? And what does it mean for us to understand a model? 110 00:10:48.860 --> 00:10:54.030 Arjun Khurana: Then we can get into… this will very easily lead us into more cognitive aspects, like… 111 00:10:54.220 --> 00:11:00.659 Arjun Khurana: What is a concept? What does it mean to think? How can we tell if a model thinks something? How can we tell if a model possesses a concept? 112 00:11:01.450 --> 00:11:06.680 Arjun Khurana: These are all questions that I think are pretty essential to… Metalearning. 113 00:11:07.120 --> 00:11:09.989 Arjun Khurana: And pretty essential to… 114 00:11:10.180 --> 00:11:16.090 Arjun Khurana: definitely mechanist… definitely mechanistic interpretability, but I think also the field of machine learning as a whole. 115 00:11:17.330 --> 00:11:23.220 Arjun Khurana: I want to, at this point, invite Emma or Eric, if you have any other comments on Metalearning and Cognition, before we actually get into the paper. 116 00:11:24.280 --> 00:11:31.669 Eric Todd: Yeah, I… so, I'm excited about this. I think one of the things that is most interesting to me is, like. 117 00:11:32.020 --> 00:11:38.589 Eric Todd: I think of Metalearning as, like, learning to learn, and so, like, now with these big LLMs. 118 00:11:38.740 --> 00:11:44.190 Eric Todd: We have models that, like, basically we've trained them to… 119 00:11:44.620 --> 00:11:57.779 Eric Todd: learn from context how to do a bunch of different tasks, which is, like, very different from the paradigm that, like, these early researchers had of, like, oh, we just have this one task, and we want it to… 120 00:11:57.820 --> 00:12:12.129 Eric Todd: to, like, find features that are, like, helpful for this task. Now we're, like, training things so that they can, like, learn stuff on the fly. And I… and I feel like this… this is, like, a really important paradigm shift, and so, like, trying to understand, sort of, like, the foundations of… 121 00:12:12.460 --> 00:12:15.569 Eric Todd: You know, where this is coming from, and like… 122 00:12:15.680 --> 00:12:22.749 Eric Todd: what… how we can, like, understand it better, might… I don't know, I'm really excited about, like, diving into these kinds of questions. 123 00:12:25.190 --> 00:12:37.999 Emma Bortz: Yeah, thank you guys. I don't have too much more to add, but thanks, everyone, for being here today. And we have some people on the NDIF team here as well. Adam… 124 00:12:38.250 --> 00:12:41.080 Emma Bortz: And hopefully we can… 125 00:12:41.550 --> 00:12:51.780 Emma Bortz: dig into some of the code in this paper, and play around with that as well. And, yeah, I'll give it back to Arjun. 126 00:12:52.950 --> 00:12:56.460 Arjun Khurana: Sure. Unless, Adam, you'd like to mention anything about notebooks. 127 00:12:58.950 --> 00:12:59.890 Arjun Khurana: Muted. 128 00:13:02.940 --> 00:13:07.540 Eric Todd: No, but I think Eric is gonna walk us through the notebook that he can file later today. 129 00:13:08.320 --> 00:13:11.399 Arjun Khurana: Cool, sounds good. In that case, let's start on the paper. 130 00:13:15.150 --> 00:13:22.440 Eric Todd: Oh, and, and, just… Yep. Maybe you already said, and I was late, but, like, feel free, other people, to just, like. 131 00:13:22.570 --> 00:13:37.559 Eric Todd: jump in with questions, you know, if there's something that you read. Like, there's a lot… when I was reading this, there's, like, a lot of language where it's, like, we've sort of upgraded how we talk about things, and so some of the things here, like, are old or weird phrased. 132 00:13:38.050 --> 00:13:44.460 Eric Todd: so yeah, if people have questions, like, feel free to just pipe up 133 00:13:44.780 --> 00:13:46.389 Eric Todd: And we can walk through stuff. 134 00:13:48.110 --> 00:13:48.660 Arjun Khurana: Yeah. 135 00:13:49.120 --> 00:13:54.549 Arjun Khurana: I mean, you can see right here, second sentence, I've already got a question on… 136 00:13:55.180 --> 00:13:57.729 Arjun Khurana: the language they're using. 137 00:13:57.880 --> 00:14:08.000 Arjun Khurana: But yes, please interrupt me. If you are scared to interrupt me, raise your hand. In the Zoom, I think it gives me a sound, and if it doesn't give me a sound, then Eric will shout out on your behalf. 138 00:14:09.030 --> 00:14:15.200 Arjun Khurana: Also, if you're really scared, feel free to use chat, but I will be checking it somewhat infrequently since I'll be on the other tab. 139 00:14:16.480 --> 00:14:18.330 Arjun Khurana: In general, feel free to interrupt me. 140 00:14:19.310 --> 00:14:22.050 Arjun Khurana: But, let's actually get into what's going on here. 141 00:14:27.130 --> 00:14:29.789 Arjun Khurana: I want to start with this second sentence. 142 00:14:30.900 --> 00:14:34.410 Arjun Khurana: The aim of designing a self-organizing neural network 143 00:14:35.190 --> 00:14:44.360 Arjun Khurana: Is to find a powerful synaptic modification rule that will allow an arbitrarily connected neural network to develop an internal structure that is appropriate for a particular task domain. 144 00:14:44.520 --> 00:14:47.200 Arjun Khurana: So, right off the bat. 145 00:14:47.670 --> 00:14:53.209 Arjun Khurana: They are talking about how their goal is not even performance of the model, their goal is the development of internal structure. 146 00:14:54.490 --> 00:15:01.249 Arjun Khurana: This will become relevant throughout the later 80s and early 90s as people are debating Internal structures of Cognition. 147 00:15:01.590 --> 00:15:06.209 Arjun Khurana: But for now, I want to draw our attention to the word appropriate here, because… 148 00:15:06.780 --> 00:15:15.110 Arjun Khurana: The way that you decide an internal structure is appropriate or inappropriate to a particular task becomes very… is very simple for very trivial tasks and very complicated 149 00:15:15.240 --> 00:15:16.630 Arjun Khurana: for other tasks. 150 00:15:16.840 --> 00:15:20.100 Arjun Khurana: Everybody debates this word. 151 00:15:20.590 --> 00:15:23.630 Arjun Khurana: Appropriate, or specifically appropriate for a task domain. 152 00:15:24.600 --> 00:15:26.750 Arjun Khurana: Throughout the entire 80s. 153 00:15:27.970 --> 00:15:31.520 Arjun Khurana: This whole section after this is describing 154 00:15:32.030 --> 00:15:36.330 Arjun Khurana: The idea of finding a learning procedure, And… 155 00:15:36.980 --> 00:15:45.030 Arjun Khurana: at the very end, they get to this core sentence, which is the thing that they hadn't figured out until 1985, or I guess technically 1983, which is. 156 00:15:45.130 --> 00:15:49.450 Arjun Khurana: How do we decide… when we have hidden units, how do we decide how those hidden units should be active? 157 00:15:52.810 --> 00:15:58.110 Arjun Khurana: This is a description of a basic perceptron. If you were not familiar before, this math should enlighten you. 158 00:15:58.320 --> 00:16:03.639 Arjun Khurana: It was interesting to me that even right off the bat, they're using the… 159 00:16:04.700 --> 00:16:11.010 Arjun Khurana: I think this is the actual sigmoid… Calculation for an activation function. 160 00:16:11.190 --> 00:16:15.160 Arjun Khurana: We will skip over these two figures for now, and jump down here. 161 00:16:16.230 --> 00:16:18.130 Arjun Khurana: They establish a net loss. 162 00:16:18.650 --> 00:16:27.249 Arjun Khurana: Which is intuitive to us now, and then they minimize… they decide that they want to minimize the error by gradient descent, which is pretty classic. 163 00:16:28.800 --> 00:16:47.240 Arjun Khurana: But the back propagation in order to find the partial derivatives of this error term with respect to every individual weight is, though very intuitive to us, a very big thing for them to be doing at this point. This is taken, I think they cite Don Lacoon, but two other people that year also came to the same conclusion independently. 164 00:16:53.730 --> 00:16:58.599 Arjun Khurana: Backwards pass with propagates derivatives from the top layer back to the bottom one is more complicated, and then… 165 00:16:59.000 --> 00:17:05.210 Arjun Khurana: They do, essentially out by hand. This is how the chain rule looks, this is how back propagation looks. 166 00:17:06.130 --> 00:17:09.720 Arjun Khurana: When you're running torch.gradientTape, this is what's happening. 167 00:17:10.440 --> 00:17:15.740 Arjun Khurana: This general idea 168 00:17:19.599 --> 00:17:21.920 Arjun Khurana: is… 169 00:17:28.910 --> 00:17:32.260 Arjun Khurana: No, okay. Here, they're doing the, 170 00:17:33.580 --> 00:17:39.569 Arjun Khurana: sort of classic stuff. This is a very, very basic gradient descent. This is a gradient descent… 171 00:17:39.920 --> 00:17:45.110 Arjun Khurana: with, Momentum, but… 172 00:17:46.940 --> 00:17:53.120 Arjun Khurana: Essentially, everything they're doing here, from this column all the way down to this point, should be very familiar to you. 173 00:17:54.370 --> 00:17:58.899 Arjun Khurana: It is interesting the way that they're thinking about it in these two figures, which we will talk about in a moment. 174 00:18:01.180 --> 00:18:05.160 Arjun Khurana: But… They sort of are… 175 00:18:05.590 --> 00:18:13.139 Arjun Khurana: in these last… these two paragraphs right here, directly engaging with these guys from 1969 that are like, hey, a perceptron can't do XORs. 176 00:18:13.480 --> 00:18:19.369 Arjun Khurana: And they agree with this. They say, here's an even simpler example, a one-layer 177 00:18:19.850 --> 00:18:26.880 Arjun Khurana: a one-layer perceptron network can't detect symmetry. You can't see if input A equals negative input B, 178 00:18:27.380 --> 00:18:38.679 Arjun Khurana: Because there's nothing… there's no hidden unit that's looking at input A and input B. Therefore, we have to be using an intermediate layer, and so all interesting problems in the future will be probably using intermediate layers. 179 00:18:42.420 --> 00:18:52.059 Arjun Khurana: There's a brief discussion here about local minima, and the fact that local minima becomes less of a problem in higher dimensionality, which… 180 00:18:52.160 --> 00:18:59.409 Arjun Khurana: Is, funnily enough, something that we still have to talk about all the time today, and still have a hard time comprehending today. 181 00:18:59.600 --> 00:19:02.440 Arjun Khurana: I think it's funny that they were thinking about this in 1986. 182 00:19:04.200 --> 00:19:07.970 Arjun Khurana: Then, finally, they describe this… 183 00:19:08.640 --> 00:19:15.040 Arjun Khurana: Pretty important sentence. The learning procedure in its current form is not a plausible model of learning in brains. 184 00:19:16.320 --> 00:19:24.719 Arjun Khurana: But, they do find interesting internal representations. And this is where I think the most interesting part of this paper comes in. We will talk about this in a second. 185 00:19:25.290 --> 00:19:28.320 Arjun Khurana: Real quick, pausing for questions on the literal 186 00:19:28.550 --> 00:19:31.630 Arjun Khurana: gradient descent, backpropagation part, or anything I just said. 187 00:19:39.070 --> 00:19:43.440 Jake Russin: I have a comment. I always… when I first thought about this way back in… 188 00:19:43.920 --> 00:19:50.599 Jake Russin: grad school… I'm a postdoc, by the way, so maybe I'm, like, the oldest one in here, I'm not sure. But, 189 00:19:50.990 --> 00:19:56.679 Jake Russin: I always thought, like, Isn't it just… why is there even a name for backprop? 190 00:19:56.920 --> 00:20:04.149 Jake Russin: Like, why… isn't it just sort of straightforward application of the rules that we know from calculus to… 191 00:20:04.700 --> 00:20:09.040 Jake Russin: You know, basically, stochastic… to do stochastic gradient descent on the 192 00:20:09.240 --> 00:20:12.609 Jake Russin: the neural network that we define in advance, you know what I mean? 193 00:20:13.070 --> 00:20:18.029 Jake Russin: And I feel like… I guess it is nice to revisit the paper, because it's sort of… 194 00:20:18.540 --> 00:20:24.869 Jake Russin: Reminds you that it's not… it's only, like, we only have that perspective, like, in retrospect, kind of, you know? 195 00:20:25.150 --> 00:20:31.010 Jake Russin: or, I don't know, I mean, I guess you could also argue that bat propagation 196 00:20:31.310 --> 00:20:34.530 Jake Russin: is innovative, just as an algorithm, like… 197 00:20:34.790 --> 00:20:40.429 Jake Russin: Did they even discuss that? I didn't… I was one of the skimmers, by the way, because I had already read it in grad school. 198 00:20:40.430 --> 00:20:41.010 Arjun Khurana: That is fine. 199 00:20:41.010 --> 00:20:45.949 Jake Russin: Like, keeping activations around and stuff like that, like, smart, like, smart ways of optimizing it. 200 00:20:46.530 --> 00:20:47.200 Arjun Khurana: Well, they discussed… 201 00:20:47.200 --> 00:20:48.359 Jake Russin: Maybe that's innovative. 202 00:20:49.310 --> 00:20:55.350 Arjun Khurana: This right here is pretty innovative, immediately being like, oh, we're going to add a momentum term to… 203 00:20:55.990 --> 00:20:57.290 Arjun Khurana: gradient descent. 204 00:20:57.670 --> 00:21:05.320 Arjun Khurana: But they are mostly citing here Parker and Lacoon doing variance on the learning procedure. 205 00:21:05.560 --> 00:21:11.930 Arjun Khurana: I don't think at this point they're doing anything with, like, carryover streams. 206 00:21:12.250 --> 00:21:17.490 Arjun Khurana: But I will agree, like, I had the same impression as well, especially when… 207 00:21:17.880 --> 00:21:26.960 Arjun Khurana: David suggested we read this paper as our first paper. I was like, it's back propagation, it's just, like, people were throwing their heads at the wall, and then they discovered the right way to do calculus, and now, looking back. 208 00:21:26.960 --> 00:21:28.559 Jake Russin: The right way, yeah, exactly. 209 00:21:30.170 --> 00:21:31.729 Arjun Khurana: I'm going to link in the chat. 210 00:21:31.940 --> 00:21:38.900 Arjun Khurana: the 1983 paper introducing simulated annealing, because I think this comes from a really interesting moment in time. 211 00:21:39.080 --> 00:21:43.049 Arjun Khurana: Which is between the development of a forward pass. 212 00:21:43.610 --> 00:21:51.460 Arjun Khurana: In 58, and the development of a backward pass in here, this paper, 86. 213 00:21:51.610 --> 00:21:54.459 Arjun Khurana: You have people sort of trying to… 214 00:21:55.140 --> 00:22:00.940 Arjun Khurana: find their way towards backpropagation and not really making it there yet, and simulated annealing 215 00:22:01.230 --> 00:22:04.499 Arjun Khurana: Is one of the ways we could have gone, but is… 216 00:22:05.090 --> 00:22:11.470 Arjun Khurana: If you look not necessarily at the algorithm they're developing, but at the way that they're describing the problem space, and 217 00:22:11.840 --> 00:22:20.009 Arjun Khurana: how to… I can't screen share this because I don't have institutional access, but if you look at the way that they describe the problem space, and the way that they're, like. 218 00:22:23.370 --> 00:22:24.370 Arjun Khurana: Oh! 219 00:22:24.820 --> 00:22:26.040 Arjun Khurana: That's interesting. 220 00:22:26.690 --> 00:22:32.519 Arjun Khurana: Anyways, I think this 1983 paper is pretty revealing about how unintuitive 221 00:22:32.830 --> 00:22:39.060 Arjun Khurana: It was at the time, just how lost everybody was in trying to meander their way towards backpropagation. 222 00:22:39.380 --> 00:22:44.279 Arjun Khurana: Because without knowing that answer in advance, it's a little bit hard to think of. 223 00:22:45.120 --> 00:22:48.329 Arjun Khurana: Especially given that they didn't have crazy computational power. 224 00:22:49.940 --> 00:23:01.529 Arjun Khurana: everyone was also, like, the statisticians weren't into calculus, the statisticians were into Monte Carlo methods, and everyone thought that Markov chains were gonna be the future. And then you had, what's her name, the… 225 00:23:01.710 --> 00:23:03.859 Arjun Khurana: Symbolic Analysis Therapist Bot. 226 00:23:04.590 --> 00:23:10.320 Arjun Khurana: on the top of everyone's minds. It was a very strange time for this sort of development. 227 00:23:12.560 --> 00:23:15.509 Oliver McLaughlin: On that, it is pretty striking. Wait, can you guys hear me? Yeah, you guys can hear me. 228 00:23:15.510 --> 00:23:16.190 Arjun Khurana: Yep. 229 00:23:16.370 --> 00:23:19.189 Oliver McLaughlin: It is pretty striking, like, the lack of statistical 230 00:23:19.360 --> 00:23:25.250 Oliver McLaughlin: language in this paper, right? Which I guess is probably historical, but just, I was reading this, and I was waiting for, like. 231 00:23:25.480 --> 00:23:40.750 Oliver McLaughlin: I don't know, like, like, they basically do, like, batched gradient descent, right? But, like, instead of talking about it as if they're, like, effectively, like, averaging out errors or trying to have, like, a more, constrained, like, variance on their gradient, they're just, like, it works better, effectively, like, it's, like, you know what I'm talking about? Like, they don't use, like, the kind of language we use now, which is really interesting. 232 00:23:42.520 --> 00:23:54.120 Arjun Khurana: I'll draw our attention right here to… we trained the network for 1500 sweeps with these hyperparameters for the first 20 sweeps, and these for the rest of the sweeps, and we introduced weight decay. 233 00:23:55.170 --> 00:24:00.470 Arjun Khurana: And then there's these, sort of, arbitrary cases here of… 234 00:24:04.360 --> 00:24:11.869 Arjun Khurana: this… Which, like, imagine doing that in a paper you're submitting now. That would be crazy. 235 00:24:13.710 --> 00:24:14.699 Oliver McLaughlin: Wait, that… is that… 236 00:24:14.700 --> 00:24:15.140 Arjun Khurana: Oh, okay. 237 00:24:15.620 --> 00:24:20.120 Oliver McLaughlin: Is that weight decay in any way equivalent to what we do now, where you, like, take, like, the L2 penalty? 238 00:24:20.590 --> 00:24:23.549 Oliver McLaughlin: What is decrementing every weight by 0.2… no, that's completely different. 239 00:24:23.550 --> 00:24:27.710 Arjun Khurana: They, they, they, it is a constant weight decay with no respect to the loss. 240 00:24:30.900 --> 00:24:36.979 Arjun Khurana: But yeah, it is very odd. It's weird to read, 241 00:24:37.940 --> 00:24:41.410 Arjun Khurana: I kind of like it, honestly. It makes me feel like it's… 242 00:24:43.100 --> 00:24:46.559 Arjun Khurana: Gesturing more at the concepts than the results, which… 243 00:24:46.910 --> 00:24:49.709 Arjun Khurana: In general, for the field, is not great. 244 00:24:49.840 --> 00:24:56.509 Arjun Khurana: all the time, but in this specific case, when we're talking about Metalearning and learning representations, I think is pretty useful. 245 00:24:59.610 --> 00:25:03.130 Arjun Khurana: Any more comments before we proceed to their weird interpretability? 246 00:25:07.340 --> 00:25:26.519 Eric Todd: So, I feel like one of… I mean, now we have, like, PyTorch that does all of this, like, keeping track of gradients for us, but, like, I think… I mean, part of the problem that this took so long for people to figure out is, like, a lot of them were, like, doing gradients by hand, or, like, very primitively 247 00:25:26.650 --> 00:25:45.350 Eric Todd: And, like, just… it was really hard. With, like, so many param… like, now we have, like, billions of parameters, it's easy to keep track of stuff with, like, automated methods, but, like, even with… I don't know how big these little models that they have are, but, like, even with this, this is, like, if you're trying to do something like this by hand to, like, show that this sort of method makes sense, it's, like… 248 00:25:45.720 --> 00:25:48.490 Eric Todd: A lot of bookkeeping, and it's easy to make mistakes. 249 00:25:48.600 --> 00:25:51.440 Eric Todd: So, I think, like, there are a lot of, like. 250 00:25:51.780 --> 00:25:54.880 Eric Todd: Just, like, sitting down and being deliberate about 251 00:25:55.030 --> 00:26:00.249 Eric Todd: Designing, infrastructure for doing this kind of thing has also, like. 252 00:26:00.700 --> 00:26:05.690 Eric Todd: Made it a lot more, like, you know. 253 00:26:05.810 --> 00:26:09.269 Eric Todd: Understandable now compared to how it was back then. 254 00:26:12.000 --> 00:26:16.059 Arjun Khurana: Yeah, this is a good point. Imagine having to do all these derivatives by hand. 255 00:26:22.210 --> 00:26:25.000 Arjun Khurana: Alright, on that note, let's get into some of the juice. 256 00:26:27.300 --> 00:26:39.580 Arjun Khurana: Figure 1, we've got this cool diagram of a… symmetry detection, network that… they, trained through… 257 00:26:39.730 --> 00:26:49.190 Arjun Khurana: These hyperparameters with A 64 input size, data set. 258 00:26:49.600 --> 00:27:00.439 Arjun Khurana: the reason this is relevant, and I think the reason they're showing this off in the first place, is to show that while a single-layer network can't do symmetry, like they expressed sort of at the bottom here. 259 00:27:00.830 --> 00:27:07.989 Arjun Khurana: A multi-level network can do symmetry, and so these networks with hidden layers are going to be really important, because that was something they had to justify at the time. 260 00:27:10.050 --> 00:27:12.470 Arjun Khurana: This… these details are uninteresting, I think. 261 00:27:13.930 --> 00:27:18.119 Arjun Khurana: Figure 2, we've got one of their new problem statements. 262 00:27:18.980 --> 00:27:21.660 Arjun Khurana: Their goal is to take in 263 00:27:22.050 --> 00:27:28.389 Arjun Khurana: Their dataset is the form of triples in this family tree, and you're trying to predict the third element in the triple. 264 00:27:28.830 --> 00:27:42.290 Arjun Khurana: So, who is… Victoria's mom, so Victoria, mother, Penelope, it is a… 265 00:27:42.970 --> 00:27:45.120 Arjun Khurana: To us, a very simple task. 266 00:27:45.290 --> 00:27:52.120 Arjun Khurana: The reason I think it is interesting here is because of two things. One is… 267 00:27:52.610 --> 00:27:55.580 Arjun Khurana: They make an expressly behaviorist claim here. 268 00:27:56.350 --> 00:27:57.760 Arjun Khurana: Which is that… 269 00:27:58.150 --> 00:28:03.609 Arjun Khurana: A layered net can be said to two know the triples if it can produce the third term when given the first two. 270 00:28:04.080 --> 00:28:09.669 Arjun Khurana: This is a behaviorist claim because they're saying the knowledge in the network is based on its behavior. 271 00:28:10.880 --> 00:28:15.330 Arjun Khurana: The second thing is they have two isomorphic family trees here. 272 00:28:15.580 --> 00:28:21.599 Arjun Khurana: With the idea being, if you train on one of these family trees, then you have the same representations for the next family tree. 273 00:28:24.040 --> 00:28:30.260 Arjun Khurana: I think in reality, they do train on both and just show that the mechanisms become shared. 274 00:28:31.640 --> 00:28:33.330 Arjun Khurana: So let's look at Figure 3. 275 00:28:33.470 --> 00:28:39.910 Arjun Khurana: Which is… activations in a network that has been trained on the Figure 2 task. 276 00:28:40.260 --> 00:28:43.869 Arjun Khurana: I think this is a really cool way of looking at activations. It is so… 277 00:28:44.710 --> 00:29:00.219 Arjun Khurana: Obviously, now we think… we are thinking so much about directions all the time, that looking at individual activations from individual neurons like this doesn't make a lot of sense, but it was cool that back in the day, they were thinking about it as an individual neuron activating and having an output. 278 00:29:00.420 --> 00:29:05.780 Arjun Khurana: So here, we've got dark for… Negative, white for positive. 279 00:29:06.230 --> 00:29:09.689 Arjun Khurana: Maybe it's dark for zero, white for positive, I forget exactly how they set it up. 280 00:29:09.990 --> 00:29:20.220 Arjun Khurana: But… It is not a fully, fully connected network, there's some divisions that go on. 281 00:29:22.580 --> 00:29:28.850 Arjun Khurana: the… Both the output input units and output linear. 282 00:29:31.160 --> 00:29:35.879 Eric Todd: Yeah, so… so I… I spent some time… I've… I made a little, notebook 283 00:29:36.030 --> 00:29:44.479 Eric Todd: With… that, like, replicates this family tree network, because I was curious, like, what the heck was going on. I'll paste… 284 00:29:44.480 --> 00:29:45.270 Arjun Khurana: showed off? 285 00:29:45.270 --> 00:29:48.729 Eric Todd: I'll paste it in the chat, if people want to play around with it. 286 00:29:49.100 --> 00:29:54.950 Eric Todd: But basically, each of these blocks is, like, One, like, the first… 287 00:29:56.300 --> 00:30:01.509 Eric Todd: there's, like, 24 people, so they have… it's like this… the architecture is weird. There's, like… 288 00:30:01.730 --> 00:30:09.760 Eric Todd: 24 people in this, dataset, and so the bottom left, like, 289 00:30:09.760 --> 00:30:10.660 Arjun Khurana: these guys? 290 00:30:10.810 --> 00:30:25.359 Eric Todd: Yeah, the bottom left rectangle is, like, there's 24 of those blocks, and, like, only one of… it's, like, a one-hot encoding, so, like, they just pick one person, so it's, like, this one is, like, Colin, for example, and then the bottom right block 291 00:30:25.360 --> 00:30:34.180 Eric Todd: Yeah. The bottom right block is the encoding for the relation, so they have another one hot encoding for, like, there's 12 relations in their dataset. 292 00:30:34.380 --> 00:30:39.120 Eric Todd: So it's like, mother, father, sister, brother, whatever. 293 00:30:39.280 --> 00:30:49.310 Eric Todd: And so this is how they, like, ask the question. They have, like, one hot encoding for the person, one hot encoding for the relation, and then… and then it's basically like a three-layer network. 294 00:30:49.770 --> 00:30:53.860 Eric Todd: So the… so the first layer is, like, goes from… 295 00:30:54.960 --> 00:31:03.880 Eric Todd: Like, it… half of it reads the… the input person, so, like, the… it's like the person encoding, basically, and then there's, like, another chunk, which is, like… 296 00:31:04.070 --> 00:31:10.879 Eric Todd: The relation encoding. And then there's, like, a joint block in the middle, which is supposed to combine all of those. 297 00:31:12.030 --> 00:31:15.080 Arjun Khurana: This, here, like this, and then… Yeah. 298 00:31:16.950 --> 00:31:21.129 Arjun Khurana: this guy… Takes in both of these. 299 00:31:21.340 --> 00:31:25.840 Eric Todd: Right, so you, like, concat… you, like, concatenate those two, blue… 300 00:31:26.190 --> 00:31:37.140 Eric Todd: blue guys into a single vector for the input to, like, this next layer in the network. And then it goes to, like, another… a shrinking layer, and then back out to predict, like, a… basically a… 301 00:31:37.340 --> 00:31:44.539 Eric Todd: a prediction over all people, like, who fits this relation. So in this case, like, Colin has two aunts. 302 00:31:44.720 --> 00:31:48.739 Eric Todd: And you see, like, two big squares where it's… the model's 303 00:31:48.990 --> 00:31:52.569 Eric Todd: Predicting, like, both of these people are his aunt. 304 00:31:55.390 --> 00:32:02.950 Eric Todd: Yeah, so I made… I made a little notebook, I think I linked it, you guys can go check it out, but, like, 305 00:32:03.980 --> 00:32:08.659 Eric Todd: Let's see. Let me just… it might take a little bit to run on Colab. 306 00:32:10.500 --> 00:32:16.569 Eric Todd: I don't know, I didn't… I didn't, benchmark the training time on… on Colab. 307 00:32:16.570 --> 00:32:18.090 Arjun Khurana: This is impressively… 308 00:32:18.400 --> 00:32:22.560 Arjun Khurana: rigorous. I don't think I would have been able to put this together based on the descriptions they included. 309 00:32:25.520 --> 00:32:28.439 Eric Todd: Yeah, I thought it was a fun- a fun little problem. 310 00:32:28.600 --> 00:32:36.460 Eric Todd: But, like, looking at this diagram, I… I… it was like… a little… funky, I don't know. 311 00:32:38.250 --> 00:32:44.380 Eric Todd: But yeah, if you want to see the PyTorch version, there's the link for it. 312 00:32:44.380 --> 00:32:45.310 Arjun Khurana: Look at the notebook. 313 00:32:46.450 --> 00:32:46.810 Eric Todd: Yeah. 314 00:32:46.810 --> 00:32:52.759 Arjun Khurana: If anyone has specific things on the notebook, I've got it open in another tab, and I can switch at any point. 315 00:32:54.290 --> 00:32:56.190 Arjun Khurana: But, I do want to… 316 00:32:57.250 --> 00:33:01.599 Arjun Khurana: At this point, if we understand the network and the problem statement that they're doing. 317 00:33:01.740 --> 00:33:04.460 Arjun Khurana: Look at this figure 4. 318 00:33:07.730 --> 00:33:15.470 Arjun Khurana: These are… The weights from the person inputs to the… 319 00:33:15.830 --> 00:33:18.049 Arjun Khurana: 6 units in the second layer. 320 00:33:18.730 --> 00:33:21.610 Arjun Khurana: So, Figure 4 is looking at… 321 00:33:24.740 --> 00:33:26.260 Arjun Khurana: Where's my eraser? 322 00:33:31.920 --> 00:33:33.940 Arjun Khurana: Figure 4 is looking at these weights. 323 00:33:37.120 --> 00:33:39.920 Arjun Khurana: Can everyone still hear me? Because my computer just started freaking out a little? 324 00:33:40.810 --> 00:33:41.940 Eric Todd: Yeah, we can hear you. 325 00:33:42.370 --> 00:33:43.460 Arjun Khurana: Great, okay. 326 00:33:50.160 --> 00:33:51.090 Arjun Khurana: Jesus. 327 00:33:55.560 --> 00:33:59.270 Arjun Khurana: Unit 1, Which they declare as… 328 00:33:59.600 --> 00:34:03.540 Arjun Khurana: The, first neuron of the 6 neurons that we're looking at. 329 00:34:03.900 --> 00:34:09.879 Arjun Khurana: has… weight differentiations for the English people versus the Italian people. 330 00:34:11.719 --> 00:34:18.880 Arjun Khurana: And the way that they determine that is by looking at the weights directly, rather than looking at the activations. 331 00:34:20.260 --> 00:34:27.270 Arjun Khurana: This is a very subtle difference, and looking at the weights or looking at the activations would have produced exactly the same answer in this case. 332 00:34:27.860 --> 00:34:33.370 Arjun Khurana: But I want to mention it because it provides some insight into how they're actually thinking about what's going on here. 333 00:34:34.630 --> 00:34:39.469 Arjun Khurana: So, you've got highly positive weights, for… 334 00:34:40.040 --> 00:34:43.549 Arjun Khurana: the Italian… for the English highly negative weights for the Italians. 335 00:34:46.370 --> 00:34:55.260 Arjun Khurana: The claim that they make based on this is that the representation of an English person is very similar to the representation of their Italian equivalent. 336 00:34:56.370 --> 00:35:11.719 Arjun Khurana: Because only Unit 1 is making the difference, and everything else is not making the difference. Obviously, we can see Unit 5 here is also making a slight differentiation, but they sort of ignore that. But Unit 4, 2, 6, and 3 all have 337 00:35:12.210 --> 00:35:17.690 Arjun Khurana: Basically, matching weights Top to bottom in this 24. 338 00:35:18.150 --> 00:35:20.350 Arjun Khurana: In this 2x12 matrix. 339 00:35:23.260 --> 00:35:25.139 Arjun Khurana: They make another claim here. 340 00:35:25.590 --> 00:35:34.810 Arjun Khurana: Which is that the network is using the fact that the two family trees are isomorphic to share structure, basically to share computational circuits. 341 00:35:35.790 --> 00:35:38.449 Arjun Khurana: So that they can generalize from one tree to the other. 342 00:35:40.370 --> 00:35:46.470 Arjun Khurana: This Unit 2 encodes generation agnostic of Italian versus English. 343 00:35:47.370 --> 00:35:52.040 Arjun Khurana: and Unit 6, encodes branch within 344 00:35:52.280 --> 00:35:56.150 Arjun Khurana: Either Italian or English without encoding, if you are Italian or English. 345 00:35:59.460 --> 00:36:09.989 Arjun Khurana: First off, this is a cool way of looking at this, let's just say. Being able to see, oh, there's white on top and black on bottom, therefore it's making the English-Italian distinction. 346 00:36:10.120 --> 00:36:20.730 Arjun Khurana: Versus, oh, the top and bottom match up in magnitude for all of these instances, therefore it is ignoring the distinction. That's a cool way of thinking about this. 347 00:36:22.150 --> 00:36:27.630 Arjun Khurana: At the same time, The way that they… have… 348 00:36:27.990 --> 00:36:32.460 Arjun Khurana: Observation… observation, claim 1, claim 2, claim 3. 349 00:36:33.290 --> 00:36:39.569 Arjun Khurana: is… Interesting, especially given the way that we make claims now with causation, etc. 350 00:36:40.240 --> 00:36:42.790 Arjun Khurana: I want to hear if anyone had thoughts on this caption here. 351 00:36:43.230 --> 00:36:45.350 Eric Todd: Oh yeah, I like Oliver's comment. 352 00:36:45.860 --> 00:36:47.029 Eric Todd: In the chat. 353 00:36:48.000 --> 00:36:53.399 Arjun Khurana: what'd he say? No causal intervention, no prose, no dictionary learning, just vibes. Yes. 354 00:36:53.950 --> 00:36:55.740 Arjun Khurana: But, at the same time. 355 00:36:55.890 --> 00:37:03.579 Arjun Khurana: In spite of that, in spite of the fact that they are not doing causal intervention, they are not looking at how these weights are being used, they're not looking at what's going on. 356 00:37:04.050 --> 00:37:06.599 Arjun Khurana: We look at this diagram, and we go. 357 00:37:06.880 --> 00:37:11.420 Arjun Khurana: Oh yeah, obviously they're right. Like, this is a very simple network, that's gotta be what's going on here. 358 00:37:14.300 --> 00:37:18.660 Eric Todd: I think it's really interesting, that, like, they're using… 359 00:37:18.770 --> 00:37:21.330 Eric Todd: The fact that their model learned 360 00:37:21.950 --> 00:37:26.480 Eric Todd: structure as, like, validation that backprop is, like, worth… 361 00:37:26.600 --> 00:37:29.309 Eric Todd: thinking about, like, worth thinking about, you know? Like… 362 00:37:29.690 --> 00:37:34.119 Eric Todd: Yeah, I think that's one of the reasons that, like, David likes this paper so much, is, like. 363 00:37:34.290 --> 00:37:42.300 Eric Todd: you know, they're using interpretability to, like, point out, like, hey, you know, maybe backprop is, like, Worth using. 364 00:37:42.930 --> 00:37:51.120 Eric Todd: And now… and now we've, like, scaled it up, and it's really hard to do this same kind of, like, just look at the weights and tell you what's going on analysis, but… 365 00:37:51.620 --> 00:37:53.359 Eric Todd: Like, as a… 366 00:37:53.870 --> 00:38:04.490 Eric Todd: As a first motivation, or, like, for, like, validation for, like, why people should care about this method of, like, hey, it's learning… it's learning interpretable features. 367 00:38:04.930 --> 00:38:08.730 Eric Todd: That, like, makes sense based on the data we're training on. I think it's pretty cool. 368 00:38:11.090 --> 00:38:12.289 Arjun Khurana: I think that's cool, too. 369 00:38:12.410 --> 00:38:14.500 Arjun Khurana: There is a historical reason for it. 370 00:38:15.160 --> 00:38:17.840 Arjun Khurana: When all these papers were coming out. 371 00:38:18.890 --> 00:38:22.769 Arjun Khurana: I mean, remember, until 19… from 69 until 83, 372 00:38:23.600 --> 00:38:31.980 Arjun Khurana: Everybody had… not everybody, but basically everybody had sort of abandoned the idea that we could use statistics etc. 373 00:38:32.230 --> 00:38:35.599 Arjun Khurana: to be able to do machine learning. Everyone was going all in on 374 00:38:36.030 --> 00:38:47.379 Arjun Khurana: symbolic architectures. The idea prevailing at the time was that if you weren't able to explicitly encode rules about how to do logic, then you weren't able to do any sort of logic. 375 00:38:47.930 --> 00:38:54.090 Arjun Khurana: I think part of the reason they were making this claim here, and they're making this justification. 376 00:38:54.250 --> 00:38:59.730 Arjun Khurana: Is in order to say, A greater claim that they never actually make. 377 00:39:00.830 --> 00:39:04.439 Arjun Khurana: I believe, throughout the whole paper. And that greater claim is… 378 00:39:05.480 --> 00:39:15.880 Arjun Khurana: If you set up your statistical method right, Then, the symbolic Relations are an emergent property. 379 00:39:22.320 --> 00:39:28.810 Arjun Khurana: This here, their interpretability, is… It seems, according to them, some kind of evidence for this. 380 00:39:29.150 --> 00:39:30.529 Arjun Khurana: I would tend to agree. 381 00:39:31.160 --> 00:39:36.529 Arjun Khurana: They trained something on statistics, and what popped out were coherent internal representations. 382 00:39:37.830 --> 00:39:41.839 Arjun Khurana: And this model does seem to be doing some kind of 383 00:39:43.070 --> 00:39:49.259 Arjun Khurana: Processing down and compositional reasoning of its inputs in order to get the right outputs. 384 00:39:51.190 --> 00:39:54.559 Arjun Khurana: It is making use of the isomorphism between these two trees. 385 00:39:54.740 --> 00:40:01.240 Arjun Khurana: And it's… Distinguishing between different categorical aspects 386 00:40:01.560 --> 00:40:05.109 Arjun Khurana: Of the individual people, and the individual relations. 387 00:40:07.270 --> 00:40:10.450 Arjun Khurana: But I do think we take for granted now that this is a, like. 388 00:40:10.760 --> 00:40:14.279 Arjun Khurana: Oh yeah, obviously this is going to emerge from statistics. 389 00:40:14.570 --> 00:40:17.740 Arjun Khurana: If you give it enough data, that was highly controversial at the time. 390 00:40:21.750 --> 00:40:24.660 Jake Russin: I mean, it's still controversial to some people, isn't it? 391 00:40:24.660 --> 00:40:26.840 Arjun Khurana: Yes, it is still controversial to some people. 392 00:40:28.120 --> 00:40:29.730 Arjun Khurana: Jeff Hinton being one of them. 393 00:40:33.280 --> 00:40:35.820 Arjun Khurana: He's cold. 394 00:40:35.820 --> 00:40:45.319 Alice Rigg: total observations of, like, the Unit 1 is primarily concerned with this, or Unit 2 is primarily concerned with that. It's not wrong, right? Because of the. 395 00:40:45.320 --> 00:40:45.700 Arjun Khurana: I've noticed. 396 00:40:46.530 --> 00:40:47.510 Alice Rigg: Right? The only. 397 00:40:47.510 --> 00:40:48.469 Arjun Khurana: It's not wrong. 398 00:40:48.470 --> 00:40:51.659 Alice Rigg: It is actually… they are allowed to make that claim. 399 00:40:54.550 --> 00:40:58.380 Arjun Khurana: This is why I think it's so interesting. All of these claims that they make are… 400 00:40:58.860 --> 00:41:11.800 Arjun Khurana: in my opinion, obviously correct. Like, there's not really a good way to cast doubt on any of the things that they say in this. But the logical jumps they're making from one to the other is stuff that would be… 401 00:41:13.920 --> 00:41:19.100 Arjun Khurana: Very… like, if we were doing this Nowadays, in a… 402 00:41:19.850 --> 00:41:22.730 Arjun Khurana: Large language model, however large you want to go. 403 00:41:23.350 --> 00:41:26.369 Arjun Khurana: It would be very controversial to make each one of these leaps. 404 00:41:26.790 --> 00:41:28.990 Arjun Khurana: And your reviewer would probably flag it. 405 00:41:29.790 --> 00:41:32.199 Arjun Khurana: the reason I bring this up is not because 406 00:41:32.910 --> 00:41:38.390 Arjun Khurana: Not as a critique of any sort of methodology, but just to bring back into question 407 00:41:38.990 --> 00:41:40.670 Arjun Khurana: This was a form of reasoning. 408 00:41:41.620 --> 00:41:43.330 Arjun Khurana: This was a form of… 409 00:41:45.000 --> 00:41:50.249 Arjun Khurana: I'll say inductive reasoning that we did about how models work back in the day. 410 00:41:50.510 --> 00:41:52.210 Arjun Khurana: That we don't do anymore. 411 00:41:52.550 --> 00:41:54.090 Arjun Khurana: And it's worth considering. 412 00:41:55.460 --> 00:41:59.729 Arjun Khurana: If we should go back to it, if we shouldn't go back to it, and why. 413 00:42:00.240 --> 00:42:06.589 Arjun Khurana: Not because it'll change what we do nowadays, but because understanding why we do or don't do something like that is important. 414 00:42:08.190 --> 00:42:15.039 Jake Russin: You mean the inductive reasoning that says that, we know it's doing something interesting because we can see that it's 415 00:42:15.440 --> 00:42:18.670 Jake Russin: Because we're looking at its internal representations in there. 416 00:42:18.890 --> 00:42:19.560 Jake Russin: And they've kept. 417 00:42:19.560 --> 00:42:19.990 Arjun Khurana: Yeah. 418 00:42:19.990 --> 00:42:21.789 Jake Russin: structure or something. Yeah. 419 00:42:21.790 --> 00:42:30.480 Arjun Khurana: So the chain of logic that's actually going on here is… Unit 1 has High activation on… 420 00:42:30.800 --> 00:42:36.320 Arjun Khurana: English, low activation on… Italian. 421 00:42:37.310 --> 00:42:43.660 Arjun Khurana: Then… This implies that Unit 1 is concerned. 422 00:42:44.750 --> 00:42:55.500 Arjun Khurana: with the… is primarily Concerned with the… distinction between English and Italian. 423 00:42:56.940 --> 00:42:59.800 Arjun Khurana: This is a very simple claim, and is probably valid. 424 00:43:00.210 --> 00:43:01.800 Arjun Khurana: Then they make another claim. 425 00:43:04.450 --> 00:43:10.050 Arjun Khurana: Other units… I'm gonna put this… Somewhere else for now. 426 00:43:14.030 --> 00:43:15.409 Arjun Khurana: They make another claim. 427 00:43:15.690 --> 00:43:18.319 Arjun Khurana: Other units, e.g. 428 00:43:19.010 --> 00:43:20.430 Arjun Khurana: Unit 2. 429 00:43:22.610 --> 00:43:29.980 Arjun Khurana: Have similar activations for an English person and their Italian counterpart. 430 00:43:32.910 --> 00:43:42.610 Arjun Khurana: Therefore, Unit 2 is not concerned with Which tree just… Where in the tree? 431 00:43:44.920 --> 00:43:47.609 Arjun Khurana: This is probably a valid implication. 432 00:43:49.120 --> 00:44:01.860 Arjun Khurana: Therefore, Unit 2 is using a similar representation, or an English person, and their Italian counterpart. 433 00:44:03.100 --> 00:44:08.849 Arjun Khurana: Therefore, the network Is making use of the isomorphism. 434 00:44:12.200 --> 00:44:17.520 Arjun Khurana: To allow it to share structure and generalize. 435 00:44:17.870 --> 00:44:23.590 Arjun Khurana: These are… this is the chain of logic from… right here. 436 00:44:23.950 --> 00:44:27.440 Arjun Khurana: to… Right here. 437 00:44:33.370 --> 00:44:43.969 Arjun Khurana: this would be… if I submitted this to open review, I would get so many critiques about, this is just purely correlational evidence. We have no idea whether the rest of the model is actually using this information. We have no idea… 438 00:44:44.510 --> 00:44:56.260 Arjun Khurana: If there is other stuff going on here that we can't see beyond just the activations. And just making a claim that it is doing something, or even making a claim that it is doing something causally, does not suffice to show 439 00:44:56.490 --> 00:44:59.679 Arjun Khurana: that… The model is… 440 00:44:59.930 --> 00:45:07.370 Arjun Khurana: Using very similar representations for two inputs, or especially making use of an isomorphism to share a structure. 441 00:45:08.710 --> 00:45:11.759 Arjun Khurana: I would agree that all of these things are obviously true. 442 00:45:13.850 --> 00:45:17.270 Alice Rigg: So how do you, how do you, like, describe, like, how do you just capture… 443 00:45:17.640 --> 00:45:31.089 Alice Rigg: like, what is problematic about the language? Like, I can… I can tell that, like, there is something uncanny about it that I would not find, like, in modern day, but I can't really articulate, like, what is… what is the… what's wrong. 444 00:45:33.260 --> 00:45:41.840 Arjun Khurana: So, the… Underlying assumptions in these chains of implication here is that… 445 00:45:45.330 --> 00:45:48.600 Arjun Khurana: If we see a pattern, then that's what's going on. 446 00:45:49.900 --> 00:45:52.760 Arjun Khurana: And… it's inverse. 447 00:45:54.160 --> 00:45:55.000 Arjun Khurana: if… 448 00:45:55.850 --> 00:46:01.629 Arjun Khurana: We don't see a pattern, then the thing that would happen in that pattern is probably not going on right there. 449 00:46:04.380 --> 00:46:09.769 Arjun Khurana: Then the second is… 450 00:46:11.200 --> 00:46:14.829 Arjun Khurana: If we see a pattern, and we've made sense of that pattern, and let's for now… 451 00:46:15.330 --> 00:46:18.359 Arjun Khurana: Assume that that pattern indicates what's going on. 452 00:46:19.900 --> 00:46:24.910 Arjun Khurana: Then that pattern has something to do with representations and the network's structures. 453 00:46:25.650 --> 00:46:31.979 Arjun Khurana: Which is a valid claim to make, except for the fact that nowadays, we do not have consistent 454 00:46:33.550 --> 00:46:37.330 Arjun Khurana: We do not have rigorous nor consistent standards for what counts as 455 00:46:37.650 --> 00:46:42.290 Arjun Khurana: The model is using a representation, or what even a representation is. 456 00:46:42.500 --> 00:46:45.790 Arjun Khurana: Nor do we have the same for structure. 457 00:46:47.650 --> 00:46:52.079 Arjun Khurana: I know this because I've written 458 00:46:52.460 --> 00:47:01.800 Arjun Khurana: papers where I claim that the model has a representation for something, and then had long, long discussions with lots of people where they go, well, what does it mean for the model to have a representation of something? 459 00:47:01.980 --> 00:47:10.609 Arjun Khurana: And the answer is, we have no idea. And then I ask you, what does it mean for a person to have a representation of something? And then we have no idea still. Which is why I think it's so important for us to talk about cognition. 460 00:47:13.020 --> 00:47:15.180 Arjun Khurana: This is a very good segue. 461 00:47:15.780 --> 00:47:21.129 Arjun Khurana: But before I segue, I want to at least leave a chance for people to have burning questions or burning comments. 462 00:47:28.250 --> 00:47:32.829 Arjun Khurana: The segue is to some… questions that I had… 463 00:47:34.590 --> 00:47:36.250 Arjun Khurana: We don't have to look at Figure 5. 464 00:47:37.510 --> 00:47:39.780 Arjun Khurana: There are some questions that I had prepared. 465 00:47:42.060 --> 00:47:47.459 Arjun Khurana: about… There were some questions that I prepared and then said I was going to revise and didn't. 466 00:47:48.260 --> 00:47:49.809 Arjun Khurana: about Metalearning. 467 00:47:51.270 --> 00:47:59.360 Arjun Khurana: Which is… And specifically about Metalearning in an individual paper. David suggested that we go through specific… 468 00:48:00.030 --> 00:48:02.680 Arjun Khurana: Things that are important when reading a paper. 469 00:48:03.420 --> 00:48:07.070 Arjun Khurana: And I think if we're talking about Metalearning, here are some things that are very important. 470 00:48:08.070 --> 00:48:10.470 Arjun Khurana: What assumptions does the paper make? 471 00:48:11.420 --> 00:48:14.140 Arjun Khurana: about… Concepts. 472 00:48:14.390 --> 00:48:16.420 Arjun Khurana: or representations. 473 00:48:20.280 --> 00:48:25.449 Arjun Khurana: Then… What does the paper say counts as evidence? 474 00:48:25.800 --> 00:48:31.209 Arjun Khurana: And you can add a for something if there's something that's relevant. So for here, for structure. 475 00:48:34.750 --> 00:48:36.070 Arjun Khurana: And then lastly. 476 00:48:40.990 --> 00:48:48.779 Arjun Khurana: Informally, how does the paper Think about a model as a whole. 477 00:49:01.500 --> 00:49:04.109 Arjun Khurana: I think these are pretty important. 478 00:49:05.860 --> 00:49:09.760 Arjun Khurana: There's definitely other more… this is definitely more conceptual than Metalearning. 479 00:49:09.990 --> 00:49:10.790 Arjun Khurana: But… 480 00:49:14.610 --> 00:49:17.980 Arjun Khurana: I would like to start with… this here. 481 00:49:18.830 --> 00:49:24.429 Arjun Khurana: The paper is clearly making a lot of claims of blank counts as evidence for blank. 482 00:49:25.410 --> 00:49:30.200 Arjun Khurana: And… Without being critical. 483 00:49:33.730 --> 00:49:35.260 Arjun Khurana: non-critically. 484 00:49:36.730 --> 00:49:43.420 Arjun Khurana: Paper assumes… X counts as evidence Or why? 485 00:49:44.320 --> 00:49:46.929 Arjun Khurana: I would love to go through and grab a bunch of stuff. 486 00:49:47.180 --> 00:49:50.689 Arjun Khurana: That the paper says something counts as evidence for something else. 487 00:49:54.730 --> 00:49:57.230 Arjun Khurana: I am happy to start, but I want to invite the collective. 488 00:50:01.030 --> 00:50:08.090 Alice Rigg: I think in absence of people volunteering to say stuff, you should call upon people who have not said anything. 489 00:50:09.400 --> 00:50:13.780 Alice Rigg: I know it's uncomfortable for them, but on average, in the long term, it is better for them. 490 00:50:16.300 --> 00:50:17.170 Arjun Khurana: Sure. 491 00:50:23.940 --> 00:50:31.839 Arjun Khurana: Then… I'm gonna first call on Calvin, because Oliver has talked you up. 492 00:50:33.660 --> 00:50:38.830 Calvin Higgins: Oh my god, I haven't done any machine learning or any of this stuff for years, but I… 493 00:50:40.530 --> 00:50:44.290 Calvin Higgins: I don't know, I was just thinking… I don't have a… 494 00:50:44.590 --> 00:50:51.719 Calvin Higgins: I really don't have many thoughts about this stuff besides… I'm not an interperson, and… 495 00:50:52.180 --> 00:50:54.170 Calvin Higgins: You know, to me, it's like… 496 00:50:54.680 --> 00:51:03.769 Calvin Higgins: the network is so tiny, like, why can't you just, like, look at all the weights and say, like, this is the explanation? Like, it is doing that, you know? 497 00:51:03.930 --> 00:51:09.830 Calvin Higgins: To me, it's just… it… I don't really understand what you guys are saying when you say, you know. 498 00:51:10.220 --> 00:51:15.350 Calvin Higgins: there's issues in this chain of reasoning. Because to me, it's like. 499 00:51:16.170 --> 00:51:25.679 Calvin Higgins: they're basically… it's so small, it's like you can just print out, basically, the code, right? Yes. And you just look at the code, and you say, this is what it does, this is how it does it. 500 00:51:25.680 --> 00:51:38.310 Calvin Higgins: Right? And to me, you know, I understand why it doesn't work when you can't print out the code anymore, right? But, like, if you can print out the code, and you can look at it, and you can understand it, to me, like, I just… I just don't see what's… 501 00:51:38.330 --> 00:51:40.949 Calvin Higgins: What's wrong with what they're saying? 502 00:51:41.020 --> 00:51:43.549 Calvin Higgins: In this, like, constrained contents. 503 00:51:45.030 --> 00:51:49.819 Arjun Khurana: This is very fair, and I should probably be a little bit more precise with what I'm talking about. 504 00:51:50.520 --> 00:51:57.840 Arjun Khurana: There are two things that, while I don't personally take issue with it, the cognitive philosophy community would take issue with it. 505 00:51:59.130 --> 00:52:10.570 Arjun Khurana: And the Mechinturp community would take issue. Number one is a practical consideration, which is exactly what you are saying. Yes, it's okay to do this methodology for a small model where you can print out all the weights. 506 00:52:11.220 --> 00:52:12.120 Arjun Khurana: But… 507 00:52:13.700 --> 00:52:20.119 Arjun Khurana: At the same time, even if you can print out all the weights, you can't look at what's going on from every angle all at the same time. 508 00:52:21.420 --> 00:52:27.220 Arjun Khurana: And so, if we're looking at something from one angle, such as Activations. 509 00:52:27.790 --> 00:52:29.830 Arjun Khurana: And we see a bunch of patterns. 510 00:52:30.210 --> 00:52:34.089 Arjun Khurana: And we make conclusions from that, specifically conclusions of exclusion. 511 00:52:35.160 --> 00:52:43.740 Arjun Khurana: we are missing what's going on in another angle, if there is something going on in another angle. Now, I don't think in this very simple network anything's happening from another angle. 512 00:52:45.290 --> 00:52:46.559 Arjun Khurana: But you gotta be thorough. 513 00:52:46.860 --> 00:52:52.010 Eric Todd: I was gonna say, I think, like, just looking at weights can… 514 00:52:52.510 --> 00:52:57.279 Eric Todd: It'll… it can be hard to tell, like, interactions between things. 515 00:52:57.450 --> 00:53:06.539 Eric Todd: like, you can see, like, simple features, but then if it's, like, the interaction between things as you chain it through, I think that's harder to see. 516 00:53:06.980 --> 00:53:09.309 Eric Todd: Just looking at weights. 517 00:53:10.510 --> 00:53:12.359 Calvin Higgins: So I guess the idea is, like. 518 00:53:12.550 --> 00:53:17.669 Calvin Higgins: in this… in this first layer, you're good, right? You can kind of say, okay. 519 00:53:17.860 --> 00:53:24.160 Calvin Higgins: you know, from here to here, like, these weights are doing X, but once you get into the deeper layers. 520 00:53:24.370 --> 00:53:27.520 Calvin Higgins: Now, the weight… now… now you have, like. 521 00:53:27.650 --> 00:53:35.960 Calvin Higgins: you know, this weighted sum is piping into this other weighted sum, and all this other stuff is going on. You can't really say as much anymore about the meaning of the weights. 522 00:53:36.130 --> 00:53:37.599 Calvin Higgins: But then I wonder, like. 523 00:53:39.980 --> 00:53:47.419 Eric Todd: like, some of these still seem like they have reasonable interpretations. Like, they… they say, like, one is about, like, the… 524 00:53:47.770 --> 00:53:48.390 Arjun Khurana: Yeah. 525 00:53:48.740 --> 00:53:53.089 Eric Todd: the generation that someone is in, and you can… you can… 526 00:53:53.090 --> 00:53:53.930 Arjun Khurana: Unit 2. 527 00:53:54.210 --> 00:53:56.459 Eric Todd: Yeah, you can kind of tell, like… 528 00:53:57.350 --> 00:54:01.099 Eric Todd: What are, like, the dark black ones? Like, Colin and Charlotte. 529 00:54:01.100 --> 00:54:05.020 Arjun Khurana: Charlotte, call in Charlotte, Victoria, And… 530 00:54:05.020 --> 00:54:08.999 Eric Todd: Yeah, so if you, like, go over to the tree, and you, like, look at in their data… 531 00:54:09.000 --> 00:54:10.139 Arjun Khurana: Charlene, Charlotte. 532 00:54:10.140 --> 00:54:12.400 Eric Todd: Colin and Charlotte are, like, the bottom. 533 00:54:12.400 --> 00:54:13.230 Arjun Khurana: And it's… 534 00:54:13.230 --> 00:54:15.160 Eric Todd: Which, like, yeah, like… 535 00:54:15.160 --> 00:54:16.960 Arjun Khurana: corresponding ones in the Italians. 536 00:54:16.960 --> 00:54:19.880 Eric Todd: Seems pretty reasonable interpretation to me. 537 00:54:21.240 --> 00:54:31.749 Eric Todd: Maybe, maybe, maybe Arjun is saying, like, if… yeah, once you scale, this sort of thing doesn't work, but I… I don't know, in my opinion, for this small network, like. 538 00:54:32.500 --> 00:54:34.959 Eric Todd: I'm pretty okay with a lot of these. 539 00:54:35.340 --> 00:54:37.689 Eric Todd: Probably all of the… most of these, 540 00:54:38.280 --> 00:54:39.890 Eric Todd: You know, look at it and… 541 00:54:40.960 --> 00:54:44.500 Eric Todd: What you see is what you get kind of interpretation of things. 542 00:54:45.190 --> 00:54:48.489 Arjun Khurana: So this is where the second objection becomes, I think, much more interesting. 543 00:54:49.080 --> 00:54:53.500 Arjun Khurana: Which is… on this sentence. 544 00:54:59.320 --> 00:55:09.920 Arjun Khurana: The idea of… A network representing someone internally, or even a unit representing someone internally, is something that 545 00:55:10.600 --> 00:55:14.400 Arjun Khurana: A whole family of cognitive philosophers immediately took issue with. 546 00:55:15.600 --> 00:55:17.070 Arjun Khurana: And the idea that… 547 00:55:20.380 --> 00:55:22.979 Arjun Khurana: this turns into sort of the Chinese room issue. 548 00:55:23.780 --> 00:55:29.920 Arjun Khurana: But… One of the claims that they make, is… 549 00:55:35.790 --> 00:55:40.980 Arjun Khurana: component, Spitting out similar values. 550 00:55:41.550 --> 00:55:42.660 Arjun Khurana: for… 551 00:55:46.880 --> 00:55:49.560 Arjun Khurana: X category versus Y category. 552 00:55:54.540 --> 00:55:57.269 Arjun Khurana: And diff, or by category. 553 00:55:58.670 --> 00:56:07.380 Arjun Khurana: is evidence of that component Has similar representations. 554 00:56:07.800 --> 00:56:09.380 Arjun Khurana: or X's. 555 00:56:12.340 --> 00:56:14.240 Arjun Khurana: as representations for X's. 556 00:56:15.070 --> 00:56:17.160 Arjun Khurana: And representations for Y's. 557 00:56:19.580 --> 00:56:24.400 Arjun Khurana: The example here being… Unit 1 spits out similar values. 558 00:56:25.330 --> 00:56:26.680 Arjun Khurana: Or Englishes. 559 00:56:30.150 --> 00:56:33.299 Arjun Khurana: Spits out X for Englishes and Y for Italians. 560 00:56:35.750 --> 00:56:43.230 Arjun Khurana: implies… Unit 1 represents Englishes as X and Italians. 561 00:56:45.330 --> 00:56:46.389 Arjun Khurana: as Y. 562 00:56:48.170 --> 00:56:52.579 Arjun Khurana: Or, on the other hand, Unit 2 spits out 563 00:56:54.800 --> 00:57:01.160 Arjun Khurana: X for English, and X for the… Italian equivalent. 564 00:57:02.710 --> 00:57:07.759 Arjun Khurana: X for youngest English son, let's say, and X for the Italian equivalent. 565 00:57:07.940 --> 00:57:13.170 Arjun Khurana: implies Unit 2, Has a similar representation. 566 00:57:14.260 --> 00:57:22.350 Arjun Khurana: for… youngest English son, And… Hora's Italian son. 567 00:57:24.260 --> 00:57:29.169 Arjun Khurana: and all other… Elements in the isomorphism. 568 00:57:32.010 --> 00:57:39.250 Arjun Khurana: I think this is perfectly true, and speaking informally, it is absolutely perfectly true. And then all of the philosophers went, what does it mean to… 569 00:57:39.760 --> 00:57:41.600 Arjun Khurana: Have a similar representation. 570 00:57:42.040 --> 00:57:45.649 Arjun Khurana: Or, to represent something as something. 571 00:57:47.880 --> 00:57:55.840 Arjun Khurana: This sounds like a stupid question, especially for mechatirp researchers who are like, obviously the only thing Unit 1 is doing is representing that thing. 572 00:57:56.800 --> 00:58:00.050 Arjun Khurana: But it is at least worth a bit of a ponder of what does it mean 573 00:58:00.620 --> 00:58:06.229 Arjun Khurana: What is the bridge between represent… what is the boundary between it represents this and it doesn't represent this? 574 00:58:07.460 --> 00:58:15.419 Arjun Khurana: Because… Jerry Thodor might look at… This chart right here. 575 00:58:17.730 --> 00:58:22.850 Arjun Khurana: And go, hey, What's with this? 576 00:58:23.350 --> 00:58:30.009 Arjun Khurana: What's with this guy? This means it doesn't have similar representations. The representations are opposite. One of them is black, and one of them is white. 577 00:58:36.890 --> 00:58:38.709 Arjun Khurana: Now, personally, I would side with 578 00:58:39.550 --> 00:58:47.639 Arjun Khurana: Rimblehart and Hinton on this, I think that they're doing some perfectly fine logic. But it is important to note that this is how they're determining something counts as evidence. 579 00:58:56.480 --> 00:59:01.210 Arjun Khurana: An invitation for discussion on this, or other examples of something counting as evidence for something. 580 00:59:03.600 --> 00:59:12.720 Eric Todd: I just… just want to flag, we have, like, 3 minutes until the hour, if we want to… Oh, shoot! Yeah, it's gone by really fast. 581 00:59:12.860 --> 00:59:16.520 Eric Todd: So, yeah, if anybody has, like, any last-minute, 582 00:59:17.330 --> 00:59:20.029 Eric Todd: comments, or… I don't know, I… 583 00:59:20.270 --> 00:59:21.890 Eric Todd: Things they want to say. 584 00:59:22.220 --> 00:59:23.630 Eric Todd: Feel free. 585 00:59:31.380 --> 00:59:35.280 Jake Russin: Is the plan to read Fodor as well? Are you guys gonna read some photos? 586 00:59:35.280 --> 00:59:39.850 Arjun Khurana: We'll read photo eventually, but not for… Not for next week. 587 00:59:40.570 --> 00:59:41.230 Jake Russin: Gotcha. 588 00:59:41.800 --> 00:59:50.399 Eric Todd: Yeah, I think… so, yeah, if you guys have suggestions of papers you feel like would be good to read, feel free to… 589 00:59:50.600 --> 00:59:56.219 Eric Todd: to send them our way. I think nec… the next… for the next time, we were planning on reading 590 00:59:56.480 --> 01:00:06.629 Eric Todd: like, maybe alternating between, like, some more foundational work, and then some, like, recent work. So next week, we were thinking… or not next week, but… 591 01:00:06.880 --> 01:00:13.949 Eric Todd: the next time, in two weeks, thinking about reading the… the JLens paper, sort of with the… 592 01:00:14.210 --> 01:00:18.839 Eric Todd: With the lens of, like, thinking about what that can tell… what that tells us about, like. 593 01:00:19.030 --> 01:00:21.309 Eric Todd: Metalearning and Cognition. 594 01:00:21.710 --> 01:00:28.690 Eric Todd: I know there's, like, a little bit of controversy surrounding, like, their… their tying their findings to some of the… 595 01:00:29.370 --> 01:00:33.689 Eric Todd: What do you… 596 01:00:33.690 --> 01:00:34.670 Jake Russin: Oh, that's like… 597 01:00:35.660 --> 01:00:36.350 Eric Todd: Yeah. 598 01:00:37.040 --> 01:00:40.349 Eric Todd: But yeah, I think it'd be an interesting paper to discuss. 599 01:00:44.510 --> 01:00:52.310 Emma Bortz: I just posted the link for the, next reading group registration, but generally they'll be at the same Zoom link. 600 01:00:52.510 --> 01:00:56.080 Emma Bortz: And every 2 weeks. 601 01:00:58.780 --> 01:01:03.650 Eric Todd: Is there a way to make them recurring, Emma? I don't know, we can figure it out later, but… 602 01:01:03.650 --> 01:01:11.650 Emma Bortz: Yeah, I haven't found a way. Luma doesn't allow that, but, we could also do… 603 01:01:12.040 --> 01:01:15.429 Emma Bortz: a Google Calendar, or something like that, as well. 604 01:01:16.860 --> 01:01:17.220 Arjun Khurana: We'll figure it. 605 01:01:17.220 --> 01:01:22.240 Alice Rigg: Is there, like, a… a targeted, like… 606 01:01:22.560 --> 01:01:25.430 Alice Rigg: era for the papers. Like, this one's pretty old. 607 01:01:25.780 --> 01:01:29.680 Alice Rigg: Like, if I'm supposed to recommend a paper, like. 608 01:01:29.680 --> 01:01:31.919 Arjun Khurana: Anything between 1958 and now. 609 01:01:33.270 --> 01:01:35.319 Alice Rigg: Okay, but, like, are you trying to uniformly sample? 610 01:01:35.700 --> 01:01:39.940 Alice Rigg: Based on the year, or the… I'll take a camera. 611 01:01:39.940 --> 01:01:45.780 Eric Todd: Not necessarily. I think, yeah, if you feel like it'd be a good foundation for, like, understanding 612 01:01:46.420 --> 01:01:54.379 Eric Todd: Yeah, something about, like, how… how we can think about learning to learn, or, like, what is thinking 613 01:01:54.690 --> 01:01:59.109 Eric Todd: You know, whether that's… Older papers, or recent papers. 614 01:01:59.420 --> 01:02:08.350 Eric Todd: We have some other papers that we've sort of brainstormed, but if there's ones you guys are, like, feel like would be great to read, we can also 615 01:02:08.550 --> 01:02:10.259 Eric Todd: Trying to include those as well. 616 01:02:13.700 --> 01:02:16.599 Eric Todd: But yeah, thanks everyone for coming. I think, 617 01:02:17.250 --> 01:02:19.719 Eric Todd: I think it was a good, a good first paper. 618 01:02:21.950 --> 01:02:28.660 Arjun Khurana: Everyone, if you're interested, check out the notebook. That… Was linked in the chat. 619 01:02:31.470 --> 01:02:40.510 Arjun Khurana: If you have problems with the notebook, or you find cool things with Jeff Hinton's network, or your version of Jeff Hinton's network that you trained, throw them in the reading group channel in the Discord. 620 01:02:45.570 --> 01:02:47.219 Arjun Khurana: I don't think I have anything else. 621 01:02:47.520 --> 01:02:48.290 Emma Bortz: And… 622 01:02:48.290 --> 01:02:49.160 Arjun Khurana: about… 623 01:02:49.370 --> 01:02:51.970 Emma Bortz: annotated paper there as well, right? 624 01:02:51.970 --> 01:02:54.850 Arjun Khurana: Yes, I will post my annotated paper in the Discord. 625 01:02:55.830 --> 01:03:03.900 Arjun Khurana: Let me stop this as well. Yeah. 626 01:03:04.280 --> 01:03:07.820 Arjun Khurana: Try to come next week prepared with your, 627 01:03:08.510 --> 01:03:17.210 Arjun Khurana: I'm gonna say, come next week prepared with your most, critical, pedantic, takes… 628 01:03:17.490 --> 01:03:24.089 Arjun Khurana: on JLens. This doesn't actually count as thinking about this, this doesn't count as global workspace. 629 01:03:25.320 --> 01:03:29.529 Arjun Khurana: Because I think that is the most fun way to go through learning about cognition. 630 01:03:35.610 --> 01:03:36.830 Emma Bortz: Thank you! 631 01:03:37.600 --> 01:03:38.469 Emma Bortz: Thank you for watching. 632 01:03:40.090 --> 01:03:41.530 Arjun Khurana: Great comment. Good comment. 633 01:03:42.850 --> 01:03:46.319 Arjun Khurana: I will then see you all, hopefully, in 2 weeks. 634 01:03:48.440 --> 01:03:49.910 Arjun Khurana: There's nothing else for the good of the order? 635 01:03:51.180 --> 01:03:51.890 Jake Russin: Thanks. 636 01:03:51.890 --> 01:03:52.710 Emma Bortz: Bye. 637 01:03:59.020 --> 01:04:00.300 Emma Bortz: Awesome! 638 01:04:01.410 --> 01:04:06.100 Emma Bortz: Thanks for that, that was awesome. See you later.