David Bau: Close the door here. Yeah, you can leave it ajar. So for the the latecomers, the latecomers. Yep.
David Bau: so I'm happy to
David Bau: to introduce Alex Loftus. Sure, my Phd student. And he's going to give a perspective on interpretability by using
David Bau: some analogies to to biological discovery. Yeah.
David Bau: So right? So I decided to give this talk on biology to an interpretability lab. Which is a pretty weird thing to do. So you know, why am I talking about biology to a bunch of interpretability people?
David Bau: well, there's been kind of this conversation in the Zeitgeist recently of interpretability. And the argument kind of goes like interpretability is really similar to biology in this sense that
David Bau: biological systems are these natural things that grow kind of on their own and deep learning systems are also these things that sort of grow on their own right. You set up the architecture, and then you set up the training data. And then you just like, let back propagation do its thing.
David Bau: And this has been, you know, talked about a whole bunch. You know. David has brought up that interpretability, that the role of interpretability is sort of analogous to the role of biochemistry. So if we can get
David Bau: closer to this sort of mechanistic understanding of how these large language models work and and how these huge systems work. Then we unlock all these doors that were unlocked analogously with biochemistry.
David Bau: and you know a bunch of people have been, you know, kind of saying the same thing. Chris Ola loves saying this, Dario Amade and his machines of love and grace. Essay said this, and so I figured that I kind of agree with this. I have a biology background from undergrad, and I worked in a wet lab. And I worked in a drug discovery company for almost 2 years doing machine learning for drug discovery. And they're actually
David Bau: there are a lot of analogies that I've noticed, and I do think that we, as interpretability researchers, can gain a lot from understanding a little bit about what the biologists have done in the last 100 5,200 years.
David Bau: So there's kind of 2 main things that I want us to get out of sort of a discussion about biology. The 1st is inspiration. Right? So what does it look like for interpretability as a field to be successful in the next? You know.
David Bau: 5, 10 years, right? 15 years, whatever right? And then the next is direction. So if we have this sort of
David Bau: understanding a little bit of what the biologists did. We can sort of envision what the equivalent of that might look like in interpretability. And so the way that I want to do this is to zoom in to a really simple research question that the biologists have had over the last 40 years, and that they've managed to solve
David Bau: quite successfully. That has led to a lot of drug downstream drug applications and a lot of sort of interesting use cases. And we understand the entire mechanism, and it's provided a lot of value to people. And so that mechanism is
David Bau: when the human brain, when when 2 neurons in the human brain fire concurrently, the connection between those 2 neurons is strengthened. And so
David Bau: how does that work? And at what level of detail do we understand how that works? This is a question that, like this phenomenon was 1st noticed in the 19 fifties, possibly even earlier. And biologists were really curious about this for a long time. It's sort of core to the entire field of memory and learning and
David Bau: and in biology this mechanism is called long-term potentiation, and the whole, every part of how this works mechanistically is pretty well understood at this point. So
David Bau: I'm just going to give like a brief overview, and then we'll we'll zoom into one of the kind of central character, the central protein that's involved in allowing this to happen, and then we'll look a little bit at how the process of understanding this mechanism occurred. So this is a synapse.
David Bau: So these are 2 neurons. So if you zoom in to, you can think of like 2 trees next to each other, and the trees, and each tree is a neuron, and the trees have these branches that are all intertangled with each other, and if you zoom in to one of the branches, you can see a synapse, and the synapse are sort of the points of connection between the 2 neurons
David Bau: and the way that they interact with each other is that the presynaptic neuron, the 1st neuron is sending a signal to the second neuron. That's the presynaptic membrane. And so
David Bau: the way it works is when the 1st neuron fires, it sends these chemicals over the synapse. That's these orange things and those chemicals land in these receptors in the second neuron, once the chemicals land. In these receptors, these are called ampa receptors.
David Bau: They're kind of like plug gates that open. So these, these chemicals kind of like dock into these receptors, and that opens these doors, and that allows chemical charge to kind of flow through this membrane from this sort of like space outside the membrane
David Bau: into it, and then that chemical movement causes all kinds of downstream effects to happen. But one of those effects is that this second neuron is now more likely to fire. And so the whole process of signal propagation in the brain occurs through sort of like network effects of this right. One neuron sends these chemicals that causes electrical changes in the second neuron
David Bau: that causes the second neuron to be more likely to fire, and then it sends signals to its downstream neurons and so on. And so that's how that's how you have. Like the signal propagation. I have a very stupid question about your diagram. So yeah, so can you point to which which is the 1st neuron, which is the second neuron, and where it's not in the neuron. Yeah. So this is the 1st neuron. It's the presynaptic membrane. This is the second neuron. It's the postsynaptic membrane. This is super zoomed into just one of the synapses. And then what's not in a neuron, not in the neuron is all of this space surrounding it.
David Bau: And how come you have like a double line for the second neuron there. Oh, this double line. Yes. Oh, this is the. This is the membrane. So this is this. Sure? Yeah, it's kind of like the skin of the second neuron. So it's this, it's this tissue that surrounds the second neuron and and like blocks, all of the electrical charge in this like sort of background space. And you just didn't bother drawing a double line around the other neuron.
David Bau: Yeah, I didn't bother it, because this is this is a bit of a wacky zooming situation. So so the presynaptic membrane is actually like way bigger than that but I just I didn't. It's not drawn to scale just so that we could, you know, show that show that this is sort of like a a membrane that like kind of is is poking out a little bit. Just to like get it.
David Bau: Some idea of the shape of it. Yeah, is this a I'm sorry? Is this a continuous or discrete signal like, is it like
David Bau: it like.
David Bau: Oh, it's discrete. So so that's like an interesting thing about neurons is they? When they send a signal, it's like a short burst. And it's like it's it's in like a like a about 2 ms is how long the signal lasts, and then it it ends. And so like, you can imagine. Like, if this is sending a lot of signals right? It's like it's it's like
David Bau: much more likely to cause this second neuron to fire, because, you know, you send a signal. It like sends some charge in, but that's not enough, and the charge starts like depleting, and then it sends another signal, and the charge gets higher and higher and higher, until eventually like this second neuron, like
David Bau: fires. But it's like a whole. It's like a discrete process.
David Bau: Is this statement. Neurons that fire together wire together indicative of how circuits are created by one neuron firing and then downstream, another one firing. And then so this, that that statement is exactly what like the mechanism by which that happens is what we're exploring right now. So so the the like, the wire together is actually exactly what I'm about to show happening. So like what is happening when it when it's wired together. So I'll
David Bau: give you like a sneak preview what it actually means for them to wire together. More is more of these receptors are going to appear on this membrane. So so, because, remember, like these chemicals bind like attached to these things, these are the floodgates. When when these things are attached, the floodgates open. So if there's more floodgates, then like more water, can get through the water being the electro electrochemical charge.
David Bau: And so so, you know, you can imagine, like a giant like ocean. And there's like one little like pipeline for water, and so like only a little bit of water gets in, but if there's like a whole bunch of pipelines for water, then, like a lot more can get in. And so those those, the 2 areas are much more tightly bound together. If that makes sense.
David Bau: Well, say, I'm still confused where? The because you mentioned when introducing this figure that
David Bau: 2 neurons firing at the same time, and I only see 2 neurons here. One fires, the other one sort of receives. Oh, yeah. So
David Bau: So that's a little bit of a subtle point. When this neuron fires a lot at this second neuron, this second neuron is more likely to fire as well, because the second neuron causes there to be more charge in this and sorry the 1st neuron causes there to be more charge in the second neuron. And so the second neuron is like more likely to have enough charge to send its own signal.
David Bau: But there's a causal like it's causal. The 1st neuron causes the second one to charge the second. One doesn't cause the 1st one to charge.
David Bau: The another aspect is like somewhere else. In the second neuron. There might be like a signal back to the 1st one, right? And then that might cause the 1st one to get more input from the second one, right? So
David Bau: but yeah, we're like a little bit in the anyway. But so that's the process right?
David Bau: so central to this process. Does anyone else have have any other questions?
David Bau: Okay,
David Bau: So so
David Bau: there's 2 types of things that can happen when one neuron sends a signal to another one right? The 1st type of thing is these floodgates open. And the second type of thing is these little guys can cause like chemical effects. These are called Nmda receptors. And so the process by which this wiring happens is a lot to do with these second guys.
David Bau: And so there's this protein that we're going to look at that this is like a little bit more complicated version of the 1st diagram. When this protein is attached to these are the floodgates, these ampereceptors. Sorry? Yeah. When this protein is attached
David Bau: to these Nmda receptors, it basically grabs ampa receptors and like attaches them to the to the membrane. So this is a protein that, like it attaches to it, attaches to these Nmda receptors. It grabs ampa receptors, and it like places them on the membrane to like cause more firing.
David Bau: And so
David Bau: And that's pretty much the mechanism by which by which this like wiring happens, you have a signal that gets sent from the 1st neuron to the second neuron. The second neuron receives that signal. When the second neuron has this change in its charge state, this protein gets activated
David Bau: the protein attaches to this receptor, and then it starts grabbing new floodgates to like, put onto the membrane
David Bau: right? And so you know, and then and then the more signals get sent, the more floodgates there are, and and that's sort of the mechanism by which this this, this wiring happens.
David Bau: And so this protein it's called cam k, 2 has 2 kind of roles, so it strengthens, synapses by adding these floodgates, and it's also sort of a memory mechanism. So you have this firing that happens in millisecond time scales. And then this protein kind of stays there, and it keeps adding more floodgates over the course of like minutes to hours. And so you have this like long term effect of
David Bau: of like more and more wiring happens, not just because of like, not just within the time span of like a single signal, but over the course of like you know a bit longer.
David Bau: And so
David Bau: you know, it's this. there's this mechanism by which this memory stuff happens. It basically gets. I guess I won't. I won't go this deeply into the into the that, deeply into the details by which that happens. But but there's basically this this other protein that attaches to it. It. It It
David Bau: opens up once the other protein is attached, and and it just kind of stays open for quite a while, and and and when it's open it's it's it like binds and like puts the floodgates in
David Bau: so so we so like the biologists understand this like pretty pretty deeply. Right? You know, we can track these. We can watch these proteins move around. We can see them attached to these other proteins. We can understand this whole mechanism right? And we can. We can watch as more of these receptors get put on the membrane. And then we can watch, you know, learning
David Bau: more learning, like more signal propagate through these neurons. And so how did we figure out? How did the biologists figure out how that works? What was the process by which they understood this mechanism?
David Bau: So
David Bau: it's a lot of decades of research. This protein was 1st discovered in the early eighties, and you know it was really like 2,009. By the time this was sort of fully figured out how this worked. So you know, I think people in deep learning like to think on these sort of short time scales right? Like, you know, everything has to
David Bau: be figured out in the next like couple of years. But you know they they it took them like 30 years to to like get at this sort of simple mechanism.
David Bau: so you know this, this, the protein that causes this to happen was 1st found in the early eighties. And there's 2 sort of methods by which it was isolated. So the 1st is, you can take the protein, and you can put it in this little tube and spin it around.
David Bau: and you can take like a brain soup and put it in this tube and spin it around. And that sort of separates all of the proteins in this brain soup by size. And so then you can grab all of the proteins that are of an equivalent size and put them in this column, and that separates them by electrical charge. And so you can isolate like individual proteins by
David Bau: by sort of like running through this process.
David Bau: and so you know this, this, when when
David Bau: when when scientists 1st started doing this, it sort of started as this process of like exploration, it was like, let's just take some mice. Let's like grab their brains. Let's grind the brains up, turn them into soup, and then let's like
David Bau: find all the proteins that are in that soup.
David Bau: And we're not actually looking for like any particular protein. We're just like kind of looking around at, like all the proteins. And we're like. And now they're all separated because we have these 2 methods to separate them, and like what looks interesting out of like in out of all the proteins in this brain soup. And so this one looks kind of interesting because it was just like
David Bau: there was a lot of it there, right? It was 4% of all the proteins in the brain soup. And they were like, maybe we should just like, look at that, because it makes up like a good proportion of the soup, right and
David Bau: And then there were these methods in the late eighties that where you can, they figured out how to genetically modify mice. And so they were like, okay, we like this, protein looks interesting. What happens if we like make some mice that that like can't have this protein like, do they just die or like, do they? What happens to them? Right? So it's like these really basic questions. And no one had any idea what it did. Yet, right? This is like 1986, 1987, kind of like that. Right?
David Bau: So the pro, the mice, it turns out with this protein. That's 4% of their brain soup like
David Bau: could give birth. The mice could grow right. There's all these normal mouse things that could still happen. But the mice can't actually learn anything. They don't learn skills. They don't learn to avoid harmful stimulus.
David Bau: They literally can't learn at all. And so all of the people who are interested in learning and memory were like holy shit. There's this counterfactual where mice can't learn anything if they don't have this protein. So this protein must be involved in how learning works.
David Bau: And so everyone kind of like all like started doing research on this protein, because just because of this sort of like really interesting thing that was like this kind of smells interesting because of this whole mouse thing.
David Bau: And so they started getting deeper into the mechanism of how this protein does its whole memory thing. I won't really get into the details of this, but one thing that they could do to study this mechanism was they could put radioactive signals onto these molecules, and they could watch the molecules like move around.
David Bau: And then they could. They could discover stuff through just tracking these these radioactive molecules right? So you know. Say, say, we want to know if one of these molecules binds to this protein. We can put the the protein in a jar with these molecules, and then we can separate them again. And then we can see if any radioactivity is in the jar with only the protein, because if it is, then then they've like clamped together, right?
David Bau: So there's all of these techniques that were getting developed over the course of the 20th century, that all were sort of different views on like different ways of understanding, the same mechanism and sort of different microscopes that they could use to to figure stuff out
David Bau: And so really careful experimental design was another part of this right. They repeated this test for the separation under a bunch of different conditions, to isolate exactly which exactly which which proteins needed to be together for this binding to happen.
David Bau: And and it was just a lot of like sort of very careful ongoing research. That sort of looked like this that eventually led to figuring out how this, how this mechanism worked. And so, you know, by 2,009 methods in biology had sort of progressed to the point where you could actually see this stuff happen, so that I think that was really cool for a lot of biologists, because they had sort of
David Bau: been flying blind for the last 30 years. And then in 2,009. There were these techniques that started appearing where you can make proteins glow and microscopes had gotten a lot more powerful. And so they could actually like watch in real time, as these floodgates sort of got attached to this to this membrane.
David Bau: you know. And so in 1999 is when they figured out that this protein binds to the to these receptors on the membrane, and that it stays there, and, like grabs the floodgates in 2,000 was when they discovered that like not only does it stay there, but it like, adds, adds the floodgates.
David Bau: and then, in 2,009, we could sort of watch this happening in real time, and so like a lot of the sort of experimental hypotheses were like, sort of confirmed by just seeing it in action.
David Bau: And so so how? How did biologists like figure this out? So so it started with this observation. Which is that? If you
David Bau: turn mouse brains into soup, there's a lot of this protein. So it might be interesting. And then there is this hint of a research smell. So if if mice don't have this protein, the mice can't learn anything. And so then you have this general sense that, like maybe this protein is involved in learning and memory. And then there is just this, like
David Bau: ton of researchers piling on to this mechanism, and a lot of buildup of different methods and different microscopes to look at the same thing from different angles.
David Bau: So you have these tools to isolate the objects of study. So you can take the proteins out of the brain soup and look at them in separate test tubes, and so you can sort of like see them on their own, instead of like being meshed up with everything else.
David Bau: You have all these tools now to discover mechanisms. So you can put these radioactive particles onto molecules. And then you can track where that radioactivity is. So you can sort of see where they're going. And then, with careful experimental design, you can understand mechanisms through doing that
David Bau: and then we have these tools to get a causality. So we knew that this protein caused learning and memory to disappear because we had this counterfactual of creating, of being able to create mice without the protein. And so they were able to get a causality rather than just correlation through these methods that caused this sort of forcing function to happen.
David Bau: And then we had these tools to zoom in more. So we could get exact protein structures by the early mid 2 thousands. That's like, why Alphafold exists. For instance, we can watch proteins do stuff in real time through more zooming in.
David Bau: And so you know. And so it was. It was sort of this math. Massive methods build up research, smell and careful experimental design over a very long time period that let the biologists get to this point. So what equivalence are there in interpretability to this? So
David Bau: so for for what we're doing, one of the things that we're doing in the lab is, Josh and I are doing this attendum projects, and that's that's sort of like isolating the objects of study. So so you know. If you if you want to understand
David Bau: what an attention head does. And you want to understand the classes of attention heads, you need to be able to isolate the properties of those attention heads. And so, if you can take a bunch of different models and put them all into the same embedding space. Then, you know, then maybe you can start to get it at general questions of function.
David Bau: And then how do you discover mechanisms? Well, the lab is getting really good at causal questions. Right? you know, we have causal mediation analysis. We can get a counterfactuals really easily. And and then how do we zoom in? Well, steven is doing what do if you if you
David Bau: If you understand what influence functions are doing, then? you can sort of you get. You get extra
David Bau: you you can. You can sort of bench. You can benchmark a class of of functions. So
David Bau: anyway, so big picture takeaways for interpretability. we're at like a really big advantage compared to the biologists. So we have
David Bau: this iteration time advantage. So the biologists have all these problems in in wet lab. So they. It takes them weeks to do anything because they have to wait for cells to grow. There's all these problems with noise. Right cells often bind to other cells. And there's these, you know, viruses get in really easily. And it's kind of annoying to set up the test tubes.
David Bau: We don't have any of those problems. We also have tools that let us see the systems in a lot more detail, and we don't have any noise in our measurement tools compared to the biologists, or at least a lot less noise.
David Bau: And the progress for the biologists happened from stacking complementary views on the same question. So they use these radio labels to track radioactivity. They have these counterfactuals that they got with these knockout mice. They use these fluorescent dyes to be able to see what's happening in real time. And they have these assays that separate out the different objects of study.
David Bau: And so,
David Bau: yeah. And then. And so our equivalents right now are, we have causal tracing to to sort of tease apart causality. We have probing methods. We have circuit tracing you know, and we have representation learning, you know. So we we have this sort of like, equivalent and much more rapid buildup of tools. So
David Bau: yeah, yeah, that's pretty much my talk cool.
David Bau: I love the I love the radio labels. You know. Example, you know. What's what. What do you think is an analogy to, you know.
David Bau: you know, sticking sticking, sticking a radioactive
David Bau: you know, atom in a molecule. Yeah, I don't know.
David Bau: I guess I guess that would be like
David Bau: it. It's sort of like tracking information flow. So it it'd be kind of like like the causal graphs where you're where you're where you're
David Bau: you know, like, like, you know, the anthropic blog post where they where they're sort of tracking features as they like move through that causal graph that that seems like the closest analogy that we have to that Chris had an answer. Chris, what did you say I didn't see your comment.
Chris Wendler: Bye.
David Bau: Bye, bye.
Chris Wendler: Only have more questions.
Chris Wendler: I also want this radio. I also want radio labels. And
Chris Wendler: I think, like one challenge that we have, and that the biologists maybe have less is that we don't have like this base layer of reality.
Chris Wendler: or like, we don't have atoms. We don't have you know proteins, or something like.
David Bau: Yeah, right? It's hard to know what the objects are for us compared to the biologist where it was like, kind of obvious.
David Bau: That's true. Yeah, because it all mixes together like numerically, yeah, yeah, I think that's that's like a base challenge that we have is like is like this separation problem of like, you know, with the biologists, it's like you put them in columns. And then you have these like really obvious things that you can look at. And for us it's like.
David Bau: you know it like, is it subspaces? Is it? Attention? Heads, you know. Is it individual neurons? Is it like sae, you know outputs right? Like, yeah, it's that's that's something that we have to figure out.
David Bau: I mean, it does seem like with the residual stream stuff that everything kind of turns into a large summation. So you do have a you can take pieces of the sum, and just like into take them individually and see what happens to them. Yeah. But
David Bau: yes, yeah, right?
David Bau: Yeah. But to like, put a radio label on like a feature and then track it as it moves through the model would be like that, I think would be the that would be the equivalent. Yeah, yeah.
David Bau: And I, yeah, I guess we don't have any like
David Bau: exact analogy to that? Yeah.
David Bau: cause like the you know, cause and and like it, could you even it like, does that even make sense to talk about like at the beginning of a residual stream? Is it like the same feature that gets like changed over the course of the stream, or like, does that? Does it make sense to talk about it like that?
David Bau: Yeah, not sure.
David Bau: I don't know. I feel like the radio is, think solving something much more basic. That is very trivial in Cs.
David Bau: something like, you know.
David Bau: what's the what's the activation here and what's multiplied by this weight? And that's how much information going to the next neuron. That's a huge deal for neuroscientists to figure out like, how much information passed from this neuron to the other neuron.
David Bau: But that's like a given for for cs, yeah.
David Bau: sorry.
David Bau: Cool.
David Bau: Yeah.
David Bau: no, you had. Oh, cool. So there's like, seems to be quite a big difference between neuron and like a real neuron. Do you think that there's anything to be gained from
David Bau: modeling the actual real neuron as compared to just the artificial neural networks that we have. I don't know. I feel like that's how we started in deep learning, like, you know, I mean a lot of the research money that was put into deep learning in like the, you know, seventies and eighties
David Bau: was like these psychologists that claimed that they were like modeling actual neurons. We in intro neuroscience classes you actually learn about like some of that work which is kind of an interesting like parallel. And then, you know, it kind of drops off really fast. But
David Bau: I'm not super confident that
David Bau: I mean, maybe I don't know it like they. They tried to do that a whole bunch in the in the eighties. And it it led to a lot of stuff like I. It sort of evolved into what we have now. But I'm not sure that it like.
David Bau: you know, it sort of like naturally veered away, I guess. Throughout the research process.
David Bau: I don't know. But you know, maybe there's something to begin to. Yeah, artificial neural network as a magnetic model of the brain. Then you have to think about which level abstraction you're thinking about it.
David Bau: You could think about it as it's implementing the same a similar function as the brain. Or you can think about it as voice learning similar representation, or you can go even deeper, like, well, it's
David Bau: actually implementing similar mechanism of single neuron, etc.
David Bau: I guess, for different. Your sites are stipulate differently.
David Bau: Yeah.
David Bau: yeah, there's also this question of like, yeah, like, what Josh said, you can zoom in like almost as much as you want right cause, like once you zoom you like.
David Bau: The level of abstraction that I was talking about was at proteins, but, like you can also zoom in to like parts of proteins. And you can look at individual atomic interactions if you want, and we understand that too. And then you can zoom into these individual atomic interactions. And you can look at like the the quantum effects of those like you can like, you know. And you obviously couldn't. You can't simulate at that level. So,
David Bau: yeah, it is sort of a interesting question. Yeah.
David Bau: One sort of thing that was lingering on my mind during the talk was how.
David Bau: I guess I think it's in part because I don't have as big of a biology background. So a lot of the metaphors aren't necessarily as immediate. Yeah. But I was thinking how
David Bau: it just sort of applies for all of science. Anything that uses the scientific method as a way of figuring out things. A lot of the stuff seems analogous like, for example, like tooling. I guess biology is a very good metaphor with, like, you know, zooming in. But like you can think of like, I don't know the
David Bau: like. Like for experimental physics, they build better machines, they let you get more precise, and then you can get more confirmation for your hypothesis. Yeah, I guess what I'm curious is, and I mean the radio labels discussion, I think, helped me see a bit more the direct, maybe sort of philosophical
David Bau: similarities that you can kind of use. But what I guess like, do you think? Because at the start you sort of motivated us, understanding more biology to have a better sense of like how to go about things. How, I guess, would you like? What things
David Bau: would be worth analyzing in more detail to gain good insights.
David Bau: Oh, like in interpretability, or like, what is your interpretability?
David Bau: What would be worth analyzing in more detail.
David Bau: I don't know. I mean, I don't know. That's a big question I like, I think I think the one of the big challenges that we have in front of us is is figuring that out like I mean, like we were like what we were saying before. I think for us it's not as obvious like what to zoom in on as it is for the biologists, because they have these like discrete, really like
David Bau: obvious things to look at and for us it's like, sort of more of a soup of matrix multiplications. Right? yeah. So that's that's hard to know is then the case or the the point you're making that there's a lot of existing
David Bau: things that have been completed. And there's lessons to be learned from the history of how they develop. Yeah, yeah, I think I think, the motivational point that I'm trying to make is you have these 2 systems that are both grown. And the biologists have actually figured out a ton. And this very particular mechanisms that exist in this
David Bau: biological grown system, and it took them for a long time to do it, but they've made a ton of progress, and they have worse tools than we have, and they have more noise than we have and and all they had was like time, and like ongoing effort, so that's the motivational point that I'm trying to make. And then the yeah. And then the directional point that I'm trying to make is A lot of this progress happened
David Bau: through a combination of like increasingly good tooling. And then sort of looking at the same questions with like a lot of different tools, and understanding the parts of them. With all these different like methods of looking at the objects of study.
David Bau: I see it. It has a question online.
David Bau: Go ahead.
Ayda Sultan: Yeah. So I'm just asking.
Ayda Sultan: There was like this resistance to ingenuity. It comes to the biology aspects, because now I.
David Bau: So what?
Ayda Sultan: Of the, for example, Dan Hendricks.
Ayda Sultan: Is it audible.
David Bau: Yeah. Sorry you were. You were cutting out a little bit at the beginning.
David Bau: but I think we can hear you now.
Ayda Sultan: Yeah. So I was just asking if there's any parallels between
Ayda Sultan: the resist the current resistance and interpretability. Was there some sort of resistance before as well.
David Bau: Oh, yeah, so so I think then you would. You would be going back to the like. 8. The 18 hundreds is sort of when this
David Bau: interpretability quest and biologists really got started like kind of the end of the 18 hundreds. I think there was. I mean before that everyone in medicine was doing everything you know, really empirically and really like
David Bau: kind of almost like alchemy right where it was like, you know, it looks like, maybe if we have these long bird masks on, then we like won't get the flu as as much. So let's everyone wear these bird masks right? That was like those black death pictures that you see of the plague doctors right? Because they thought that like there was something in the air, right? So it was like, you know, and I think they didn't bother to look at mechanisms
David Bau: for quite a while. Until until you like.
David Bau: until there was like some results that started to appear that that made people realize that looking at mechanisms was useful.
David Bau: but yeah. So I think I think there's but I there's a little bit of a difference, which is that I think biology like there wasn't this same sort of like
David Bau: big separate culture of like economically motivated you know, kind of just build attitude like I like, I think, for us.
David Bau: we have a whole culture of people in industry that like are getting a lot of economic value right now out of sort of ignoring mechanism and just continuing to build out big systems.
David Bau: And I don't think there was the equivalent of that in biology in the 18 hundreds. Actually a good example that I learned this when I was pre-med was like the 1st time that someone like medically recommended hand washing as a procedure for doctors. This guy named Ignaz Semmelweis, and
David Bau: he worked at a hospital where they did a lot of deliveries, and then also this, the surgeons or the people that would do that they would get done with working with dead bodies, so they'd go from the dead bodies to delivering babies, and then the moms would get sepsis.
David Bau: and he was like he was like, we need to start washing our hands because of like
David Bau: there's all these moms are getting sepsis, but because, they're like these really proud surgeons that, like they they like were like Super, confident that they knew what was going on. But they weren't really looking into mechanisms. And there was like that economic factor, too, of, like, you know this is a waste of my time.
David Bau: They didn't wash their hands for so long, but I think by knowing the mechanisms you know, that we know now today in biology, there's so much like better practices. Which might be like kind of an analogy.
David Bau: For like interpret, what interpretability could give learning, you know. So my my understanding of the early. So you know, I I think there was an interesting. So if you go back and look at the history.
David Bau: There was a split between in biology. There was a split between botany and medicine.
David Bau: You know the the biologists.
David Bau: You know, we're sort of in the tradition of Darwin.
David Bau: They, you know, we're very scientific
David Bau: in their way, but the field was all dominated by breeding.
David Bau: and all these breeders. They were like botanists. They would look at physical, observable things in plants. And notice, if you breed, if you bred the plants in different ways, that different physical attributes would show up
David Bau: differently.
David Bau: And and they didn't really think that looking at the chemistry of these things
David Bau: was a productive way of thinking about biology. There were biologists who looked at chemistry.
David Bau: but they didn't call themselves biologists. They called themselves, you know, physicians.
David Bau: They looked at medicine, and you know the idea that you could invent some potion, that
David Bau: the drink that cures your disease, or something, is like an old dream. People have been working on that forever. So a lot of chemistry is related to the, you know, the medicine people were very interested in interaction
David Bau: between the body and chemistry, but I think that crossing this divide like
David Bau: the chemistry people that that didn't look at questions of heredity like that was
David Bau: that. That's that's like the botany people.
David Bau: right? You know the biologists, the botanists.
David Bau: They think about heredity. The medicine people just think about
David Bau: you know what? What if aspirin is? Gonna make you feel better, or something like that.
David Bau: and what the chemistry of that might be.
David Bau: And so this idea that you might cross
David Bau: between the people who are doing heredity and the people doing chemistry
David Bau: was this cultural divide in the field? It was like 2 separate fields. And I think that's where a lot of the early resistance of
David Bau: like biological interpretability came from like like the the biochemistry which is like not, it's like the backbone of the field today.
David Bau: It it couldn't really emerge as long as these 2 fields were separate.
David Bau: and it took a while for people to realize
David Bau: that it was a legitimate thing to do to either. Look at heredity when you're looking at.
David Bau: you know chemistry at the same time. Or look at chemistry while you're living.
David Bau: Yeah, that's a good example. Yeah.
David Bau: So I think this is. It
David Bau: is exactly what we're facing today, actually, because the trainers are the breeders right?
David Bau: And and I don't think that they think it's illegitimate to
David Bau: look at, you know, representations and
David Bau: matrix multiplications and things. They just don't think it's relevant to breeding.
David Bau: Right?
David Bau: And and you know, so that's the bridge.
David Bau: sweet.
David Bau: Any other questions?
David Bau: Cool? Alright. Well, yeah, thanks. Yeah.