David Bau: Great.
David Bau: So yeah, go ahead and project to whichever one you explain. First, st all right.
David Bau: do I use my computer audio, or we use the audio over there. Use. Use the audio in the room. It's okay. Are you guys able to hear us?
David Bau: I don't know if anybody can hear.
Sumeet Multani: Yes, yes.
David Bau: Okay, great
David Bau: that someone in or they want. Analyst. Yeah.
David Bau: okay.
David Bau: what are you saying? She's online on her iphone, are you on? Are you? Are you on your way in, or are you gonna watch from remote.
Koyena’s iPhone: I'm on my way in.
David Bau: Okay, alright.
David Bau: Okay. So damn. So today, I'm giving an update on end. If we got a lot of cool, exciting changes in year 2.
David Bau: Basically, in this presentation, I want to talk about the different use cases of Ndif, and like hopefully, get some feedback from you guys on like what would be useful for you and your experiments, and maybe like, categorize certain types of experiments.
David Bau: And then I'll talk about the new features that are coming and like how we're addressing these different use cases.
David Bau: So
David Bau: to be useful to you guys, Ndif can do a few things. Obviously, we can make insight better. We can make the Api better. We can make it more expressive. So we can do more things.
David Bau: which we're already doing. Of course, we can have more learning resources. So we can have tutorials, workshops lots of implementations for different papers using ndif and insight.
David Bau: And then, of course, we can just broaden the applicability of. And if and you know,
David Bau: satisfy a lot more people's use cases. So you know, people who maybe weren't using it before. For some reason. Now, they actually have a reason to use Ndif and a lot of those changes come with actually changing. You know the server side ndif code. And so that's what I want to talk about today is how to broaden the applicability of Ndif to researcher use cases.
David Bau: so I was trying to think of some different ways to categorize the different use cases for end of
David Bau: I have 3, but only talk about 2 of the slides.
David Bau: Small experiments categorized by like fast. They're exploratory. There's usually not, you know, super critical. But you're just trying to like try something for the 1st time, or you know, do like an exploratory analysis on a new technique or a model you've done before. You know iterating on your experiments.
David Bau: and you know, to make all that actually possible, it needs to be fast. You need to be able to, you know, if you want to try running larger lens on a new model that you haven't seen before, or like your small experiment. You don't want to be waiting a while like I don't know, at least personally, I would just be like, well, forget this, like, I'm not even going to bother with it. If just to try this little tiny thing which I'm not even sure about. In the 1st place, I have to wait a long time.
David Bau: And
David Bau: usually in these type of experiments, you're targeting a small pool of models like you're saying. You know, my entire study is going to be on the llama A or 70 B, or maybe you know, just all the llama models, but it's not going to like span across all language, general, like Gemma and Uk, and such.
David Bau: so yeah, so you know, short waiting times definitely important having a large pool of models. So you can, you know, do your little experiments on, say, like any model? Because this doesn't. This doesn't prevent you from doing these little experiments on, say, some random model that I've ever heard of. You know people send these things as a code I see like this weird string from hugging face like, what is this model? But I'm sure you all guys have these use cases all the time.
David Bau: A a big thing for this use case and the other use cases. But I'm highlighting in here because of just how important it is, for everything is
David Bau: reproducibility right? So like you want to make sure that your experiments on the model that you think you're running on is the same every time. And really, that's just a matter of making sure. At least the same seed is locked in for your experiment, the same configuration, and something that we don't really do right now. But we're gonna have, for, you know, another important thing in year 2 is like the same revision. So you know, in like face, you can specify revisions.
David Bau: for, like, you know, the different training checkpoints of the model.
David Bau: not as important is, you know. You don't necessarily have to have, like, you know, super short waiting time for like uncommon models. You know, I think there's an understanding of if you want some weird model as long as we can serve it for you at some point.
David Bau: I think that's probably good enough.
David Bau: or you know that particular use case. Oh, I didn't finish this. But but there definitely should be an understanding of like the behavior of like, okay, fine. You're not gonna have this weird model for me. But I want to know, like, when will I be able to run it?
David Bau: yeah. So I can actually like schedule and plan for the future that I connected essentially.
David Bau: Oh, yeah.
David Bau: So, on the other hand, I just, you know, large experiments which I you know the characteristics are systematic. You're probably, you know, running some script that does some large sweeping experiment over many models or large models. It's time consuming. You'd probably be fine running this for like a day, you know, a couple days.
David Bau: and usually these are more critical where you're like, okay, I have an experiment
David Bau: or I have some technique that I've iterated on. And I'm kind of happy with. Now, I want to actually run it on a huge data set or a bunch of models. And I want to build my table for my paper, right?
David Bau: So, you know, doing grid search over parameters.
David Bau: benchmarking over tons of models for your technique, studying training dynamics. So you know, going over multiple different checkpoints.
David Bau: and then, just even if you might not use a lot of models for these large experiments, you might have really long jobs again, iterating over a data set doing sort of optimization loop. Or, you know, large models like 4 or 5 B, and you know even bigger ones. Those will still be long jobs, even if it's probably a smaller set experiment.
David Bau: So again, reproducibility in a large pool of models are important here as well.
David Bau: having fast data integration. So one big, what one problem with these type of use cases is for, like, you know, data set iteration.
David Bau: You kind of have to upload the data set as you're running it to Ndif. But we definitely need at some point, some way to cache common data sets, or you to upload things to as an S 3 bucket that you can connect to. Because you guys do this a lot and it certainly is not tractable right now.
David Bau: or are you reading over large distances.
David Bau: having job status and feedback? So you know, if you say like, Okay, well, here's my job that's gonna build my main table for my paper that I want to kick off. I don't want to just kind of like just throw it into the cloud, and then hopefully, it works. I have no idea. So you should be able to get, you know, feedback, either in your console for sure and then ideally check on it via email, some type of
David Bau: notification like when things are done. Did it break?
David Bau: What happened?
David Bau: And then, Apis, for you know, automating this process so like, if you want to build like some type of framework on top of this and rely on it for the future. You wanna
David Bau: have all the tools for you to actually be able to build those automation tools.
David Bau: And then, you know, one of those things is job cancellation. So like, if you want to run a long job. And you're like, Oh, well, I actually forgot one line of code. Now, that's gonna take 2 days. Well, you should definitely be able to cancel your job
David Bau: for so for for these type of jobs, having, like immediate access to any model in the world isn't as important as like the smaller experiments. Again, you want a large pool of models, but you don't have to say like, if I if I tell you okay, you can run your 3 day long job. But it's gonna happen on Monday. And you know today's like Friday, that that should be okay.
David Bau: Yeah. And so before I go on to like how we're addressing these.
David Bau: maybe just like a little open discussion on
David Bau: the general categorization architecture of like these different use cases that I've laid out.
David Bau: Specifically, I I'm looking for use cases that have to do with these type of like sweeping over many different types of models, iterating over data sets which
David Bau: you know. I don't think Nikhil's here, but you know, for his paper. He had a good idea over a huge data set. And that was a huge problem, because even if it were 99% of the time, well, if I have one of the rows break well, he has to have all this code that retries and resumes for a specific point of his experiment. Run?
David Bau: Yeah, any questions on that.
David Bau: So I think, so I actually wrote that piece of logic here. So what what was happening is that sometimes when the model. And if side actually sends us the data.
David Bau: the data sometimes get corrupted somehow. I don't know how that happens. Yeah, that is, I actually forgot what kind of error, but it's it's a decoding error, some kind of a decoding error. So the data got corrupted. But it happens so rarely. Like every 100 requests. It will happen once.
David Bau: and that's why we need to have this try cache thing
David Bau: But then it got solved.
David Bau: I thought that
David Bau: I thought that it's resolved now, right? Like as a month or 2 ago. Yeah, because I remember that you you specifically said me to remove that piece of logic, and I removed it, and then we ran everything again. And it was it was fine. Maybe it's because we were using we had a single vm, and we were using like Min I/O as our object store before. And now we are using an Aws bucket.
David Bau: And that's also you're doing 4 5 b, or like, is this happening? 70 B or anything?
David Bau: So I believe that we were doing it for 4 or 5? P, yeah. So so that was the only like our distributed model, and like that, had so many different problems because it was distributed so like.
David Bau: I would imagine it's something to do with that. But probably that's why you know, it's not stupid anymore. So maybe that's why I don't know, probably. But I am assuming that that problem is now resolved also, I I was waiting for you to finish but and I wanted to say that sometimes the jobs get queued and the users definitely should have an option to cancel their jobs. But you are already addressing that. So yeah, so awesome.
David Bau: Yes, yeah.
David Bau: Which sponsored session polls.
David Bau: So yeah. The access to buckets also? Awesome. Yeah.
David Bau: I don't know. Chris, do you have any other thoughts like that's probably internal like, yeah, are you talking about having buckets that you can put data in yourself. Yeah, like, if you just want to specify their own S. 3 bucket, be able to connect that to your experiment and pull it from there versus like, you know your crappy laptop. Oh, I see. Okay.
David Bau: Chris, do you want to explain verbally.
Chris Wendler: I don't think there is much, much to explain. Just it's I imagine it being useful to, you know. Be able to download data from my bucket. Yeah.
David Bau: Yeah, I guess, too, because you could use forget that the jargon they use. But there's different like protocols for fetching data from aws or sorry from s. 3. That could be like really fast. And if and if or insight allows you to use those interfaces, it might mean that you can make your experiment actually a lot faster if you're able to like batch, grab data using the Aws or the S 3 protocols versus, just like
David Bau: having it serially uploaded from insight directly.
David Bau: Yeah. So sorry. Go ahead, Chris.
David Bau: Yeah.
David Bau: So I'm I'm just trying to get to. I'm gonna show you a really cool demo of like a lot of this stuff is pretty much done. And I'm really excited to show you guys. But like, so so what does a lot of it do so to have more models where we we can't host every model in existence at the same time.
David Bau: But of course we can on demand, swap in your model when you've requested it. So that's probably going to be like the biggest offering and piece of end, if and this is a ton of stuff around that in order to make that like work and make that nice. But it's when you request any hugging face transformers model, it should be able to be deployed. You request Strand, and then, if somebody else comes along, you know, at an appropriate time, remove yours and put it back up.
David Bau: We want shorter waiting times for this, this on demand model allocation. So we want to essentially have tiers of how readily available model is so at its most available right? It's already on Gpu. It's ready to serve
David Bau: below that we'll have a CPU caching mechanism, where, if
David Bau: you've already requested a model, and it's come down. We actually don't totally take it down. We just move it to CPU first, st and then the next time you request it, it's actually up, and half the time it would normally be if you loaded it from disk.
David Bau: And then finally, we have a bunch of models already downloaded. Those are, you know, cold storage. And we can, you know, host, this for you? As requested, just like with warm. But it'll be slower, obviously, because it's from disk
David Bau: for reproducibility we want deployments by revision. So one of the use cases,
David Bau: that have been noted is like doing all more checkpoints or something. Right? You want to iterate over a ton of different checkpoints for the same model, to understand how it changes over its training.
David Bau: But right now, when you request a end of model, all really, that we're we're looking for is the repo id. So you can't really specify a specific revision.
David Bau: Yeah. And I have some
David Bau: questions about exactly how that should work in the sense like, if you don't specify a revision, should it just kind of like use, whatever revision is up, or should you always have to specify a revision, no matter what?
David Bau: I don't know if you like already, have like a thought about that, like, I imagine a lot of the time when you're like loading a hug and face model. You don't specify like revision equals
David Bau: the checksum or the
David Bau: usually not. Usually it defaults to main. But if you want to iterate over like different checkpoints, then you should be able to do that. Yeah. But but then, like, let's say, like you request
David Bau: to run a job on like like, you know, Omo, or whatever, and you don't specify the revision, but the current deployment revision is like the second to last one. Do you care like? Is it like? Oh, no, I really need the most recent revision. Usually I don't care. No, no, but I think that I think they do care. I think they do care. I think that you should, I think reproducibility is important. It should just give her like, it should go to me. Yeah.
David Bau: okay, yeah.
David Bau: It always defaults to me, though. Right? That's right. But like, if somebody else before you said like, I want the second to last revision, and then we deployed that, and let's say it's like 4 or 5 B, and you know, it's going to take a while to redeploy like the latest revision. Yeah.
David Bau: I think you want main. Yeah, I I don't. I don't think you you want to have a situation where
David Bau: you know I ran a 4 h experiment, and then
David Bau: and then tomorrow I run it again, and it's different. Because, you know, it's a different checkpoint.
David Bau: Yeah, okay, is it possible that the main ones are always deployed? But like you just upload the revisions
David Bau: your cash? No, I mean, like, you know, we can't like. We don't have enough resources to deploy the main of like all the main models, and then have space for people to do revisions. Yeah, to like, if
David Bau: someone just doesn't specify a revision, it's just it should be the main. Yeah. Yeah. I mean one option could be. You have main is like always like hosted or not always hosted. But like, if if you if you if a user wants a revision. It will bring up the revision. The experiment will run, and then it'll swap back for Main afterwards.
David Bau: Hello, good mix
David Bau: so we want a lot more transparency into the system, because it is going to be, you know, significantly more complex.
David Bau: So users should have an understanding of how long it's gonna take to deploy ideally. You would even ask you to say, like, Okay, it's gonna take an hour to deploy like, do you still want to do it as of now. It'll it'll just, you know, tell you a time, and we'll do it for you. But
David Bau: finally you'll have a position in the queue, you know, which is something we should have had a long time ago, but like because of
David Bau: one of the dependencies we have is just really hard
David Bau: and then for some models that are, you know, maybe up all the time you want to like, be able to see the schedule and say, oh, oh, I know 4 or 5 B is going to up. Be up in this week, and then 4 or 5 B instructors gonna be up the next week. I can plan for my experiments that day.
David Bau: Yeah.
David Bau: So demo wise?
David Bau: yeah.
David Bau: So I have gpt 2 up and deploy right now.
David Bau: So I'm gonna run, generate on it, get some 10 new tokens and then return those back to me.
David Bau: Okay.
David Bau: nice cool. Okay. But now I'm gonna ask for Lama 8 B.
David Bau: Instead.
David Bau: And what this will do is this will go to the server. Say, Hey, please give me llama A. B.
David Bau: It won't be up. So the system will have to take down Gpt to kind of launch like a 1 Gp machine and then deploy Ab.
David Bau: so then saying, Hey, okay, here we're deploying it for you. Please stand by. We're dispatching you request to the newly deployed model, and then, within like 6 seconds, it should deploy and execute.
David Bau: Cool.
David Bau: Nice, that was pretty fast for Lum ap,
David Bau: So what this actually looks like from your perspective is our brand new status page. Whoa! Look how pretty it is.
David Bau: So we're going to get the model status which is going to show you these different tiers.
David Bau: And right? So we just requested lama A. B. So now it's running.
David Bau: it's hot. It's ready to serve.
David Bau: and we just brought down Gpt. 2. But, like I said before, bring it down as an immediately, you know, destroy the entire deployment. It just moves it to CPU. So now you know that like, if you now request, Gpt. 2, it's going to be up pretty fast.
David Bau: And be ready to serve.
David Bau: And then here are all the different models I have in my hugging face. Cache on cold storage.
David Bau: Yep,
David Bau: So for some models, right? Really, like large ones and ones for specific offerings. Say, like a 4 or 5 E pilot program, we're gonna want some models deployed all the time. So you know 100. These will be up where we guaranteed it for you.
David Bau: And before that would be a very manual process where we went in changed the config, redeployed everything.
David Bau: How we're doing it now is we have this Google Calendar, which you, as a user. Could you know, attach to your own calendar?
David Bau: I'm gonna say, tomorrow, I want 70 B to be deployed and then the day after that, I want 8 B to deployed.
David Bau: and the status page will reflect that.
David Bau: Yep. So Ap is already deployed.
David Bau: But I guess yeah, it'll start again in on Monday. But yeah, 70 B will start on 8 pm.
David Bau: For 24 h and be available
David Bau: I also got these nice little like copy buttons on the top right of these, where, like, it'll give you a little snippet of insight code that allows you to run this model. And
David Bau: so, you know, if you didn't know how to do it.
David Bau: That's good.
David Bau: So right now you are not imposing any restrictions on any kind of models like you are not whitelisting any models, and also like not restricting any like imposing any restrictions on users like.
David Bau: I can just run 10 experiments, request 14 or 5 P models, and I should be able to just do that. So so here are the, I guess, restrictions. One, of course, with
David Bau: plug and face. You have the trust, remote code flag that's obviously false here, so like you couldn't just upload your own model, you know. Write anything in the model architecture, and then also run it. It has to be one of like the transformers model Gemma, Gpt. Lama. So and that's that's a pretty big pool of models, the main models. But you know, you couldn't execute any code you want via models.
David Bau: Let's say I have this access to all of these models
David Bau: like you have. What do you mean? Like you have gated access? You're saying
David Bau: so I I'm assuming that for Llama you need to like, fill up some sort of consent from form, supply that, and then they give you access to the model. Have that access? Yeah? And then no, no, you also have that. But let's say I want to.
David Bau: I'm just trying to understand like, how generous you guys are actually or that's how generous and it is. Let's say I want to just to mess with you. Let's say I have like I have. I am trying to run experiments on like
David Bau: 14 billion parameter models. And I have access to those models.
David Bau: Let's say those models do not even require any kind of access. Can I just load them absolutely. You can make a loop right now. Do what I just did in the loop with different models. It would bring one down. Bring one up, bring one down. Bring one up. Okay.
David Bau: Are you sure that you guys want that?
David Bau: Well, so some restrictions will be again. So a lot of this stuff is like stuff we just have to like tune. I don't really know. I'm just gonna say, yes, we can do it. We have a minimal deployment. If if not, we should probably have a wait list.
David Bau: Well, no. But I mean, like I was. Gonna say, yes, you could deploy any model. But obviously there will be some restrictions. But, like how fast you can change models like a model should be up for some minimum amount of time before being able to be evicted, which is functionality here. But in in the sense of if you're willing to wait the minimum of time. Yes, you could loop over like a million models and do whatever you want. And then, should you? Should you tell, how does the user know that that's the deal.
David Bau: That it's only up for this, hey? You know what I I really, I have this great idea. Now that we have end. If I'm gonna make
David Bau: want to write a paper where we study, you know, 500 different models right? And then and then they start and they can study, you know, 3 models, but then they they get honey potted.
David Bau: you know. It gets slower and slower because you have some hidden.
David Bau: you know. Scheduling thing is it? Is it just hidden that you know they can't actually do 500? Because
David Bau: it would. You know, we have some policy that, you know, makes it really slow
David Bau: to do that? Or is there some way right now? It's not like slowing it down based on like some user request. It's just saying there's a minimum amount of time for a model to be up. And that is reflected both on the status page. And then so it does. It. Does it say so? It doesn't. It doesn't say so. I I don't have a time configured right now on mine. I just have like it's why? Because I want to like show, bring metals up and down. But yes, if I configured it to be like an hour, it would show like an hour left. It's pinned for this long.
David Bau: I see. Yeah, I see. So so if somebody, if if if the cluster is sort of quiet.
David Bau: nobody's using it, and then somebody starts their project where they're they're gonna study 500 models.
David Bau: and then they they load the 1st 20
David Bau: and then now they're all pinned for an hour.
David Bau: does that mean that nobody else can use the cluster for their regular models? Yeah, yeah, there probably should be some type of rate limiting for deployments as well as just requests. We have. Yeah? So that's yeah. That's so. I'm
David Bau: like, I, I worry that
David Bau: the graduate students will be very creative in the types of research that they want to do. But I worry
David Bau: that if we if we kind of come out and we say, Hey, you know, you can do anything. And actually you can't. Yeah, that's why, like I, you know, I'm a broken record, and like, I think on release day, we should have a white list with like one model in it. You know what I mean, and then and then gradually bring up, because
David Bau: because it's not just, you know, is the system capable of doing something. There's an interaction
David Bau: with. You know what the clever researchers are. Gonna push the system to do.
David Bau: And we have a unique chance to
David Bau: shape that traffic with a white list. If we don't have a white list. And
David Bau: you know, we're we don't have any control over what that traffic is. And now we can add a white list later. But then we'll make everybody unhappy, because people will be like. Well, I
David Bau: already, you know, started my nerps submission with these things. And it's not on the wait list. And
David Bau: you know what I mean. Yeah, I think that's a good idea. I think that the end game that fixes all this is like again, like a credit system. Right? Yeah. But but yes, yes, yes, but it's going to take some time. Yeah, right?
David Bau: Yeah, question about that. So let's say, I deploy a model right? It was not already deployed. So it has to be uplift for an hour. That's the rate
David Bau: you've said, and then somebody else and all the resources are taken, and somebody else asks to deploy another model. Will the request fail because the other model is still deployed for like an hour? Is this like canceled, or is it in the queue until the other one goes down. There's no available space.
David Bau: and the same thing happens if it's gated and we hadn't
David Bau: on our end, like had our own base token. Get approved for it. They'll just throw an error back to the user. I don't think it says specifically, Well, yeah, I don't think it says specifically, the model is gated. But I think in the I think there's there's a particular
David Bau: return status from the Controller that says this model actually doesn't fit like. It's not just that there's not space to place it. It's just it won't work on the cluster. And then it says in the description, like, maybe it's gated, maybe it's too large. And in the case of a gated it would probably be more clear to the user. But yeah. So then, I just have to like, keep watching the status page to see when there's like
David Bau: a model goes down for me to be able to like, send a request in.
David Bau: Yeah, I guess you could just make a script
David Bau: might be what people do in practice.
David Bau: Oh, so the model like has to stay up for at least an hour. Is that how you set it up right now? We almost certainly need to do like more interesting things than that. But that's like the most basic thing. Oh, like if I want to just run an experiment for a couple of checkpoints. But the experiment takes like 5 min. So what I want to do is I just want to upload the model like 1st checkpoint. Run my 5 min experiment, then take it down and like load the next one. So it's not possible with this as it works right now. No.
David Bau: I think maybe we can have, like in the request when you request a model that's not up. Maybe the person requesting can have a flag that like, oh, just take it down after this experiment, where there's a map, there's a default, and then you could suggest anything less than the default. So if you want it for only 30 min, then you put in 30 min.
David Bau: but you can't put 2 h. You can only put Max 1 h, but if you really know it's going to be quick, then you can. Yeah.
David Bau: So another thing I was a little bit confused about, let's say I ask for a specific revision of Omo.
David Bau: Are you like rewriting or swapping the main model with the revised model.
David Bau: As far as we're concerned, they're just like different models altogether. Like, it's just like, yeah, yeah, like, the main model might not even be up
David Bau: things right? What you're saying? Yeah.
David Bau: So let's say, the main model is also up. Let's say it is something just you just offer as default. It's all running, and I ask for a specific revision of almost. Now I have 2 different models. One is, whatever the default. One and another one is whatever I if I ask for a specific revision, it will give me that. Yeah, right? Okay.
David Bau: There, you know, there's really no like time savings to be had to like.
David Bau: even if we like. Wanted to like, swap out like the parameters. Yeah, you don't want to do that. Yeah.
Emma’s iPhone: I have a question,
Emma’s iPhone: can. Once the models up if other users start using it? Is there something where it's like, okay, multiple users are using it. Let's keep it up.
Emma’s iPhone: Are you considering that.
David Bau: I mean, yeah, again, like all of these things, I have like on a list like to consider like tune. It.
Emma’s iPhone: Yeah, yeah.
David Bau: Yeah, I I really, at least personally, I feel like I need to just
David Bau: see how people use the system and see how it behaves to actually know which one of these to use and how to set the values.
David Bau: But yeah, like, that is definitely something we should consider like, are there things in the queue how many people are using in the queue when last time it was used so right now, just will inform all the users that the model was evicted and just fail their requests. In theory that could be changed. To then try and submit the models be scheduled again before actually failing them. If it doesn't actually get scheduled, I'm going to anticipate a question. I see Submit's hand is raised. But I'm I'm here in person. So I'm going to ask this question first, st
David Bau: which is so, how do you? So you're gonna learn from the user behavior.
David Bau: So how do you know what the user behavior is that you're gonna learn from
David Bau: talk with them, or or or or do we like, do you need like really specific
David Bau: logs? Or, yeah, yeah, things like that logs to see how long people are waiting. Do we need like analysis tools to extract the relevant stuff from the logs so that you can use it to decide on. Yeah, I mean nothing that complicated. Well, I guess maybe there is some like algorithm we could find like, given. You know, users are waiting this often, and you know users requests fail this often because
David Bau: they're not having employment. What is it? The optimal best values? I guess that's probably possible for some things, or at least some heuristic to get better.
David Bau: But yeah, we kind of confused how often people's jobs don't get ran because it was too full. So do you think that we're logging everything
David Bau: so that any of these questions could be asked? Well, there's the queue has a state which any moment in time can be fetch, and the State contains, like a snapshot of everything in that point. In time and in theory
David Bau: there could be a tooling built on time. And you can you get access to the state via the queues pinpoint? Or is that is that logged. It's ephemeral, but it could be. We already know from logs how long people are waiting queue if they're jumping out because it was.
David Bau: I don't imagine that the state is very large. It feels like it's the kind of thing that you could just log exhaustedly so that you can analyze how the queue is dating. I imagine that being sort of a source of truth that would be used by testers would be used by
David Bau: would be fetched by, for example, the status page to get information and insight of what's going on in real time. Yeah. And if you guys are, if you guys are hoping to learn from behavior and debug analysis. If things are going wrong, you have this dump of them for the last whatever week. And yeah, why a week. Why not just deploy it. Yeah, right? I don't know. Unless unless we're talking about terabytes of data, it may grow quickly. I mean, it depends how much of it, you store, but each tick, each tick. So you I mean, it's like a
David Bau: I don't know. It's pretty. I don't know. I've been logging something for 15 years pretty exhaustively, like every every click or something.
David Bau: And you know we're we're we're only a hundred gigabytes after 15 years, and think that
David Bau: probably can log a lot more than you think. Everybody right? Yeah,
David Bau: So so sumit. You had a question.
Sumeet Multani: Yeah, it's it's related to democratizing access. Right? So is it weekly or monthly? How do you keep tabs on? Say, I have no greater than 100 billion parameter model, and then cumulatively over a week.
Sumeet Multani: I can't run more than, say, 400 B. Or use more than certain, gpus. Are we keeping tabs on a user
Sumeet Multani: up?
Sumeet Multani: How do we keep a track of this in terms of just utilizing Gpu's in a model size, just being small and medium and doesn't go large?
Sumeet Multani: Or are they allowed to do large? And they can do multiple large models.
David Bau: So, or or is your Meta question like, how do we make it fair? So that one user doesn't, using the system as their own private cluster?
Sumeet Multani: Yeah.
David Bau: Right? That's why like it either has to come from just like general system behavior, like we're saying, where, if people are, a bunch of people are using one model, we shouldn't come down. And you know, 1 1 person for question. Bring it down or it comes from like you're saying like tracking the user and actually putting like restrictions on. Okay, you can only have a hot swap happen twice a day or something. You know, I I don't know.
David Bau: I had to know. That's like a whole. I think that's pretty. No, I think. Well, I think you could have a user can only do X number of model requests per day, or like they have a number of
David Bau: models they can request for, and it could be something like that that might exclude certain types of important research. I was gonna ask, like, I totally get the idea of wanting to learn from whatever users do right and adjust to that. But do you?
David Bau: Maybe you have, like a preset expectation of how they might, how you'd want them to go about it like you can bake that into the system, show with credits, or whatever, or you could just put up general guidelines in a blog post or some tutorials. So new new users just kind of know how to behave right like. For example, you're saying, if the model, if everything's full, then my request isn't even going to the queue. It's just getting rejected.
David Bau: So does that mean I should be pinging the silver like every couple of minutes
David Bau: until yeah, until it's free. And if so, that seems like a set script. That may be
David Bau: we can at least like provide right? So people aren't coming up with their own.
David Bau: You know, scripts that do this every second, or something like like I think
David Bau: it makes sense to bake things into the system. But I'm wondering if there's just a way to set general guidelines and have good faith that people will follow them.
David Bau: Yeah, yeah, like, we talked about something exactly. We're talking about like a retry scripts. Or I can like built into how the remote back end works. Yeah? And again, like, it doesn't have to be like coded in as much as it's just
David Bau: building into the queue, so that if if in the queue the user request is not able to get through, it still keeps, sits there and keeps trying to adjust the
David Bau: an attempt to have it be scheduled until it is scheduled, so you can know that you submit your job, and it'll just sit there in the queue until there is a free spot for it to to go in.
David Bau: That seems like sort of what you're asking for, because just giving them a few scripts to do it. I mean my thought on a lot of this kind of on a Meta point from someone using the slur, the Delta Slurn, and having to do that for like just deploying like learning how to deploy the system and having to deal with that is, I think there's like a process of like
David Bau: when you use a thing and you start to hit some roadblocks where you kind of
David Bau: figure it out yourself, like I had to figure out weird. Not that weird stuff, but with Slurn just like, oh, if I just submit a job, and it sits there like I can just leave it, and it will at some point go back up. But then oh, if I forget about it, and I need to look on it like there's like a sort of
David Bau: user side to it, which I think you kind of figure out how to make it work. I I don't know if that sounds very exclusive, exclusionary or anything but
David Bau: that was sort of my experience using Slurn. So I'm not sure it may translate to this setting where people quickly realize. Oh, this thing is hot. And then a lot of people are using this. And if I want to make sure my thing works, I need to do this, and they try that. And then, yeah, kind of
David Bau: don't you think you want the experience to be better than your experience with slander? I mean, that's that's true. That's true. But I think.
David Bau: yeah, I mean, you can't know until you have that experience using fluent.
Sumeet Multani: To.
David Bau: I think in this case you kind of have to see 1st what happens. And then you can say, Okay, there's these annoying
David Bau: things that could be smoothed out a bit and.
Sumeet Multani: Yeah, I think that's the opposite of what the people are saying. Here, at least, what I get is you start with a gated experience, and it only gets better over time rather than the opposite, that they're allowed this white field stuff of doing everything. And then this guideline becomes like in a document which is like 5, 1015 pages. Every time a researcher needs to run an experiment that these guidelines that are changing over time.
David Bau: How to how to deal with. Like, you know, we make. We make a big system that doesn't work very well.
David Bau: and then and then expect people to develop a lore of like how to deal with this really bad system
David Bau: rather than dealing with the system.
David Bau: you know, maybe doesn't do much, but works really well.
David Bau: I guess, like the way I'm thinking about it, too, is like, you know what it's capable of. But nobody else like does right now, I like people using, and if still think that there's only 5 bottles up like. So you don't have to tell them. This is all we can do like. Look at all these things we can do right now. You can just start telling them all right. Between these 5 new models you can pick a new one to hotloud right and like slowly scale it up as kind of new features in a sense that will get community engagement.
David Bau: Without having to kind of like. Test everything all at once.
David Bau: If you also have the bonus that every time you offer a new set of models, it's like you can make another announcement, and
David Bau: right, and say, Oh, if you've been waiting for the latest multimodal models.
David Bau: you know we have 3 of them available to you
David Bau: right? No video yet. Check back next month.
David Bau: Right?
David Bau: So is is that how you think we should roll this out? Yes, okay.
David Bau: yes, yes. With that in mind, I'd be happy to kind of help by whatever guidelines like, have discussions about that or like. Basically, when you do do this announcement, you can immediately attach in some tutorials and some guidelines of like.
David Bau: okay, request one model each time, right? Or something like that. Even if the system can support more right?
David Bau: Something to tell users so that they know how to operate within the system, agreed.
David Bau: Okay, go ahead. It's very exciting. It's gonna be huge. It's gonna completely change the complexion of the service. You know the access more models.
David Bau: Yeah, yeah, if you guys start building a white list.
David Bau: do you know what strategy you would do to like. Choose the models first, st like, am I allowed to provide more collaborators in our in our lab has a nice list. She's she's been pulling the, you know, existing end of users.
David Bau: And you know we have a list of models according to how many votes they've gotten from researchers. So we know.
David Bau: you know, if if you gave me one more model today, I know which one I would put on that would make the most researchers happy. It has been pretty hard to get community engagement just like putting up a poll on the end of this code did get a lot of votes like. So we would welcome suggestions.
David Bau: Protein language models.
David Bau: Is there? Is there a specific one, can you? Can you?
David Bau: I'll probably find one that's like
David Bau: most most people would want. I think he has something like, how big are they? I think they're reasonable. Pretty small. Yeah, I can load them on CPU like OS, but
David Bau: be really cool.
David Bau: Oh, yeah.
Sumeet Multani: Back end related question.
Sumeet Multani: I don't know if it's this. So how does the state of the world change? Once we have so right. Now we are using, say, 4 or 6 a 40 s. And one h. 200. That's the shared infra from Delta. Right? We have this reservation system. Essentially, we are planning to add another H. 200, which will be dedicated to us, which will be beefed up with all the CPU and RAM
Sumeet Multani: until we get to that point. Do we anticipate any major issues for everything we've spoken given, it's shared in front. It doesn't have the CPU and the Nvma.
David Bau: Well, so I mean, it does have some CPU, I mean, like, I feel like the the hot swapping, you know. Warm stuff can't really.
David Bau: you know, be as amazing as it could be, unless we have the large CPU machines. But I mean they have more CPU RAM than they have Gpu RAM. So like.
David Bau: probably they have like a terabyte each. And then there's like each node has their own. So yeah
David Bau: and then and then the Nvme. Like we're right, not even considering right now like doing the local Nvme as a another tier up
David Bau: offering is all just from the shared drive and delta. Yeah, it's pretty fast from the shared drive. Anyways.
David Bau: work. Ib takes 10 min to load. That's pretty quick. It's crazy.
David Bau: And how much does it take to RAM
David Bau: program? Yeah, if you were to swap it out. Yeah.
Sumeet Multani: Sorry. What was the last part.
David Bau: david was asking how long it takes to swap from C.
Sumeet Multani: Be you.
David Bau: For 400.
David Bau: Yeah.
Sumeet Multani: Hmm.
David Bau: Do we know for 70 p.
David Bau: So I've been trying to run
David Bau: and on Saitama, because that's the only 2 Gpu machine I have available, and I can't run anything on Saitama right now. It's so slow I can't run it on Beboo, because it's too small.
David Bau: Oh, I see. And and we don't have a dev environment in Delta. That's what Michael Screenshot is ready. So we so we're still waiting. Okay, cool. That's no problem. I don't know. Are people like using Nagoya heavily right now? Or like, I was thinking, maybe I could deploy this, yeah, always up machine. Right? Yeah. Okay? I think? I asked.
David Bau: is there some something physically wrong with machine, you think? Do we have to like
David Bau: I didn't unscrew it and clean out the fans or something. Yeah, yeah, the past 2 weeks. I keep trying. And it's like, I'm trying to literally just reboot the thing. Yeah, I mean, we try that. Yeah, yeah, I think actually, Saitama might need a full, clean setup because it has been slow for a while.
David Bau: I see.
David Bau: Do you think somebody's mining bitcoin on it. I actually could not really find anything. So it's just slow in general, I don't know why.
David Bau: Yeah, maybe maybe just maybe just wipe the machine. Yeah, yeah, we we need to do that. Yeah.
David Bau: But I mean, like, maybe not. But I would like to put up like a demo version of this, or maybe some people to access and give you their feedback.
David Bau: What? What is? What would be a good motivation to to use this system?
David Bau: And if if I deployed like a version of this on like a bit, maybe end it
David Bau: unlike, say, Hakona with like 8 Gpus, and where you could like hotspot any model out. And like 5 protein language models.
David Bau: I'll be your test breaker, you know.
David Bau: Yeah, yeah, a board would really use it. Yeah. Well, I would definitely use it. Yeah, yeah. Alex isn't here. But like I was, you know, somebody who like, does a sweep over multiple models. You can ask, Alex, yeah. Yeah.
David Bau: I don't know if anybody's like deadlines are trying to like run their experiment on
David Bau: a bunch of different models for a table and a benchmark or something that would be perfect. But
David Bau: I'm running them. But I'm training. I'm not sure if it's it's his own models. Yeah. So what's what's the vision, for when a researcher has their own models.
David Bau: is there a vision for this?
David Bau: They get their own ad, not really. I mean, like.
David Bau: yeah, I mean. I I guess in theory it's like, not even
David Bau: that much different like, it's just we don't host a model. But you're running the same code, and your code can include like an entire model architecture and training. Right? You would just have to, I guess. Tell us how much Gpu you'd want. Oh, so what if it was a standard architecture
David Bau: but your own checkpoint?
David Bau: And so are you. Are you planning the whole model, or are you like?
David Bau: But it's a standard. What llama or something? Yeah, it's like, it's so I'm using the hugging face Pythia model. So like, I can export it to a hugging face checkpoint. Actually, yeah. So I mean, technically, you could do that now is like you would just have to reinitialize all the weights in the model which, if you access the weights. It gives you a clone of them, because you obviously you can't change everyone's
David Bau: But I don't have a way to like swap in your weights, or you can't if he if you can't just refer to that name. Oh, yeah, but but what you could do right? It's easy. He couldn't optimize the parameters.
David Bau: No, no. But let's say he optimized the parameters already. Oh, okay, then, yeah, absolutely. Okay.
David Bau: Yeah. Yeah.
David Bau: But again, is your experiment like about the training, or is it like you train it, and then you can do stuff with it, both like, okay, so I'm running some benchmarks over the checkpoints as well, like the ones that I've saved. You could absolutely do that. You could upload them. That's cool. The whole weights have to get sent back after.
David Bau: No, he's saying, upload them to hugging face like from end of no, from what I'm imagining is you do the fine tuning on end of. And then you want to save that checkpoint somewhere.
David Bau: Yeah, I'm saying there's nothing technically stopping you other than the fact that you'd have to like swap in the weights every single time, like kind of recompute the model. But like I have, that's 1 of the things that I want to add, anyway. So
David Bau: but for Kareem's actual experiment that he's actually running, he could. He could use this now what once you put in the White List.
David Bau: he can't. He's not going to be on the wait list, but for our demo version for the next month, or whatever. Yeah. Yeah.
David Bau: So I'll try working with you guys. And Alex and I'll meet with you hopefully, we can find something interesting to do.
David Bau: Also, have you like considered doing like 2 different policies, one for larger models, one for smaller models.
David Bau: like for the smaller models. You can just swap and swap back really fast. But, like for the large models, you have a limit kind of, okay, yeah, I mean, there's so many different things. Yeah, like that. That could be helpful. It could be like one easy thing to do. Because if I'm like, if I want to just float back and forth, the 160 million model like it should be like pretty. I should be able to like. If you allow me to do it pretty fast, it would be cool. But
David Bau: just just curious would using Pythia remotely. Make sense, I mean, I know you have access to Gpus, but it's not like, do you imagine it being useful, based on the speed it would take to use it to be able to swap a lot of them on end of versus. I mean, not for me. But I feel like a lot of people who don't have a lot of access. Gpu, I feel like if you had all of them on end, if and they were all cached on CPU, if using the ones it would be faster than using locally.
David Bau: Oh, yeah, true. Because you don't have to load that. You don't have to download them. Yes, yeah. Problem.
Sumeet Multani: Is there a way to visualize the current population? Mix in terms of the distribution for small versus large experiments? I'm guessing people move on from small to large eventually. But do you see that? Or do we track that? And do we see that journey of the user by the user as they grow.
David Bau: Yeah, yeah, we have that information like we, I mean, I imagine the proxy for how big the experiment is. Just the length of the experiment. Is that what you're saying.
David Bau: Like how much time.
Sumeet Multani: And it's lent over a certain period of time. I'm I'm guessing we we can bifurcate the existing user base
Sumeet Multani: easily. Is is that how we measure and track.
David Bau: Yeah, we could do that.
Sumeet Multani: David, do you think that would be a good data point.
David Bau: yeah, we we I mean, we can. We can talk about it to me. I think that would be a good project for you to spearhead.
David Bau: Then, you know, to visualize it easier behaviorally.
David Bau: Okay, yeah. And so like, at the beginning of my slides, I talked about, you know, kind of ways to make end of better. And you know one of them is all these learning resources, and that's what Amir has been working on. So
David Bau: he'll tell you about some of the resources we've been making for incentive.
David Bau: Do so. How about that? I just like join the zoom? Yeah, because it sounds good.
David Bau: Give me a second.
David Bau: How much time do I have? By the way, just so many negative 1 min, so go ahead and take half an hour.
David Bau: I'll take much less than that.
David Bau: while I'm while I'm loading this up.
David Bau: Hello, everyone!
David Bau: I'm on mute.
David Bau: I'm
David Bau: summer fellow with the end of team doing outreach. So I'm mostly writing and reorganizing the tutorials for insight to help get user engagement and also help people who are kind of new to insight or just the field of interpretability know how to get started.
David Bau: So
David Bau: let me share. So I'm here. What? Tell us a little bit about your research background as well. Yeah.
David Bau: Right. I did my undergraduate at Stanford, where I worked with Chris Potts, and I'll be starting my Phd. There in the fall, working once again with Chris Potts. So I do. Causal analysis of language models. But it's kind of generally excited about interpretability and how it intersects with linguistics. What kind of insights can we draw from language models?
David Bau: Yeah. Always happy to chat about research. And I will be chatting about tutorials.
David Bau: So I'm just gonna quickly explain how I'm thinking about reorganizing the tutorials
David Bau: on for the insight website and then might not even go through the Walkthrough, but just have like a call to action for this right hand side applied examples that I'll talk about. So
David Bau: take take, go ahead and take some time. It's okay. But I wanna, I want to emphasize the call to action because because that's that's relevant to this code here.
David Bau: So
David Bau: right now, insight has just a tutorial for different methods based on what we thought users manager doing. But there's no clear kind of structure around it at the moment. Right? If you're a new user. You should already know what activation patching or Das is. And then you're just looking to see how it's implemented in insight.
David Bau: What I wanted to try to do is actually pretend. Like you're not necessarily familiar with these methods. You might know what language models are, what activations mean. But patching and causal analysis and logic lens might all be new to you, because these are recent papers, right? And even if you kind of read the paper, insight is very well designed such that you can actually learn from the implementation itself.
David Bau: So
David Bau: this left hand side is kind of what I'm calling walkthrough tutorials or a curriculum of tutorials, or we'll take the main tutorials and then say and reorganize them, based on categories and subsections to kind of have a flow for new users coming in who want to learn about
David Bau: current interpretability methods.
David Bau: And that's cool and all. If you'll kind of
David Bau: time to get started right or want to use insight for something specific
David Bau: Another complete different set of tutorials that I think would be very exciting would be ones that actually give you hands-on experience with recent findings and interpretability.
David Bau: Right? Because these methods aren't just designed to be methods. They've usually discovered something exciting, right? Like, I don't know linear representations of truth and models, for example, right? Or the fact that llamas seem to think in English, even when they're translating from French to Chinese, or something like that.
David Bau: And so giving users the ability to actually quickly recreate these results. You know, on just a 1 billion parameter model or something hosted on indef with one or like a handful of examples that they can quickly iterate on, is a really cool way to
David Bau: play with actual results for recent papers. And I think it's also a cool way for researchers in the field to kind of
David Bau: advertise their work right and get people to think about what it is they've been discovering. So I'm actually going to start by talking about these applied examples that I'm going to call mini papers, for, like the the rest of this
David Bau: the vision here is, I want
David Bau: researchers to be able to kind of take the recent paper
David Bau: and give the gist of their main results right? Because, you know when you wrote the paper, you had to make it rigorous and scientific, and emphasize, you know the the key takeaways, or whatever. But that might not have been what actually got you interested in the project right? Or it might not even be the most interesting result
David Bau: that you got right. And oftentimes it's not even what you're thinking about at the moment. So oftentimes the thing you're actually presenting in the paper. And then the discussions you're having about it might be 2 completely different things.
David Bau: Tutorials are a great way to capture that later part. Right? What you're actually thinking about at the moment. What are the interesting takeaways from your paper.
David Bau: So the idea is to kind of add, I don't know visualizations, animations, but mostly just interactive examples
David Bau: for you to share what Union founded
David Bau: doing your research project here on the right. I just kind of included a few
David Bau: animation kind of exemplifying a discussion that
David Bau: David Bau had in the group chat about difference in means and why it works. If you go to the geometry of truth, paper that introduces difference in means, there's this really nice figure 4 that tries to convey the same information.
David Bau: But you know, due to space constraints, whatever. It's a single figure. It's kind of dense. It's pretty, Mathy, and it takes a while to pulse. So this is exactly the kind of thing you can do with a tutorial. We can kind of explain what's going on. So what I'm showing here is, imagine we have kind of orange and blue dots in that paper. These corresponded to representations of true statements versus false statements.
David Bau: and you can see that they're easily clustered, which is kind of cool. That means that you might guess that the model is representing them linearly.
David Bau: and if you do that as a researcher, one of the things that might go to immediately is okay. Let's train a linear probe to actually cluster these things.
David Bau: and you might then, like a logistic regression, probe it'll find the line of best of like most separation right? Which is that purple line you see on the left.
David Bau: And you get a clean separation. But what's interesting is that that direction of separation isn't necessarily meaningful.
David Bau: In fact, it's probably not going to cause the model to behave in the way that you want.
David Bau: What I'm showing here in the purple arrow is kind of. If we were to take that linear probe and use this direction as an intervention in order to steer the models. And you can see that what happens is that it doesn't perfectly match the distribution.
David Bau: There was a pretty fun story you can tell here about the fact that correlation just gets in the way.
David Bau: So so this is a cool kind of example from from David, and so I'll go into it just a little bit imagine the story. Here is something like
David Bau: you're a teacher. And this is your 1st year teaching this class.
David Bau: And you're keeping track of just kind of 2 things
David Bau: the number of assignments that a student is done in, and their average grade on those assignments.
David Bau: and then the blue and orange actually correspond to whether or not they passed the final exam. So everyone in blue turned in a lot of assignments, had a pretty high average grade.
David Bau: and they passed everyone in orange didn't turn in a lot of assignments, had a pretty low average grade. But you can also see this correlation with people who turned in more assignments had a higher average grade.
David Bau: like, okay, linear regression probes that just had to find the best line of best separation
David Bau: pick up on that correlation.
David Bau: You can see that the purple line actually picks up on the fact that the average code kind of goes up as the number of assignments.
David Bau: 1010 goes up.
David Bau: But if you want to, then do an intervention based off of that.
David Bau: you're actually going the long way, because
David Bau: while both number of assignments and average grade can tell the same story, you can see the average grade is actually just enough in this case to like completely separate
David Bau: the 2 clusters.
David Bau: And so, if you were to use that as your proxy, and to be like way to correlates with this other thing that we want to ignore, you end up doing this weird adjustment where you say, Okay, let's raise everybody's guides up and have them turn less assignments in
David Bau: right, and that will help them all pass the class, which is
David Bau: maybe a little bit counterintuitive, because you can tell that turning number of assignments actually correlates, you know, with a higher grade and helps students pass the class.
David Bau: So this is why, having
David Bau: unfounding correlations here actually causes a problem for this intervention.
David Bau: The story you might want to be able to tell is more actually.
David Bau: how do we make sure that these 2 groups of students end up lining up based on the metrics we've captured about them. Right? So in this case, for example, difference of means will literally just take the mean student who passed the class and the mean student who failed the class and just looks at what's the difference between them. And here you can see that it would actually push in both directions.
David Bau: So you'd want students to turn more assignments. You'd also want them to get a higher grade on these assignments. So it's kind of a fun story to tell. But but my point here, like the point I'm trying to emphasize here is that
David Bau: this figure exists in the original paper, right? But the original people couldn't tell
David Bau: this whole story with like a professor trying to improve the class, or whatever.
David Bau: because they had to submit to a conference right?
David Bau: But with these tutorials you can go into the examples that you care about more. You can give intuitions behind. Why, some methods work and some methods don't. And it's a great way to kind of
David Bau: get people thinking about the same thing you're thinking about, don't you think is like half of research? Right?
David Bau: with that I'll just kind of
David Bau: a little blurb on the ones we have so far. So we kind of well, probably releasing 3 of these
David Bau: soon. So function vectors is one that already existed on the inside website. We'll just kind of repurposing it. The geometry of truth is the one I just talked about. Robert Sorres was really cool, and wrote his own tutorial, for how Llms use their depth or really don't use their depth.
David Bau: we have a really cool one on memorization in language models. And how you can tell which tokens in the input actually trigger memorization. And when you can tell that you can do funny things where, if certain tokens don't trigger memorization, you can completely mess up a sentence, it'll still be fine. So a lot of Mmlu questions at this point are memorized.
David Bau: But their numbers might not be the thing that trigger normalization. If it's a math problem. It might be more the phrasing. So you can actually ask a model. Some question from Mmo. Completely change the numbers in that question, and it will still give you the same response.
David Bau: That's 1 of those kind of
David Bau: neat findings that again didn't make it into the main section of the paper. If you go and read the appendix, it's there, and if you go and talk to lisaulture
David Bau: they'll have cool things to say about it. So this is kind of their chance to showcase it.
David Bau: We have one on look backs.
David Bau: and then this is my call to action. We want more, and we want it from you all. So
David Bau: if you have a cool paper that does anything related to interpretability that you want to talk about, and better yet, you think you have like a fun, just minimal example for
David Bau: talk to me. Please. You know we can work together. I can start drafting it, and we can have a chat if you already have something up even better. But I think this is a really great chance to kind of connect between, you know.
David Bau: teaching and developability to new people while also
David Bau: getting your own work out of there right and having people think about it.
David Bau: So so keep that in mind.
David Bau: I'll pause here.
David Bau: Before continuing, because the next thing I wanted to talk about is actually like the main
David Bau: tutorials. And how are we thinking about reorganizing them? And that can be a really long discussion.
David Bau: So
David Bau: so I think when I was 1st listening, I misunderstood sort of the attention, and I got really excited about something else. But it seems really because this is really actually in my mind, super cool, because it gets closer to research, becoming something that gets
David Bau: like rigorously evaluated and has some sort of like version control aspect where you can standardize like, okay.
David Bau: here's in the standardized library format exactly what I did. And I can now control. For things like, you know, different models. I can maybe control for like different data. And you can kind of run more like larger value like replicate, you replicate results and see how much, how much they generalize.
David Bau: I think that would be a really cool vision. The the way I was thinking about this originally is like something like a kaggle dashboard, you know, which is, I think, kind of what what you were saying right? This idea that
David Bau: not only can other people actually access and use it.
David Bau: they can build on top of it rigorously, test it so on.
David Bau: but that might be asking a lot of researcher who already has spent time kind of creating their own code base. So I'm trying to start like a little bit more minimally than that. Just kind of say.
David Bau: let's say, create. You know, just that main figure from your people.
David Bau: And it doesn't have to be a perfect creation, right. It can just be like the gist of what's going on there. And that should be enough. We so chose to be like, Wait, okay. So if I just tweak this value.
David Bau: Do I get a different result or something else? Right? And then maybe they have a chance to follow up with you. Maybe they do something that spins on top of that
David Bau: and we're heading towards that field of, you know, mutual kind of testing validation.
David Bau: Do you have a vision. For what would incentivize a researcher to create a mini paper like if you were to try and sell it to them? So
David Bau: why should they make them any paper. By the
David Bau: you. You all tell me I I gave you my pitch.
David Bau: They can be like a blog post for someone who, I guess doesn't already have it. Or they can just link it. Be like, okay, yeah, here's a tutorial.
David Bau: Yeah, I'd also emphasize that this is more interactive than a blog post, which I think is kind of cool.
David Bau: Presumably, if you wrote a blog post about what you did, you needed to run some small code to recreate an example.
David Bau: If you share that code online.
David Bau: people can immediately play around with more than just that one example, you put on a blog post site.
David Bau: Also, a lot of people like frequently visit the insight website from a lot of different backgrounds. So you might get a lot more visibility on your research and your work than from your own blog posts or in your own like website or labs website. Maybe
David Bau: I'll be upfront here and say that I think the thing that will encourage researchers to do this the most
David Bau: is, if people are already doing it like.
David Bau: if we have like 20 mini papers already up
David Bau: and you go to the Nsa. Website. And you see that it's like, Oh, they're getting traction. This could be something cool for me to do.
David Bau: It's more likely it kind of like, you know.
David Bau: picks up. But but we have to kind of get the ball rolling.
David Bau: I have a pretty good number.
David Bau: you know, if y'all can help me get to 10, that'd be amazing.
David Bau: I think there's a couple of there's a couple of other things that we could motivate, you see? Because you answer, I think
David Bau: I think it's you know the standard right now, if you want to release a piece of runnable code
David Bau: is, you try to see if you can get it to run in Collab.
David Bau: then people will just read a collap thing.
David Bau: But then, if your models that you want to demo on.
David Bau: or just a little too big.
David Bau: and you can't do it on Collab anymore. And then people will just put something on Github and say, Well, I guess you gotta go find your own Gpu and do it right.
David Bau: But but you know, if if you do it this way. Then
David Bau: then, you know, you can do it on larger models, but it it may be
David Bau: the process of putting a demo up here
David Bau: should also allow you to get your like Kareem's models on the White List, or something it's like, Oh, you know, to Demo, my thing.
David Bau: you need to, you know, have access to all 3 checkpoints. And
David Bau: but you know, if you're if you're a mini paper contributor, maybe that that puts your models on whitelists so that people play with them.
David Bau: That's great. Does that make sense? Yeah.
David Bau: yeah, I think that there's something about.
David Bau: So I I agree it's about the community. I I think that
David Bau: it. Maybe if if we can somehow normalize the idea that you know the best papers
David Bau: come with a runnable demo and end, it is devoted to providing everything that you need to have.
David Bau: Yeah, at least your figure won't be demoable.
David Bau: If that would, if that would be quick.
David Bau: Not so Cuz asked about
David Bau: links to these kind of tutorials. I can start sending them out. But also.
David Bau: if you wait for an announcement coming hopefully Monday, we will just put them up on the official on site website. And just, you know. So
David Bau: sending out the announcement for people to to start looking at these links, that's
David Bau: yeah. That that's something I was considering kind of actually going through in this presentation. If people interested now I can do it. I imagine that, like another big
David Bau: block of is just okay. What exactly do I mean by Mini paper? Right? And I'm like.
David Bau: kind of gave a vision of what I mean. But it's held without an example. So that's that's something that hopefully, all of you can just like start looking at soon, because we'll have some examples up. And that's something I'm also just kind of happy to chat about if you hit me up, or we find a time to do it.
David Bau: you know, once you have a template played. It's a lot easier to kind of think about how to structure you all things with relation to it.
David Bau: Cool crescentalex
David Bau: life, Chris, let's let's chat.
David Bau: I I'm have a different perspective. I think a lot of other people in this room, because I'm kind of new to the this world. And I actually use the tutorial page a lot and I have been recently as well. But I was just saying, I think it also be cool, because
David Bau: a lot of people that I know that aren't into this stuff as much they like. Read a paper from anthropic or open AI. And then they're like.
David Bau: Oh, this is a really cool paper, but they can never recreate it, you know. So maybe looking at papers that get a lot of traction on Twitter or something like that, and then adapting them into a cool little tutorial like this on the end insight website.
David Bau: Could be like a really great way to, I guess. Add more Mini papers. Yeah, I really like that. I guess that's something I should have said as well if you have an example of a paper that's not your own that you would want to see a walk through for
David Bau: also come, talk to me. I'd be very happy to
David Bau: to do that. That was
David Bau: The geometry of tooth is kind of like the closest one I have to like a people that I think has gotten a lot of traction like a difference in means. Stealing is kind of like gets talked about.
David Bau: But yeah, I think you're definitely right about the kind of topic. Open AI papers.
David Bau: yeah. Or you know any number of organizations definitely.
David Bau: Do you think like a good metric of like, how satisfied your goal is would be. If, for example, David could teach a class where every single lecture had a Mini paper, a relevant Mini paper for it. I mean, yeah, I wouldn't be mad at that for sure.
David Bau: That's about. That's about 27, many people
David Bau: you do that.
David Bau: You took up 20 of the the allotted 30. We can
David Bau: can keep going with the walkthrough. You can show a little bit of like the walkthrough. That's okay. You're only here for, you know, a few weeks. So so let's let's let's let's see the walkthrough cool. Yeah, just
David Bau: this is meant to be like a discussion. So I'm gonna put up like an overwhelmingly big slide, and then we'll just start talking about it. Okay.
David Bau: can I add one small comment, yes, oh, yeah. Sorry, David, in his deep learning class. You're like making us do some Demos for some of the papers, right? Maybe, like some of the once you like that, you should upload them to the paper. Yeah, that'd be amazing. While we're still on this, the other vision here. It doesn't have to be a published paper right? Like, kind of with kaggle dashboards. It might just be a student project. You did, too. Right?
David Bau: I'm kind of calling these mini papers for now, because I think it sounds cool, but
David Bau: I think the idea long term is really anybody who wants to contribute a tutorial showing something they've been thinking about. It's kind of a cool discussion Forum on this right where before this, you know, on the discord, you'd have these like long threads. And you're sending code blocks, and you're linking your github now, you could just
David Bau: put this tutorial up on the onsite website and then link people to that right
David Bau: as a way to get a discussion going. So I think student projects would actually be a really cool way to do that, because you'd have a collection of them. And then you can just ask students, Hey.
David Bau: are you willing? Are you cool? If we put this up on the insight website as an example, something you could do?
David Bau: Yeah.
David Bau: So that might be something we can. We can keep thinking about
David Bau: cool. Alright. Let me. Yeah, this is the the overwhelmingly big
David Bau: slide to talk about. So now we're coming back to just
David Bau: alright. This is what I talked to before is kind of, like a list of many peoples that are not necessarily related in any way.
David Bau: Now let's talk about the actual main insight tutorials and how to organize them.
David Bau: And I tried to come up where they sort of
David Bau: categorization of how I think it might make sense to talk to somebody about interpretability if they're new to it. This is not
David Bau: an ordered list of things. It's not like you should start on the left and go to the right. I imagine each of these as kind of separate tracks.
David Bau: depending on what your purpose is.
David Bau: and roughly they correspond to probing on the very left. So that is given, you know, representations of the model. What interesting information can we decode from that
David Bau: in the middle it's stealing. So what manipulations can I perform on the model to control its behavior in some way?
David Bau: And then causal mediation, analysis, causal abstraction, something something causal, you know.
David Bau: grounding what you're doing in theory to actually understand
David Bau: what these are presentations mean. And these are the distinct things. And I'm happy to have like long conversations about these. But the things under them
David Bau: can definitely share categories right?
David Bau: Like. I'm not fully convinced of any
David Bau: anything I've done here so far. And I have this like spreadsheet while I'm organizing these, and I move things around like every 5 min. So this is more of like a cool discussion with people who
David Bau: but you know, have all been thinking about interpretability for a while. And now that
David Bau: call different levels like in the field, and from like somebody who's pretty new to somebody who's
David Bau: deeply experienced in it.
David Bau: Could just be something fun to discuss right if you
David Bau: rolled back the clock and had to reenter the field. How would you think about teaching yourself interpretability? Right?
David Bau: I can just kind of keep talking a little bit more about examples within these, if that would be helpful. So, for example, for predicting this idea of probing, you could train linear probes. You can do mean mass probing, which was that example I showed you. You could just
David Bau: not stand a probe and use the model's decoder as your probe in order to read intermediate values, and that that's kind of like the lenses, right? Like the logic lens.
David Bau: You can think of a logic lens as a linear probe that's just defaulted to the unembedding layer. That is
David Bau: what it does.
David Bau: It's just a multi-class probe with the classes the size of your vocabulary right? And you're trying to decode
David Bau: what the next login might be.
David Bau: Yeah, come on.
David Bau: yeah, in the middle control. I'm mostly thinking about stealing, but in my mind I think stealing is actually really
David Bau: related to the just idea of adapters on models like.
David Bau: like, you have this steering vector that's an external thing. And you're using that to change your model behavior.
David Bau: that steering vector can also be a learned representation. And now you have left, you're familiar with that. It can also be a learned weight decomposition. And now you have lower
David Bau: right. These are all ways in which you can just kind of have an external adaptation applied to the model in order to control its behavior.
David Bau: And then understanding is, I could talk about this one for a while. But causal analysis is really cool.
David Bau: There's lots there, right like you can talk about patching. You can talk about causal alignment.
David Bau: I kind of tried to wake it up into these 2 columns, because I think that's how a lot of people think about it in a gist. It's like
David Bau: you can either look at what activations are important and then tidy into it with them.
David Bau: or you can come up with your interpretation about what the model is doing, and then find the representations that correspond to them.
David Bau: And roughly, these kind of correspond to what I'm going to call causal mediation, analysis, and causal obstruction.
David Bau: Good.
David Bau: I know it's kind of funny, because I'm like throwing this at you after thinking about it for a while, and I'm like, what are your thoughts? But you can also just ignore this completely, you know.
David Bau: and throw in things that would be cool to learn about right or cool to teach.
David Bau: If you haven't think I really like the control section
David Bau: like the way you categorize everything. But like, I guess.
David Bau: going into this, I think that category right now is really applicable to what I want to do. And so if I was going to learn, I would just look at that column just like this is a tutorial. Okay, I'm glad to hear. Yeah, this. This is definitely something I was thinking about, just like
David Bau: you might have different purposes, right in interpretability, like I think predicting is, he's super
David Bau: interesting thing. You could just be doing without thinking about how it actually controls the model? Right?
David Bau: And you can try to control without trying to understand.
David Bau: Did you already make these? And are they going to be released on Monday, too? Or is it just a new paper?
David Bau: This is the grand vision. Some of these are already up. And on Monday
David Bau: I will stop color coding this.
David Bau: for the causal stuff is the goal to just like outline different broad areas of causal interventions that you can apply on models like standing like like you could start like neuron. And then like
David Bau: collection of neurons and like the whole
David Bau: output and would be like, I guess the question is like, would you be kind of trying to find like what the researchers have converged on focusing? Maybe like single neuron stuff, you just like point through in that tutorial, you point to a couple of examples of like where a single neuron like does a big change. If you positively ablate it and and and see what it does.
David Bau: that's a good question.
David Bau: And like.
David Bau: it's yeah. I've been thinking about it for a while, and I finally converge to your wave, which is crazy. Yeah. So the way I'm thinking about doing it right now is kind of having an single underlying task, which is the the level task which is, imagine I have a sentence like
David Bau: The city of France.
David Bau: The city of Paris is in France.
David Bau: That sentence is kind of saying, the city of Paris, which is like my entity.
David Bau: has lots of properties. The one I'm talking about is the fact that the country is flat on site.
David Bau: And what you kind of learn with activation patching is, you can actually
David Bau: into the plane so that the city of Paris now goes to Italy right?
David Bau: Even more interesting though, is, you'd want to be able to do things like the city of pillars.
David Bau: Change the country to it like, say, Brazil, but keep it so that it's Europe. So we're only editing the country without editing, say, the Continent or the language right? And what I'm thinking is showing how different causal interventions kind of get you closer and closer to that.
David Bau: because if you just do the residual slam you'd be able to change the country, but you'd also be changing everything else.
David Bau: If you do. Attention heads, you might get close
David Bau: linear representations. You get even closer.
David Bau: that's that's the gist right now. It's almost like a Mini paper for the big survey paper that the mediator
David Bau: exactly. It's it's always hard to kind of like
David Bau: break these up between the applied and the curriculum section. So that's something thinking about as well.
David Bau: So what is the plan? Each of these mini papers will have one of these color codes
David Bau: like predict control and understand many papers can kind of pick and choose right a lot of the time they get rid of both for many people they can have like tags.
David Bau: exactly like the way I'm thinking about it is, each of these is like a Mini lesson, and a mini paper is an applied example across
David Bau: things like like, once you've picked up the right skill set, you should be able to do this. So
David Bau: so when you say something like a curriculum, I imagine, like something like a something like a syllabus like there will be an index of things, and once you click on that index, it will expand. And then there will be like multiple tutorial series. Under that. Like, is that the plan? Yeah, yeah, this is what I'm outlining here. But that plan is separate from the Mini paper at the moment.
David Bau: So okay, and if it's supposed to give research access right? It's not only interpretability.
David Bau: So a lot of the times when I see like this control papers.
David Bau: I don't think of them as interpretability, but
David Bau: because as long as something is connected to the computation graph.
David Bau: you will be able to mess with it to make the model say anything
David Bau: as long as you are making a strong enough intervention, right? So maybe it should not be a curriculum for interpretability
David Bau: should be something like, or unless like, this is something. You you are really strongly
David Bau: motivated by, like it probably should be like a
David Bau: I don't know. Curriculum for things. Can you do within? I mean things can you do with model internals?
David Bau: That's fair. I'm I'm very willing to reframe this as curriculum for research with model internals
David Bau: which is a what I imagine insight is time to provide. I think that we can. We can take take ownership of the word interpretability and define it as we like. I think.
David Bau: You know the idea that you wanna control some, you know, change some behavior.
David Bau: and that the best that you want to do when you're doing that activity is just change it at random, I think, is is
David Bau: It's probably not a very generous way of looking at what those researchers are trying to do I think that you know that if you're trying to control the system, you try to control it in some predictable way, and eventually they're they're learning to interpret. You know what they're doing.
David Bau: you know, they're just using different tools and interpretability from another perspective.
David Bau: using a different set of abstractions.
David Bau: And so I I, you know I do have. So a a general question is, are there methods that you think
David Bau: are not represented here, that you think a lot of people
David Bau: would want to learn about or read about.
Chris Wendler: In my experience, like people that have, like the standard machine learning, deep learning, training.
Chris Wendler: always have, like a similar set of concerns when they talk to interpretability researchers right? And
Chris Wendler: one of these concerns is like.
Chris Wendler: why would we bother studying like a single checkpoint?
Chris Wendler: If we have no guarantees whatsoever that this thing generalizes to even a second checkpoint
Chris Wendler: trained on the same data, just starting from random a different random seat. Right? That's the standard question that I get from people that haven't tried doing any
Chris Wendler: any of this kind of research.
Chris Wendler: So I think that that one should be addressed somehow, like a in a conviction building section, where you actually show that a lot of the results kind of
Chris Wendler: transfer across different models.
David Bau: Yeah, that's interesting.
David Bau: So there's there's some. There's some like framing material that's beyond just methods
David Bau: that that's sort of not represented here, which is interesting.
David Bau: Yeah, I really like that.
David Bau: The the closest thing we have to that right now is that the existing tutorials we have often go through a small model, and then at the end give you the option to use, and if to replicate this on a large model.
David Bau: and you can see that the results transfer. But that's never made explicit anywhere. So I really like the idea of
David Bau: like making that part of the introduction to that site like to these tutorials as a whole.
David Bau: Do?
David Bau: So I'll I'll name 2 random methods.
David Bau: Okay, you know, one would be like salience methods.
David Bau: Do those live in any of these categories here.
David Bau: If you're talking about attribution patching, then? Yes, otherwise. No.
David Bau: But I think you're totally integrated ingredients, shop line policies.
David Bau: Okay, what? What are your thoughts on on those, because they don't have to do with model and tunnel site.
David Bau: Well, they, you know, in order to compute them, you need access
David Bau: to model internals, sailency maps right here. Yes.
David Bau: yeah. And and a lot of applications of these. They're not just work, you know, input to output salience. People will also
David Bau: do up to instate salience or in state. So they will. They'll do salience over
David Bau: parts of the computation graph which I think are interesting, which which you know. So which which
David Bau: you know you would.
David Bau: It's more direct directly, you know, aligns on model internals when we do like, yeah.
David Bau: yeah. Have you ever met Gabriel Sarpy?
David Bau: Yeah, he's he's not starting with us until January, but he's on the is. He's in the end of the community. It might be worth it.
David Bau: Conversation with him because he thinks those things. I don't.
David Bau: Yeah, this is a good time to say, too. I have, like a very strong language, model bias. Here I am a linguistics so like, I think, saliency. All of these are very good formation models. Well, that's at least I don't have. That's the heritage. And so the funny thing about Gabriel is trying to.
David Bau: He's he's he's trying to make it a thing for language models to right. He's been trying to popularize that. It is kind of making a comeback to attribution. Patching like attribution. Patching is a saliency map. Yeah. But you know, even even just direct application, integrated gradients. I think you know you could argue
David Bau: that. That it's worth a look, and he's been. He's been spending some time on that.
David Bau: It's it. Yeah, I feel like it's a natural question, and something that maybe is more.
David Bau: something that interdisciplinary research was well at. So say, can you?
David Bau: Yet they want to know
David Bau: when they ask why they don't want them to say, Let me send that to
David Bau: the other. The other thing the other method that I'd ask about is is
David Bau: influence functions. Oh, yeah, do influence functions live in any of these categories. So there's a whole, like
David Bau: separate section of
David Bau: cleaning and influence that I'm just not covering at all right, like you can think of this as just literally a single forward pass.
David Bau: When people use these models, there's the whole training part that comes before that forward pass. And then there's the whole inference part that comes with.
David Bau: you know multiple forward passes.
David Bau: and there was a lot of work to be done like we socialize on both of these.
David Bau: I think that a long term I'd want to get to those as well. Influence functions live on the left, right? And I have lots of thoughts on sampling on the right. But
David Bau: And that's something that I've been thinking about. Just kind of putting.
David Bau: Can I say something? So both valiancy maps and inference functions? They are approximations of causal mediation.
David Bau: Right?
David Bau: So I mean, we just cannot do causal mediation on them, because it's just takes so much time.
David Bau: I mean, and we then we technically can just put them on cross notification or approximation from that meeting
David Bau: that does give a nice pedagogical branch of like. Not every causal thing is is achievable. So we approximate it.
David Bau: Yeah.
David Bau: So you guys should take a look at the material as it's released and give Amir feedback.
David Bau: Yes, please, feedback.
David Bau: Yeah. Thank you. So.