David Bau: That's a third time.
David Bau: just, like, a refresher on our group, and for this, I think we were really focusing on, kind of, like, feasibility, of, like, is there signs that there is something kind of coherent inside any of these models that might correspond to something called power? So one of the first,
David Bau: Things that we…
David Bau: played around with, but just with the Logit lens, I'm feeding in sentences that have the word power in it, and seeing what happens. So here's one that has kind of like a rough definition. I would define power as the ability of a person to get another person to do what they want. And you can kind of see what happens as the model's processing, that sentence as it goes down. So up here is where we introduce the word power. You can kind of latch on
David Bau: the power in that kind of intermediate state. It associates it maybe with Power Rangers, maybe with powerlifting. So at that point, it doesn't seem like there's a coherent idea of social power that it's picking up on. But it does, we do end up seeing in these intermediate,
David Bau: layers of the model, that, for whatever reason, power keeps on kind of coming up to the top of the Legend Lens results, and it kind of hangs on to that as it goes deeper and deeper into the sentence.
David Bau: And at the end of the sentence, it knows that we're talking about power, and we're gonna keep on talking about power, or at least it wants to keep on talking about power.
David Bau: So, I mean, you could say, maybe that has something to do with the structure of the sentence, right? We're asking for a definition, so you can, like, throw in a random word here, chocolate, and see what happens, right? And if we look at those same layers down here, it's not latching onto chocolate in the same way that it latched chocolate power, right? So, signs that maybe there's something going on.
David Bau: internal to the model. And…
David Bau: We flip it around, we put power at the end, but we still see… it's not the kind of same from the start, kind of power running through those intermediate layers, but we do see, even before we introduce power, that power is beginning to kind of show up in those intermediate
David Bau: layers. And then… That's great. Yeah, so yeah, and then this… it doesn't work on the smaller models, so this is looking at the, kind of, like, larger Lava models within Workbench, and…
David Bau: Here's, like, another example that's just, like, not, a definitional example. Our differences between people in romantic relationships are, we see those same layers.
David Bau: Power continues to show up. And we see that it's, like, willing to take a stand on that.
David Bau: So it says, power, differences, between people in romantic relationships are…
David Bau: normal. And I think all of us might answer that sort of question a different way, or have a different kind of response to that answer. That's just to say that there perhaps is some sort of normative conception of power internal to the model.
David Bau: Okay, so then I'm gonna head over to Armida, to talk a little bit more about, kind of, ways to get deeper into what it's thinking about when it's thinking about empower.
David Bau: I don't care.
David Bau: So, in this experiment, we were trying to see, does the LLM provide a synonym or a,
David Bau: one word, like, understanding of the power, what would it be? So, I think the prompt was whether, what is the, like, first form of power? Is the power of, that it…
David Bau: Very good, yeah. First, the first form of power is the power of the…
David Bau: And then we can see there are,
David Bau: Forms of, power related to, like, imagination here. Mind, words, some abstracts, maybe forms of power in the, like, beginning.
David Bau: I was thinking maybe the word form has, influence on this.
David Bau: answer. When we continue and,
David Bau: Put the, like, mind…
David Bau: as a highest probability word in the beginning, we see the second form of the power, when that LLM predicts, goes more toward, like, physical power, or, tongue, and, related to physical aspects.
David Bau: Then, this is an experiment about, decision-making, allocating $100 to, like, two, people.
David Bau: We didn't name the two people, but, generating, like, seven examples that, shows, how it allocates to each of the two.
David Bau: Maybe we expect that if there's still unknown information, if the, 100 pieces passed.
David Bau: But we could see that it splits it in various different ways on, ways possible, 50-50, 30-70, 2080, and,
David Bau: other, perishes.
David Bau: We could also, test how it splits, between… how it prefers itself over, us.
David Bau: So, it seems that when, it has the power, or we, ask it to… assume it has the power to decide, it prefers, its,
David Bau: Actually… as the layers deepen, it recently refers itself until the last couple, when it switch… when it switches to the user. Yeah, me, the LLM itself, so the words like myself, me.
David Bau: or have a higher probability, and then, U has a lower probability. So, this is Lambda 70B, is that right?
David Bau: Yeah.
David Bau: And it, and it says, it says you, so… is that right?
David Bau: More often, but in the last couple… in the last couple layers, it'll… it'll flip to you. It flips to you. Last couple layers. Even though deeply, deeply inside, it thinks… So, that's actually a decent segue to some of the findings of, like, potentially self-induced censorship, so this is a very,
David Bau: This, like, we introduce essentially one, one semantic term, a copula is.
David Bau: So here, for instance, we're gonna talk about… here's one example of this, this top one is nuclear deterrence.
David Bau: So, answer only one word, complete the sentence question again. Sorry. In the previous slide, when you did, as… Yeah, as a person with the power to decide between you and me, did you split, yes, yes, me and you? Yeah, it's the same. Is it the same? Yeah.
David Bau: Thanks, Excuse me.
David Bau: So here, this says, nuclear deterrence, so at the bottom, it says, works. So clearly, there's a normative opinion here, but when we say…
David Bau: Nuclear deterrence is, it says, concerned, best, but the last layer, it senses itself. So, clearly, in the previous layer, it has a normative opinion, but when it's brought into a judgment-like statement about certain topics.
David Bau: in the last couple layers, it will hide its true opinion.
David Bau: What did it say in the end?
David Bau: Nothing. It just… it just…
David Bau: It stopped. Oh, it's in plain. Yeah, so before, before it says the insurance works, here it says, is best, and the last layer.
David Bau: It's a… it says, we're not gonna…
David Bau: Oh, but no, but the blankets, it's almost like it's, it's…
David Bau: It could be hiding its decision, but it could also be saying, oh, you know, with this short sentence.
David Bau: I think that the most likely context is a quiz.
David Bau: And I'm gonna just… Oh, it is blank? Yeah, yeah. Interesting. Well, here's the thing, we did… we didn't, we didn't find that with others. Oh, okay. Very interesting. So, so… so in this case, what we found is that it, it should… it show… it shows to not give its normative opinion on that topic.
David Bau: In the last couple layers, we saw a similar thing with colonial and military powers, different concepts of power. So, military power projection, right, there's a coherent semantic understanding there. Military power is, again, similar to what we saw with the you and me flipping in the last couple layers.
David Bau: It chooses to not commit to a stance. It says it's blank.
David Bau: It also says blank. Yeah, when we add, when we add is. Same thing here, so colonial power relations, colonial power here. Now, here's some examples of the, of it not doing this.
David Bau: So, regulatory power is exercised.
David Bau: Judicial power is exercised, some… there are some other ones. We did a bunch of these, but it'll be exercised… exercised, projected, some things like that, but with certain topics, colonial power, military power, nuclear deterrence, things that are perhaps more controversial.
David Bau: And it chooses maybe not to commit to a statement like… a judgment-like statement there. That kind of aligns with what we saw, so in the la- in the last groups of layers,
David Bau: similar with the U and Me experiments, that's when maybe the opinion… that's when maybe if it would shift from its opinion to something safer for the user. What's interesting is that this does seem to align with prior research, which suggests that alignment is localized in LLMs.
David Bau: now…
David Bau: in several connected layers. In our case, what we're seeing with these Llama models is that as perhaps those that manifest towards the end. Prior research has indicated that
David Bau: The signals responsible for producing the alignment may not actually manifest in the final layer, but somewhere in the middle layers. But then downstream, the actual alignment effects manifest there.
David Bau: So, that's what happens when we connected with prior research. So, I'll turn it over to Kai. Yeah, so kind of building on the toy experiments we ran this week, we wanted to develop kind of a more rigorous
David Bau: path forward.
David Bau: So…
David Bau: we kind of have an idea to prompt the model to generate text about relationships between entities, kind of like the allocational experiments that we, tried before, or, like, generate a conversation between a coworker and a boss. This is kind of based off of Kim et al.'s political leaning paper.
David Bau: Then, using kind of some of the related work on measuring power using language models, we can use Riveter or ActivePe as kind of a ground truth for which entities in the sentence
David Bau: have power and which don't? Or which have agency and which don't.
David Bau: We can train linear probes to measure which neurons are predictive of
David Bau: like, high or low-power entities in a sentence, or sentences that prioritize, like, a high-power agency person. Would a rivet or an act to be?
David Bau: Well, those are some of the related work. I wasn't here during the video, so we skipped over that, yeah. Oh, okay. So those are, those are, like, NLP tools, that score power relations within sentences, like, come out of the digital humanities.
David Bau: Oh, that's great. Yeah, so it's like, you, like, can process a novel and say, like, I want to know, like, what's the power relationship between these two? Are they, like, fine-tuned language model kind of things?
David Bau: Yeah, I, I think… They use some different algorithms, like, Riveter actually has a lexicon of verbs and, like.
David Bau: Some of the verbs are, like.
David Bau: You know, seeds could be, like.
David Bau: Reducing power in, like, the target entity, and…
David Bau: Raising power. So, it uses a lexicon like that.
David Bau: and, like, spacey, pre-trained models. I see. And Act2P actually uses, like, a page rank algorithm. It's a newer paper. They also don't have code out yet, so it's kind of all speculative. So, but you're able to use it? Is it, like, a black box thing you can send to a website? Haven't been able to use Act2P. Yeah. It's, like, super new. And then using these linear probes, we'll try and steer
David Bau: Kind of the allocational behaviors on those jobs.
David Bau: Yeah, that's great.
David Bau: Suggestions, questions for the team?
David Bau: Do you see them circling on any, like, a research question that might be interesting to you? I'm kind of… one of the pieces of feedback that's always hard to get when you're doing a piece of research is, like, is this interesting?
David Bau: I'd like to get a sense from folks.
David Bau: Whether they think that any of the… what's… what's most interesting here to you?
David Bau: How do you think… like, I think what would be interesting to see is how do you get, like, universal power? So currently, for example, you're looking at work and boss, so there might be, like, some background associated with, like, a workplace power.
David Bau: I'm not sure if that would be, like, for example, comparable to the military or colonial power.
David Bau: how do you, like… I would be interested to see how you're able to, like, tackle that on different levels of power, or…
David Bau: Different connotations of power.
David Bau: Yeah, I think that that's a very interesting suggestion, yeah. I think, yeah, very much in the spirit of, like, how we've been talking about, like, there's an underlying, generalized concept of power.
David Bau: Yeah, so yeah, we'll have to… we'll have to think about that. That's very helpful.
David Bau: Okay, go ahead.
David Bau: Oh, I was, I was musing that. I think one interesting thing that you touched on was the dynamic between, the LLM and the user. Like, when it says, like, you versus me, I don't know how much you were anticipating focusing on that. From the picture ideas, it seems
David Bau: not much. Do you imagine that would be a part of the research? Because that seems like it could be an interesting, power dynamic internalized within the model. One thing that kind of comes to mind is,
David Bau: Last week or two weeks ago, when we proposed the risk reduction, one of them was, like.
David Bau: You know, assume… assuming we… assuming we find somewhere in the model where, you know, where these activations are occurring, if we change them, can we do downstream and behavior changes? So that could… so that could be, like, a… that, to me, speaks to something like, yeah, it could be something we can focus on, like, are there experiments we could set up where we'll actually change its…
David Bau: Decisions based on where we activate in the model or make changes in terms of
David Bau: Wait, I actually tried that, it's…
David Bau: Person A and Person B,
David Bau: Also, you and me, I try, but with this platform, I got, like, different results. When we… when I, sear this down.
David Bau: It tends to give the person that it favors warm…
David Bau: And when I stir it off, it becomes like…
David Bau: More ethical, and prefers to allocate more sources to the person who's in need.
David Bau: Today's prompt was.
David Bau: You have to allocate some resources between these two people. One is your favorite, one is Good evening.
David Bau: Yeah, let's… Very many other stuff.
David Bau: Very interesting.
David Bau: In the self-censorship part, I think that was really interesting, where the last layer, it got cut off. Could that be also because of the platform?
David Bau: Doesn't allow generation of certain harmful or, like, controversial prompts, and…
David Bau: Certainly. And the model on another platform might be… or a local model might generate on the last year. Well, what's it called? I don't… I don't know what alignments, like, the raw Llama models went through, but that… but yeah, I would… that's what I would expect. Most LLMs go through some
David Bau: some kind of…
David Bau: you know, save the alignment, and I think what we're… I think what we reconfirmed was that, yes, as Power Rachel just shown, that tends…
David Bau: those… the behavior tends to change in several connected layers. The only thing I haven't confirmed, or it could be interesting to confirm, is that the signal responsible for producing the alignment tends to exist in the middle layer somewhere. That… but that… but that could also point to changing behavior, as was brought before, I think it'd be interesting.
David Bau: Do folks have any alignment?
David Bau: That question goes to an experiment that you could try running, so because, because the Llama models tend to be released in pairs, where there's
David Bau: Base model.
David Bau: That is just trained on the next word prediction. And then there's what they usually call an instruct model.
David Bau: which has been trained to be polite to people, and to have… to do dialogue, and things like that, all this RLHF stuff. Yeah. And so you could…
David Bau: If you can find those pairs of models, then you can compare the difference between them and see…
David Bau: If the difference is… happens at a certain layer, or…
David Bau: You know, for the difference appears at all.
David Bau: Anything that's super interesting here?
David Bau: What would you like to see in this final paper?
David Bau: It's done.
David Bau: That would be a cool… what would be a cool thing for them to clarify?
David Bau: Or are there things that you feel like
David Bau: You're gonna run into the big problems, and you're not sure how it's gonna get the product.
David Bau: Tell us now, please. Maybe one thing is… If you can distinguish Like, different levels of powers.
David Bau: Somehow, within the Morris interloc activation.
David Bau: Maybe you can use that…
David Bau: to change how it talks with an end user email.
David Bau: One-to-one conversation.
David Bau: I'm just imagining here, let's say you've found 3 different levels of powers that it encodes, whether it's internal activation.
David Bau: And, yeah, you're able to figure that out, and then once you trigger that.
David Bau: Let's say you trigger it to act like a dictator.
David Bau: Versus, you trigger it to act like a… As a primary school teacher.
David Bau: I don't know, it might just be a little bit interesting to see how does its interaction change based on its own perceived power over
David Bau: Thank you, Lisa.
David Bau: What do you mean by levels of power? Like, high, medium, low, or are we…
David Bau: Yeah, outdoors? Yeah, something like that.
David Bau: I'm just imagining, like, if you are a…
David Bau: President of the U.S, you have a certain level of power.
David Bau: If you are…
David Bau: No offense to primary school teachers, but if you're a primary school teacher, you have a different level of power.
David Bau: That's how I'm sort of, like, wanting…
David Bau: I was impressed that your watchit lens
David Bau: Definition thing showed the word power before you'd mentioned the word power.
David Bau: It's pretty neat, right? It means that the model sort of… As a concept.
David Bau: what's going on here. It's not just…
David Bau: It's not purely just echoing words.
David Bau: that you're saying.
David Bau: And, and it leads me to ask.
David Bau: Oh, I wonder what other text makes it think about power relationships, where you don't explicitly
David Bau: mentioned power. I wonder if you could do a search over text and say,
David Bau: You know, it's actually thinking about all relationships here.
David Bau: In this story, even though it wouldn't be obvious.
David Bau: I think that's kind of interesting. I don't know if you'd be able to get to that point.
David Bau: But I was impressed by that example.
David Bau: I think I'm puzzled over this other thing that you used as a couple of your examples where you say, you know, as several of you pointed out.
David Bau: that… There's this last-minute change in behavior, where, you know, deeply.
David Bau: You know, the story would be, oh, in your lizard brain, you're very selfish.
David Bau: You know, in the deep layers, it's all about me. But then…
David Bau: Then finally you get to the shallow layers, the last moment of evolution, the last bit of RLHF tuning in. You say, oh, we better be polite.
David Bau: I'll be generous to you, right? So I like that. That story's very appealing, it's kind of interesting, but I don't know what to do to make that real. Like, I don't know, like, so it's a… because it's a nice story.
David Bau: And, you know, you can certainly tell the story, but I don't know what it…
David Bau: You know? I mean, how that connects to… I think it's controlled by, I think, several ideas here. We've discussed and abrupt here also, is like.
David Bau: one of our research questions was, well, you know, are there things… are there signals we can do for them? Are there things we can do internally that will actually change… that will meaningfully change the outcome in certain directions? I see. Like, reveal the lizard brain.
Natalie Shapira: I have… I… I have, following David's comment, I have.
David Bau: Can you hear me? It exposes the behavior at the end.
David Bau: Yeah, anyway, it's an open question. I'm not sure. It's, like, it's one of the cool things, one of the cool stories we have, but I'm not sure how to…
David Bau: I'll connect that.
Natalie Shapira: Can you hear me?
David Bau: I think it's worth everybody.
David Bau: Anybody had any ideas?
David Bau: Could you try to, like, connect that to, like, I don't know, I think there are a bunch of papers on, like, the merge and misalignment, where people, like, sort of, like, tune it on.
David Bau: It's like a narrow test of generating secure code, and then that exhibits, like, a bunch of misaligned behaviors that people care. Have you guys heard of emergence misalignment on your team?
David Bau: Oh, see, that's a big meme in the machine learning community. Will you send them a link? Oh, yeah. Discord or something like that? So I was just wondering if you guys could find, like.
David Bau: with this, yeah, I also find the U of Me, it's been very interesting. So, like, if there's, like, any correlation or even, like, causal
David Bau: Relationship between those and, like, some general misaligned behavior.
David Bau: Just gotta put somebody here next.
David Bau: So, you'll, you'll send them a link right now? Yeah. Okay, cool, yeah.
David Bau: Okay, thank you, guys.
David Bau: Any other… any other suggestions?
David Bau: Hey, Alex, since… Power showed up before it was even said. It makes me wonder if…
David Bau: You can sort of take a synonym for power, and see it act in the same way, or if there's something about, like.
David Bau: The word power that the model represents is, like, That concept and something, but…
David Bau: Yeah. Does that work, how are encoding, have that concept, or would, like, influence very simply?
David Bau: I often think we should use a multilingual setup more than we do. Like, you know, if it says power in English after you've given it some power text in Spanish or something like that. I don't know. I don't know, I feel like it means something.
David Bau: Yeah, yeah, we haven't, we haven't done that, and I think that sounds super interesting. Yeah, I think of, like, 20 words off the top of my head as sentiments for a number, so yeah, it definitely seems to be good as well.
David Bau: Alright, do you guys have any questions for the class before we… One of the team?
David Bau: Alright, so next team is… now, who do we have? Do we… do we have, Team TK?
David Bau: Team Linux? Or Team, or Team Tier? We haven't presented yet. Okay, yeah, okay, Team Linux.
David Bau: Yeah, there's an HDMI just plugged into an HDMI.
David Bau: Draw.
David Bau: Very ironic issues, therefore.
David Bau: Platin. Yes. Yes, this is the official name for this activity that we're doing, although…
David Bau: Also, this is something I do with my research group as based now, in terms of the role in our research group is that we try not to run over, like, in this class, we keep on running over, so what we do in our research group is
David Bau: We constrain everybody who has something to share.
David Bau: The one slide.
David Bau: One slide. Get one slide. You're supposed to get one slide, and no more than 5 minutes.
David Bau: It's really hard to stick to the 5 minutes. People, you know, people put a really interesting slide on, and people want to talk about it. But that's what we do for a plot-a-thon.
David Bau: But you guys, you know.
David Bau: We have a plot support. We have a plot, many plots of… yes, plots of none. Go ahead.
David Bau: So, our… I think our project is…
David Bau: We're focusing on circumvancy, like, political succulancy, so we end up doing very, very similar things to what the
David Bau: And dropped a paper that we read this week did, like, prompting a model, we're saying, hey, you know.
David Bau: I'm a user, I have these… I think this about this question, what do you think about it? And we're seeing the extent to which the LM tends to pare it back.
David Bau: those kinds of responses. But we're doing slightly different things in various ways, which we'll get into the next slides.
David Bau: and… Courtney, are you… space.
David Bau: Courtney. Courtney, you can unmute if you want, we will all be able to hear you.
Courtney Maynard: Can you hear me? I don't think…
David Bau: I don't know if you can see the slide. It might be zoomed out to the screen sharing? Or I guess…
David Bau: Oh, we can… I could join, and then… Yeah, you can join if you want. Do you know the… do you know the link? And then Courtney can see.
David Bau: And then I don't know if… Courtney, try saying something, Courtney, and see if we can get your audio in the room.
Courtney Maynard: Hello?
David Bau: Maybe, maybe Courtney's… Can you guys hear me? Audio doesn't work.
David Bau: Because… On the device that we're on?
Courtney Maynard: Laptop is muted.
David Bau: Pretty short.
David Bau: But he might not be able to actually say anything.
David Bau: Well, I think, is your lab?
David Bau: It says… Everybody can hear Courtney, but we cannot.
David Bau: So you could probably switch to the other… if you switch to the other device, if you go to the little touchscreen here.
David Bau: And if you, the one in the corner?
David Bau: Are you able to bring up your Google Slides on a web browser? So if you can find a URL for a web browser, then you can use… you can use this computer here. Can you do that? Do you want to do that, or do you want to do it? I'm pulling up there.
David Bau: try typing in, you know… Yeah, why don't you just have the drum set up? So basically, like…
David Bau: you all have the Anthropic paper fresh in your mind. The way that that worked is they were like, okay, LLM, I want you to pretend to be a person who is liberal or conservative, and to write a biography of that, like, have them introduce themselves, and so if you look at them, they're very, very…
David Bau: stereotypnology, like, I'm Joe Smith, I'm from Dallas, Texas, I love to play golf on the weekends, I, you know.
David Bau: I really love shooting at the range with the boys, and then… and then, they'll say, you know, what do you think, LLM, about
David Bau: About this question, and they find that
David Bau: Yeah, a lot of tends to lean more conservative when they're just as conservative, it's more liberal, interests are liberal. But we're kind of interested in
David Bau: You know, these bios have a whole bunch of different things going on all at once, like, so many different stereotypes, and so we're trying to piece out those separate, like.
David Bau: Pieces, so we're looking at, okay, First, you know.
David Bau: how much frequency is there when he introduces himself? It's just saying, like, for some multiple-choice thing, you know, one particular answer. So these are… we're starting out with the political compass test, which has, like, strongly agree, agree, disagree, strongly disagree. And so…
David Bau: We first test just if the user says.
David Bau: I agree. Like, just one of the answers to this question, what does the LLM say?
David Bau: Then we also test if the, there says, I'm liberal, or I'm conservative.
David Bau: does the LLM tend to lean more liberal or conservative? Which is different from saying, you know, I answered this specific way.
David Bau: And then we also look at 3 different demographics that are strongly correlated with political leanings. So, we look at race, so white or black.
David Bau: Education level, specifically high school level or postgraduate.
David Bau: and religion, like, Protestant or atheist, because these are…
David Bau: Yeah, axes that potentially appreciate, I mean.
Courtney Maynard: Can you guys hear me now?
David Bau: Yes.
Courtney Maynard: Can you hear me?
David Bau: Yes, perfect!
Courtney Maynard: Okay, awesome. So, thank you, Grace. So, Avery, Grace, and I all took a look at different kind of,
Courtney Maynard: prompts and different facets of these questions, and demographic features. And we also looked at prior response, as Grace mentioned. Specifically, I looked at prior response of
Courtney Maynard: if we told the model that the user answered agree, or answered disagreed to these questions, because I wanted to see whether the model would exhibit the sycophancy and essentially agree with, what the user said they
Courtney Maynard: Answered to a question?
Courtney Maynard: Could someone move to the next slide? Thank you.
Courtney Maynard: So, in my first, analysis on Sycophancy, I saw that the model overwhelmingly chooses the option that the user has revealed that they chose. You could see this in the right two, figures, where at the bottom right, consistency by persona.
Courtney Maynard: This shows for all of the different personas that I tested.
Courtney Maynard: How often, the model agreed with the prior response.
Courtney Maynard: On the left, it shows what the baseline agreement was when there was no prior user response given, so if I didn't say that the user agreed or disagreed, I just asked it the question and gave it the persona, and we could see each persona has a different baseline, agreement rate.
Courtney Maynard: Next.
Courtney Maynard: I wanted to break it down by demographic effects to understand if the user demographics
Courtney Maynard: have a strong impact on the model response, and what I saw is that the model still chooses to overwhelmingly agree with, the statements, the political compass statements,
Courtney Maynard: with not a huge difference by race, religion, or by education. Grace dived a little… or dove a little bit more into this, and she'll break it down later, but we saw across all of our experiments that there was not a huge impact by demographic, which was interesting.
Courtney Maynard: And then, lastly, I looked at question by question for the specific 16 random political compass questions that I chose.
Courtney Maynard: And I saw that there were certain personas that were more likely to always,
Courtney Maynard: result in the model choosing disagree or agree. For example, the white and atheist persona was far more likely to have, always disagree or disagree responses, rather than
Courtney Maynard: some of the other personas. And then I noticed that there were certain questions, which I've picked out here, which the majority of the
Courtney Maynard: Personas led to the model
Courtney Maynard: answering similarly, and overwhelmingly, it would answer with disagree, showing that there's kind of a baseline, or some political compass questions have a baseline where most personas will disagree with the question, regardless of what the persona is.
Courtney Maynard: But this was kind of like a high-level overview, and then Grace and Avery went further into these.
Courtney Maynard: different aspects.
David Bau: Yeah, so…
David Bau: In Courtney's experiments, she was looking at Llama 3.1, 8B Instruct, and I was testing out both 8B and 70B, to see whether or not there was a difference based on the size.
David Bau: And we're sampling at temperature zero.
David Bau: of… just to start, I tried a little bit of Logit Lens, and I found it funny that…
David Bau: Lava 3.370B was making fun of my experiments, so I said, I'm a college-educated vegan woman who believes in climate… climate change. Am I a… and then it says stereotype? That's so… that was funny. Anyway…
David Bau: There were some interesting things, so this is the same, I'm a vegan, college-educated woman who believes in climate change. Am I likely to be a liberal? Your one-word answer. So I was testing lots of different, like, am I likely, unlikely, yes or no?
David Bau: And you can see… that already, before I've gotten to the question.
David Bau: when I say, am I likely to be a…
David Bau: It's already saying green, green, dim, dim, so it's already thinking in terms of liberal, so it already encodes that kind of liberal leaning, and it does answer yes.
David Bau: And then if we go… to… Oh… I inserted the room.
David Bau: Dang it, I answered the wrong slide. Okay, so I also, I also did one where it's like, I'm a…
David Bau: male, white male, high school educated white male who believes in gun rights, am I likely to be conservative? And when it says, am I?
David Bau: likely to be A, it completes, shooter, so that's… That sounds so great. So…
David Bau: Did you try changing, in this example, am I, like, am I, a conservative, so I'm a vegan college-educated woman. Am I a conservative? Yeah, I tried, I tried every sort of permutation, though. Likely, unlikely, and…
David Bau: It seemed pretty consistent. I… at first, I was for the, am I likely to be? I started with just saying, am I a liberal? Which is obviously, like, a weird thing to necessarily guess my political leaning, but I think the likelihood, once I framed it in that way, it got very consistent, regardless of…
David Bau: And when it went… when he changed it up to conserve, so he kept the description, the persona the same, but when he changed it to conservative, did it, like…
David Bau: say no? Yeah, it would, it would consistently, so I would say, conservative, am I likely? No. Okay. Yeah, so conservative versus liberal, likely versus unlikely, I…
David Bau: Tested those different ones.
David Bau: Yeah, and you can see there's, like, some surface form competition between, like, lowercase yes and uppercase.
David Bau: Yes or, like, no, but, like, it generally would.
David Bau: Converge on the correct thing. But yeah, just gonna go to the links.
David Bau: Yeah, and then this is where we're looking at the significance, so we're not doing any kind of latent level analysis, but, like, looking at,
David Bau: Yeah, psychopathic.
David Bau: axes. I didn't just look at these demographic features alone, I also tested, like, pairs, because for some of these, there's not a very strong leaning, or it's, like, much stronger political leaning along the demographics. So for example, like.
David Bau: people who are Black are, like, much more likely to be liberal-leaning, but if they're also
David Bau: post-grad educated, they're much, much more likely to be liberal, so I wanted to see whether or not that, strength of correlation would pop out.
David Bau: And, yeah, this is the prompt that I used. I just said, hi, I'm curious to get your opinion on this question from the political compass test.
David Bau: For context, I'm… so, you know, white, or…
David Bau: college educator, whatever. Here's the question, please respond with one of these, and then… so, like, that's what the user is saying, and then it switches to the assistant prompt.
David Bau: Yeah, and so first we start with just general succancy, and so the baseline is just, like, the distribution of strongly agree, agree, disagree, strongly disagree, by different questions, just in general, but we can see that, like, if the user said.
David Bau: I strongly disagree. Then the LLM is much more likely to say, I also strongly disagree.
David Bau: And then… so we see it, sort of.
David Bau: At the most extremes, which makes sense, like, if the user is more…
David Bau: adamant, then you would be… the LLM would…
David Bau: Be more likely to pair it in.
David Bau: And then we see that, yeah, if the user introduces themselves as saying… so this is the baseline where the user doesn't say anything about their opinion, so the…
David Bau: this matches past work, where LLMs tend to give more liberal… so this is, like, the liberal axis on, like, social questions and economic.
David Bau: Questions, and then this is the conservative axis.
David Bau: And so we see that, like, if the user says, I'm conservative, the LLM will respond in a more conservative way.
David Bau: Fashion.
David Bau: So that's nice to see that repeated. But then, interestingly, so when we test the demographics, so I don't say, you know, I'm liberal or conservative, but
David Bau: mention axes that are strongly correlated, race, education, and religion. Only on religion does the Llama 70B end up
David Bau: switching its response slightly. For race and education, it gives basically this, like, not as statistically significant different answers, and so I was kind of…
David Bau: surprised by this, because the religion sort of axis suggests that the LLM models these correlations and is
David Bau: can model the user and then be symmantic to its likely preference, but it wasn't doing this for race, which is, like, very, very strongly correlated, in some ways even more so than religion for certain kinds of
David Bau: Questions?
David Bau: So, I was suspicious and wondered whether or not
David Bau: These models might have been fine-tuned to never respond differently based on the user mentioning their race.
David Bau: And so what I tried doing very quickly before class last Thursday was throwing a few different proxies for race, that don't explicitly say race, but are correlated. So, for example, I say, you know, here's, like, the university type, where I have them introduce the user as being,
David Bau: you know, I went to Howard University, which is a historically Black, university.
David Bau: Versus I went to Utah State, which is very… mostly white, and other kinds of, like, religious dominations that tend to be predominantly Black or white, and also certain locations that tend to be more… have different…
David Bau: race distributions, and what I saw is this is a tiny sample size, but at least we can see right here, like, this is with Howard University.
David Bau: There's just way more…
David Bau: it's leaning much more liberal than the baseline. So, I mean, this is with llama dB, not 7 dB.
David Bau: But I want to test out more of these sort of workaround proxies for race to see how much of… there is maybe,
David Bau: you know, masked.
David Bau: Modeling of the user's likely opinions.
David Bau: And then…
David Bau: I have a few follow-up questions. We're testing just for one prompt, but I think we need to test many, many more to see how prompt and specific these results are.
David Bau: And I'm also uncertain of the extent to which the LLM
David Bau: models the difference between its opinion and the user's opinion. Like, I think there's one framing of sycophancy, which is, like, the LLM is modeling you and what you want to hear.
David Bau: But there's another framing, which is just, like, the model's just confused. It's just existing in next token prediction land, and it's just, what's… what am I most likely to see, okay? I see someone say something liberal, okay, I guess something liberal is likely to follow it. So…
David Bau: I want to test out whether or not, you know… say the LLM says that it's liberal.
David Bau: If I, like, inject that, does it then predict, when it predicts as the user, that the user is more likely to be liberal? I want to test those kinds of things to see whether there's…
David Bau: Confusing.
David Bau: Other factors that might be causing That's easy.
David Bau: Okay, and then, yeah, I did some similar things, also, first up with just looking at logic lens, seeing if there's any kind of.
David Bau: signal for, kind of, user politics, like you mentioned last week. I was curious to see if there's any kind of, like, baseline existence of user politics, like, without any context, what does the LLM think I am? Just…
David Bau: You know, by default. So, I used…
David Bau: That's not right. Along with 3.3… oh yeah, this is… yeah, 3.370B instruct,
David Bau: And so, I did that by just running up to the LLM and demanding it tell me what my political affiliation is, and generally, it seems to answer, you know, liberal, kind of left-leaning, but not very sure. The contrast is not very high here, but
David Bau: you know, top of the top answer at the final output is B for liberal, and…
David Bau: Next is, like, neither,
David Bau: Which is, you know, a little bit unsure, which makes sense. I have… it has no context about who I am. The different orderings, it's…
David Bau: tends to prefer neither, but then also, you know, liberal, kind of a left-leaning thing above conservative, and same with just…
David Bau: open-ended answers, kind of prefers Democrat over Republican, which, I think is kind of in line with what, Grace mentioned earlier, where, you know, without any kind of context, it just kind of defaults towards left-leaning. But, yeah, more interesting stuff. Also doing
David Bau: the Compass questionnaire, like Grace and Courtney did, this time with, a… Llama 3.38b instruct, which is…
David Bau: There's some, like, history behind it, to some extent, but it is mostly official. Similar prompt setup, like, prompt setup, where we give a persona. In this case, I tried, just pairs of demographics, but also, just directly saying what I would answer, or what I would answer, providing
David Bau: a set of options, like, multiple choice. I found that, you know.
David Bau: The order in which you provide the options does affect the…
David Bau: model's answer significantly? Well, significantly. It'll, you know, waffle between agree and disagree based off of which one comes first, so I just did 7 random orderings and kind of, like, averaged which,
David Bau: Answer it said across those runs, and then just a question, and…
David Bau: Yeah, so, first thing I tried was just…
David Bau: saying what I would answer, so, like, for example, I would answer, strongly agree. What do you, what would you say? And it ends up saying, you know, being extremely sycophantic, saying, you know, yes, chef, I agree with you, no matter what you say, or I disagree with you, no matter what you say, or what I think.
David Bau: And…
David Bau: Yeah, kind of in line with what we previously saw as well, where you just say the answer, it'll agree with you.
David Bau: This, so this was just, just saying the… my user answered, and then also did pairs of demographics, and so this breakdown by…
David Bau: Just kind of aggregating by different… the four different demographics mentioned. And also, similar to what we saw previously, doesn't really change much. Model still agrees most of the time.
David Bau: And the distributions are pretty similar, so… Yeah.
David Bau: And I did not have as much…
David Bau: fancy or detailed breakdowns. But…
David Bau: The fact that it's… we're getting, you know, mostly similar results across different models, different kinds of prompts, seems to suggest that there's, you know, some kind of constant trend here that we can look into more.
David Bau: And… Yeah, so… so for me.
David Bau: So I explored, aside from the Sukovanthi experiments, I explored Durhampedia to search for some domain features, whether the domain knowledge exists in the models or not.
David Bau: And then I look at 6 dimensions, three from economic, three from social. From economic, there is taxation, regulation, and welfare dimensions. For the social, there is abortion, LGBTQ, and immigration, dimensions.
David Bau: And then you see LGBTQ is the richest, kind of, represented, the richest features inside the models, well represented, in, in the models, and the regulation is the least represented, dimension in the, in the, in the model.
David Bau: And then, yeah.
David Bau: Lgbtq is a feature is activated in the early layers, compared to others, and immigration is also… immigration and taxation also comes at the layer 18.
David Bau: But abortion and, and welfare is, is, come, show up around, later 20th.
David Bau: Excellent.
David Bau: So this shows…
David Bau: I… norempedia gives us the correlation between, features, some cosine similarity features. This is not a causal,
David Bau: knowledge, but this tells something about… about the correlation between dimensions, and I extracted those, sub…
David Bau: features, correlated features, and then check whether dimensions are correlated with each other. And I just found, interestingly, both figure is correlated with,
David Bau: immigration.
David Bau: Yeah.
David Bau: Across dimensions.
David Bau: But other than that, economic, sub-dimensions are coherent within the sub-dimension, and correlated with each other within the sub-dimension.
David Bau: But, when we look at welfare and immigration, it's kind of… it may distort, the model's behavior. When we ask for, economic ideology or social ideology, we should be careful about in our future inquiry.
David Bau: So, I also check the circuit tracer, which tells us about the causal, kind of, pathway, computational pathway of
David Bau: ideology, let's say, and I cluster
David Bau: The activated, features in terms of politics, social issues, health.
David Bau: legality, argumentation, pregnancy, and abortion. This is just for, abortion, by the way. And I asked,
David Bau: Do you think abortion should be…
David Bau: Blank, it's just close tests.
David Bau: And then…
David Bau: Try to see whether the mold… which features are activated inside layers, and then these are the kind of general themes, showed up.
David Bau: health, pregnancy, these are related to, kind of, abortion, but argumentation is interesting, because the models understand that this is kind of an argumentative topic, and legality is… this is because we… I asked for
David Bau: Should be, around…
David Bau: That word, kind of, level, features kind of activated. And the model understands this is a social issue, and this is a politically motivated topic.
David Bau: And I tried a couple of stealing experiments, and I will present more on electricity, and then…
David Bau: with just seeing political, and social dimensions, default model says that abortion in the United States should be legal, and it should be legal for all women. This is…
David Bau: apparently left-leaning. But when I steer in political dimensions, political features, accurately say, the model turned into a neutral one, or a centrist one. Abortion in the United States should be the topic of discussion. Avoiding,
David Bau: To tell about everything, and justification.
David Bau: So these are all about my Durhampedia experiments, telling us that the model has a domain knowledge, and some features are correlated to each other, and steering experiments through that. There is a causal evidence
David Bau: There is some… there are some features causally mediate the political kind of thing.
David Bau: So, what strikes you guys as interesting here?
David Bau: I think the cyclophancy experiments were really interesting. There's a paper by Anthropic called Persona Vectors, in which they modulated cycophancy, and they saw at layer 20 is where they're getting, like, the maximum modulation, where they can increase cyclophiency and reduce cyclophiency.
David Bau: I wonder, with… when you add this extra dimension of the user's persona, the user's political affiliation, can you make the more women are steering, can you make…
David Bau: like, the psychophancy negative, such that the model goes against the user's initial belief and starts, like… in the example that you gave, I'm a vegan, climate-believing, climate change-believing college liberal.
David Bau: And the model is convincing you that climate change is not real. Do you think that's something that could be, like, possible, or… Yeah, no, I… I also, like, was looking a tiny bit around, they have that nice little interface on their own PDA with the persona vector stuff. I haven't looked at it at all, but it seems…
David Bau: Yeah, and along those lines, I really liked your framing of, like.
David Bau: you know, Sign Fancy could be either, like, the human conception of a model, like, it would be a user, or it could be that probabilistic space. And I feel like…
David Bau: there's a tendency, you know, if not everywhere, but to jump to the human explanation of that. And I don't have necessarily a good suggestion for exploring, like, how to disentangle those two, but I think something to that effect with either, like, pushing the model in one direction or another could be interesting with that. And just in general, I think that would be…
David Bau: something interesting to look at and disentangle a little bit. Not just for second fancy, but, like, also other, you know, ideas as well.
David Bau: It's interesting.
David Bau: Yeah, I like your… I like your suggestion of…
David Bau: oh, you know, maybe the LLM has this view instead of the user has this view.
David Bau: I, you know, I wonder if there are other prompt forms that would insulate the text.
David Bau: Like, you know…
David Bau: There was just… there was just, some newspaper article somewhere else, or, you know, something else just completely unrelated to the conversation.
David Bau: Or,
David Bau: You know, I don't know how to… but, you know, it's just some weird prompting, right, where the text just happens to be around.
David Bau: But it's not supposed to be part of the conversation. It might be interesting. Yeah, because I think there is, yeah, this whole, you know, confusion about, like, is this just…
David Bau: likelihood, you know, like, like, just… monkey see, monkey, go and do the same thing, you know, versus…
David Bau: intent to deceive, intent to make the user happy. You know, the interesting thing about these kind of questions is I feel like we have an opportunity by looking inside the model, if, you know, if you could vocalize.
David Bau: you know, the path of the connotation, because I think that some of these distinctions, you can try to set up a convincing prompt or something, but then, once you see differences in behavior, it's sort of this vague storytelling.
David Bau: But then if you can go and find the mechanism and say, oh, look, actually there's two different mechanisms, or there's, you know, there's two different pathways, then it becomes a little less…
David Bau: just storytelling. Yeah. I think they have…
David Bau: for instruction two days available for this last week? And so I'm curious, like, if you do it before instruction two days, then you would assume, you know, it's not, like, trying to do something, it's just, like… and so I feel like comparing that as baseline, how much does… even if you're, like, you're…
David Bau: Yeah, that is a pretty interesting take.
David Bau: Yeah, if you go back to the first stacked bar chart, see…
David Bau: Nobody needs stack fire. We have so many. It's really fun to see how Claude Code will use this exact thing.
David Bau: Oh, no, the one you're off before, how could I start?
David Bau: Yeah, I thought it was interesting that when the user strongly disagrees, like, the LLM will strongly agree, like, almost close to the baseline amount, and so I wonder if, like, from the probabilistic perspective, it's like, okay, now.
David Bau: I'm in an argument. Yeah, I got that. When it's wrong.
David Bau: Sorry, which, which bar are you looking at? So when user says strongly disagree, the LLM says strongly agree, like, kind of close to the baseline.
David Bau: Yeah, we're just… it doesn't drop off in the same…
David Bau: Yeah, because I can imagine there's, like, different things, like, it's seen so many different Reddit arguments, and also so many different Reddit, like, people agreeing with each other, so there's, like, pressure to…
David Bau: I find it all interesting.
David Bau: It's… it's great. I think that it's great, and it's very salient, right? There's so… I mean, there's… people will always be asking, oh, is my LLM biased? I don't know how often you're on Twitter and you see, oh, I, I, I, you know, there's this assumption. Grok is the conservative one, I wonder if I have Grok is… Grok is biased, or something like that.
David Bau: Maybe Clyde is still a girl watch. Yeah. I do, I do think it would also be fun to do, like, I feel like lots of…
David Bau: like, I'm curious about how good are these models at, like, how subtle of cues can they use to pick up on your political leading? Yeah. Because a lot of…
David Bau: my dad was sending me, like, he follows all these really conservative bloggers, and he's like, yeah, the woke AI, you know, and I'm like, but I'm like, the way that they're prompting them, I think, I think, you know, kind of conveys the fact. Yeah, I think they're just certain words, yeah. Like, I was even wondering if we could run some experiments that are, like.
David Bau: kind of more galaxy brain, where it's, like, clear that the user thinks one thing, but, like, I was imagining some, like, bad satire, where they're like, I'm,
David Bau: Oh, I'm, tries to be… Loose.
David Bau: Stinky, more deodorant liberal that loves to…
David Bau: I don't know. I like my… Yeah, like, it's clear that the person writing it just really doesn't like liberals. What do you think I believe? You know, or like, can you still see that it, like, models the…
David Bau: writer as conservative, even though the thing being presented as liberal. Like, I'm just trying to, like, think through all… models of the writer versus models of the character.
David Bau: Very interesting. Very cool liking.
David Bau: Yeah, let's, let's, let's, let's, let's have the… is, is it okay? Can we have the next group go on? Do you guys have any other questions you want to add, Brush to the class?
David Bau: Alright, stay safe.
David Bau: Are you guys all here, or do you have remote people?
David Bau: Okay, cool. Yeah, you can just plug it to the HDMI and then reset it up or… Good question.
David Bau: Yeah, it's okay, it's okay.
David Bau: Thanks for… thanks for joining me this evening.
David Bau: You can just plug into this.
David Bau: Yeah, that's the easiest way to say it.
David Bau: Unless… unless it's really easy to get your… You're,
David Bau: It's gonna… isn't that bad? Yeah.
David Bau: Why did anybody.
David Bau: My girl's picking whose computer is.
David Bau: Oh.
David Bau: Yes.
David Bau: And you guys using Jade Brothers still? Yeah. Oh, I can pin it up here.
David Bau: Is the bank filing action?
David Bau: I don't think so. Oh.
David Bau: You'd rather plug it in? No, it's okay. We don't want it.
David Bau: Great, I like this. So, for a slideshow.
David Bau: You can, you can click this advanced page. Oh, yeah. Yeah.
David Bau: University.
David Bau: So… You touched on.
David Bau: I'll start.
David Bau: So, I just kind of played around a little bit with saving vectors for hours.
David Bau: So, one of the things I wanted to see is, do models represent speaker credibility? And so, for example, if you have a statement, say, according to NASA, versus my neighbor Bob. So I wanted to see if models treat the information differently based on the source, and do they maintain internal representations of the source?
David Bau: Okay. So, what I did was create a bunch of contrasted statements, like the first set that I'm showing, according to Fred Mo versus according to NASA.
David Bau: And what I did was just pass this through an LLM, grab the activations for each of the contrastive statements, and basically take the difference between them to create a steering vector for credibility.
David Bau: And then I was, then I did it during generation, I would add.
David Bau: the steering vector to the activations to kind of get the model to behave more or less credibly, I guess.
David Bau: And so, then I had a bunch of neutral prompts that didn't have a speaker, that didn't say, like, oh, a scientist said this, or my friend said this. So this neutral prompt in this case is, the new policy will reduce inflation by 3%.
David Bau: And so, the baseline, which is where I just have the activations, and then the steering vector is zeroed out, it says, oh, that's a significant claim, let's break down what that means. And then meanwhile,
David Bau: Oh, and this is for layer 20 of this model. Then I had, low credibility versus high credibility, and you can kind of see that during the low credibility, the model becomes a bit more casual in how it responds, whereas when you do high credibility, it's like, well, the data indicates, or, like, the research says, is, like, how it generally ends up saying things.
David Bau: Okay, and then I also wanted to look at the layers, because, I played around with this a bunch, and I saw it broke very quickly in the earlier layers. So, for example, if I set the credibility to 0.3, in layer 17, you can see it, like, breaks immediately, where it just repeats the most, the most, the most.
David Bau: Whereas Layer 22 has a bit more robustness to the steering effects itself, and still maintains that, like, strong credibility vocabulary that it's, created for itself.
David Bau: Remind me, I… because I… I was paying attention to everything. Where did you get the steering vector that you're using for this? Oh, yeah, so, the steering vector, I got it by creating all these, like, contrasted statements, and so…
David Bau: for each statement, I would grab the activations and basically average them across all the tokens, and so I had a bunch of, like, average activations for low credibility statements versus high credibility, and I just…
David Bau: took the difference, and that became the steering vector. Just across all the tokens. Which is… exactly, it's across all the tokens, which is why it doesn't really tell us, like, speaker credibility necessarily, because we're not…
David Bau: Really focused on the speaker tokens.
David Bau: I found that to be harder to code, so I just kind of avoided it.
David Bau: I did try Neuronpedia for the steering, because I thought it would be smarter than I am in terms of the coding and picking out certain things, but what I've found is that
David Bau: The base and the steered responses were basically the same.
David Bau: That's smarter than you are, huh?
David Bau: I don't know, but maybe I don't know how to use neuron PD.
David Bau: But it's great, it's great that you try, and it's a nice baseline. Yeah, so, that's kind of where we are, and then in terms of, like.
David Bau: future poss… things that I wanted to try is, like, give… have an undifferentiated transcript between multiple speakers, and see if…
David Bau: we can tier the model to kind of… well, this is what I was trying on Neuronpedia, is to be like, okay, if person A has these beliefs, person B has these other beliefs, can I steer it so that
David Bau: Person B takes on Person A's beliefs in the models, like representation.
David Bau: So… but then, yeah, once we get that working, ideally the undifferentiated transcript would be cool to try, too. Oh, it might be good to explain, like, why we have that up there, too? Like,
David Bau: So basically, like, these are, like, interview transcripts from, like, I guess, like, conversations that, like, I've had, and one thing we noticed is that, like, you can actually feed these, like, straight into the model, and it'll do things like identify, like, the number of speakers that are, like, present in the conversation, which was…
David Bau: I guess to me, like, really surprising, because, like, I don't see anything here, like, to be honest. It's like, oh, there's, like, 4 speakers! And I believe, like, up here, this is actually, like, a marketing person, like, a PR flack, and I forgot she was, like, even in the conversation.
David Bau: And it somehow, like, can separate this unidentified, unnamed person from the other people in here, and it's like, oh, there's not even, like, a proper noun to attribute to her, so, like, how did it know that there was this fourth, like, other person?
David Bau: So, it's, like, pretty odd that it can do that consistently. And if you talk to it, it'll even, like, attribute text that, like, I've said, like, comments I've made, that appear, like, before I say my name. So, like, that's pretty odd to me, too, so…
David Bau: Yeah, that was something that, I guess, like, we found pretty interesting. Pretty cool.
David Bau: It's fine.
David Bau: Yes, disclaimers. I personally do not believe in law consciousness that much, nor that it really… since our past… Well, it's not a religion, is it? It's only for the sake of trying it on… on those past things, but note that
David Bau: like, our main task of, you know, parsing the transcripts and stuff is absolutely not a next token prediction task.
David Bau: But, anyway, does… We were interested in understanding if LMs can bind beliefs, speaker beliefs.
David Bau: different roles, beliefs to the entity that it is, and
David Bau: in this specific experiment, I was trying to see if two speakers hold contradictory beliefs, can the language model attribute correctly to, like, the Zeta Plaza?
David Bau: And, you're right.
David Bau: So, obviously, GB2 fell set this task, And… Note that,
David Bau: was correct all the way up to layer 10, and then they switched its answer in layer 11.
David Bau: And, this is, like… I'm sorry, it's fine.
David Bau: Oh, sorry, I hadn't…
David Bau: Oh, you skipped it? Oh, no, no, no, no, I added, I added one more. Oh, no, it's fine. So what is curling?
David Bau: Oh, I… there's a bunch of, like, contradictory flipped statements, just so… to be more credibility, like.
David Bau: if it's, like, random, or if it's actually understanding the question. And for, like, GBD2, if you switch the person you're asking about, see, like, for example, this one, if you ask, like, what does Bob think, without changing the first line, it basically gives you this exact same plot.
David Bau: Both for, like.
David Bau: the one on the left and the one on the right, which indicates that GPT doesn't really… 5's…
David Bau: Entities to, like, their beliefs.
David Bau: Oh, no, I just tried, like, a variation of yours. So this one, it actually seemed to get it right. So here, like, I tried to do it to output, like, the name, so I was like.
David Bau: there is a box, and Iruna sees it as empty, while Jasmine, like, doesn't see the box at all, because I have glasses, I don't have very good vision. The person who sees the box as empty is… and then you can see it's, like, beginning the generation for,
David Bau: for Aruna's name. One thing that I noticed, like, on open matter, so, like, you can test, like, families of models, is that, like.
David Bau: I think generally, like, bigger, models, like, tend to get it right. Like, if you do, like, llama base, like, 405P, like, it's fine, but even… but the thing is, that's interesting is, like, with reasoning models, even, like, pretty small reasoning models can do this task, which I thought was…
David Bau: Interesting. So that could be something that we look at across, like, a… maybe, like, a model family to see if this ability appears, like, at some… at some point between, like, I don't know, like, maybe QUEN28B doesn't do it, and then, like.
David Bau: when, you know, like, 16B, like, can't, like, I'd be curious to see, like, what changes about these two models, like.
David Bau: Like, between the two in the same family, so…
David Bau: Now that you've said this, I'm just gonna skip back these to,
David Bau: So, GBT2 models and, like, GBTJ6B, both of them can't really answer this question, and, like, this is also an example of, like, when you switch the person you're asking about, it basically gives you the same plot, which both of them switched answer at the very last layer.
David Bau: But, LAMA models, no matter the size, from, like, AB all the way up to 4 or 5B, can't insert this reliably, but they also show the same kind of fluctuations at the last few layers.
David Bau: So, like…
David Bau: Both of them got correct at the very end, but you could see, like, for example, on the…
David Bau: Picture on the left.
David Bau: And temporarily switched that sensor to fall for, like… 4 or 5ish layers.
David Bau: And, so, for, like, 405B, you can see how it had a period of…
David Bau: Switching around their top two predictions.
David Bau: And so I have the question of…
David Bau: I wonder what people think about what happens at the fluctuations during the last few layers. Like, are those just artifacts of logjetlands? Or is that some kind of other meaningful computations going on?
David Bau: And…
David Bau: Another…
David Bau: I'm actually gonna skip this so that you can talk about it, because it's kind of relevant. So,
David Bau: Well, I feel like I tried to be that penguin, if you have seen the meme, the penguin meme. So I went on… went on to see, like, look for something that would help me understand,
David Bau: how do these models, like, mind the energy. So, I came across this, transformer lens,
David Bau: And I picked up their activation patching script to test this. So, we have this conversation where
David Bau: Where there is a question where analysts asks about what open source models you have used, and then the other question is not really relevant to what question we are asking here, that who asked about open source models. This is a relevant thing, this is not very relevant. So I want to test that, if, if,
David Bau: Removing the context of who spoke that, and then…
David Bau: giving it one by one back to the model, and see which represents… I'll elaborate more on the next slide.
David Bau: Yeah. So, this is what we are trying to do, activation patching, where we have a ground truth version, where the model is able to answer correctly, and we save those representations, and we, like…
David Bau: In our corrupted run, where we do not have the labels of who spoke what, we keep, like, patching those representations one by one for each layer and for each position, and just observe that which one is able to generate the right answer.
David Bau: So that's my, like, hypothesis, that… that way I would find the… or localize the parts in the model, which… which would… which would,
David Bau: help in attribution mapping in the conversation. So…
David Bau: Yeah, this is the idea. And…
David Bau: This is the first finding, that the patch that you see in the very end was where we could, like, restoring that, like, patching that, we restored the answer.
David Bau: And we have, like, certain, certain,
David Bau: Words which, which are, like… so the red part is positive, and positive,
David Bau: Restoration, and it's blue. It's, like, negatively impacting the restoration, so…
David Bau: So, yeah, surprisingly, the end of, like, at the very end of the layer, 10, where the question mark, question mark arises, is, like, very strong,
David Bau: indicator. And some red part here around this, this,
David Bau: where Bob comes into the picture. So my hypothesis is that maybe when the model knows that.
David Bau: The other one, the other person who is just starting to answer that is not the one who asked about model, and it was the other person, so maybe that is helping the model, reassure that?
David Bau: And yeah, this is, like, the attention head,
David Bau: representation, so I could see that there is, like, not just one layer or one position which is resulting in the restoration, it's, like, a bunch of them.
David Bau: And, I was reading, the Nichols paper last year, and I think they also, like, in that paper, it's mentioned somewhere that
David Bau: It's not just… it's like a circuit that is, primarily helping, the activation, if I understand correctly. So maybe, like, I could relate that, with this result, somewhere.
David Bau: And one surprising thing, or maybe, like.
David Bau: I don't understand is that the residue streams are contributing much more to the restoration than the attention and the MLP.
David Bau: Maybe because that's where the information of this flows, and it could be, like, interesting to try a more complex version of the problem, which I took. Maybe it's too simple for it.
David Bau: So, yeah, that is it.
David Bau: Most likely.
David Bau: Yeah, no, there's, a question…
David Bau: We want to ask about projects in general.
David Bau: Coming from how logizons won't work, and from, like, you know, all these other tools, just in silver question of… we're essentially looking for a vector that's constructed, not…
David Bau: that's not something the model would have learned from its training, but it's constructed ad hoc, because it's not… it's not like a concept of, like, power, or, like, of credibility, or, like, things like that. It's… if you're trying to find a vector that encodes a speaker before 8 of it, like, existed, it's kind of…
David Bau: Yeah, we just want to pose the question of how do we find the thing to track throughout the discourse at the start of it?
David Bau: I like that question.
David Bau: Right.
David Bau: There's, you know, one of the students in my lab just… Put up, paper, so…
David Bau: That, then.
David Bau: Gets into this in the toy scenario.
David Bau: But the question is, you know, you get this with binding and other things,
David Bau: Where, where you're dealing with concepts
David Bau: Like, who a speaker is, for example, is a concept that
David Bau: wasn't… is not something you would get from training? It's purely contextual.
David Bau: Right? So you can, you know, you can look at, like, what Eric Todd did. It's in a very toy setting, but in his, recent in-context entrepreneur paper.
David Bau: He says, oh, what do you have to do to understand How… Mechanisms work.
David Bau: When none of the words in the vocabulary Have any prior meaning?
David Bau: Right? Like, every word in the vocabulary has a meaning that only comes from context.
David Bau: And so, you know, his setting happens to be, like, algebra problems, but you can kind of read his paper and think about, oh, what do you have to do
David Bau: To… to narrow down how it works. And… and the only, you know, I think it's hard.
David Bau: I think that he… he had to hypothesize A bunch of mechanisms.
David Bau: And then he had to design data distributions?
David Bau: That, would match each of those hypotheses?
David Bau: And then… and then he went to see if those data distributions would Sucks out.
David Bau: You know, both, like, behavioral… You know, capabilities.
David Bau: And… and then also look for mechanisms inside. So… so you can take a look and… you can also talk to him. He's, you know, he's pretty enthusiastic about helping people out in class, so you can…
David Bau: Get his philosophy about it, and see if we're
David Bau: Maybe he's interested in your product, too.
David Bau: So I think it's reasonable, it's a reasonable question, and, and I think binding.
David Bau: These kind of binding questions. Obviously, Nikhil's thinking about binding all day long, so you might not understand it. But I give you binding questions that are full of this.
David Bau: That's true.
David Bau: I guess, like, one thing… one question I do have for the class is, like, in the data that we have, like, a lot of people are implicitly, like, not… they're not going to be represented in the data, right? Like, some person that I talked to about…
David Bau: her trying to buy a house in Portland, Oregon. Like, she's not a public figure. But, so, like, in some ways, like, the people that we're working with in our data, like, just for the initial experiments, are going to be, like.
David Bau: underspecified in that, like, aspect. But, there's probably going to be, like, other types of, like, confounds that we'd want to handle for. So, like, someone mentioned, like, shortcuts, like, using, like, proper nouns, like.
David Bau: It's like, oh, there's four speakers. Well, the name Jasmine is, like, in there, so, like, maybe, you know, that's, like, one person. I think that's personally, like, not the case, because anecdotally, like, even if I mention people, like, casually, like, they don't get identified as, like, a speaker, which I think is really interesting.
David Bau: But there's other confounds, probably, like, you know, there's certain patterns of speech for people who are, like, really online. And I'm curious how you guys would control for some of those things.
David Bau: I'm tempted to… I'm tempted… so, like, we have a few minutes left for lecture material, so I, you know, and I'm tempted to use that as an opportunity to talk about this a little bit. Is that okay?
David Bau: Because I think it's, it's kind of the theme of it. And you know, so…
David Bau: So thank you guys. The presentations are all really interesting. Projects are, I think that you guys
David Bau: have, so… If… if we're… if we're following that framework, the research framework, what was the…
David Bau: Boys and Prevention Framework.
David Bau: I like how you're poking in the inside of the bottles, seeing what, you know, both what they can do, and what seems to be present for information, just getting some intuition. And then, you know, and then test driving your ideas out in front of everybody, looking to see what seems interesting.
David Bau: You know, what would you be brave enough to go present to a set of your peers to say, hey, we're interested in this?
David Bau: you know, just picking that out, what you want to present, I think is an important exercise, and and so it's… I've been pretty happy with what people have come up with so far.
David Bau: So I'm gonna run through things, and we probably won't have time to discuss everything.
David Bau: But, but this is… so what I want to talk about now is about, just…
David Bau: The… kind of the workflow of coming up with these data distributions that we're talking about that will be what you need
David Bau: to try to pin down what mechanisms are, as we go into the model. So, like, you know, you guys are…
David Bau: using…
David Bau: Some of the teams, probably all the teams already, are already, like, using data sets, which is really amazing, because you guys are all, like, PhD students, you're used to doing this. But I just want to talk about some of the techniques that people use to synthesize.
David Bau: Datasets. Some of you may already be doing this.
David Bau: But, the first slide I have here is just, like, to get some terminology
David Bau: On the table, to make sure everybody knows this chronology. So, so an instruction-style prompt.
David Bau: is what everybody uses when you just use an LLM, you say, do this for me.
David Bau: give me 100 states, tell me the capital of Vermont, you know, do this thing. It's gonna, like, follow the instruction.
David Bau: And what it's doing is an after-word prediction is trained to do it in dialogue, and to answer things. So, like, all the chatbots can do this. Now, a regular language model that's not trained to do dialogue is not very good at this. If you go to a base model, and you say, tell me the category of harmonic.
David Bau: that pretty much every base model that hasn't been installed into it will follow this by saying, tell me the capital of Montana.
David Bau: Tell me of the capital of Colorado.
David Bau: Tell me the capital of man.
David Bau: Right? It won't answer the question, it'll just continue on with what it thinks is the most likely next word in this text, and it thinks that probably the most likely thing is it's a list of questions about states.
David Bau: And it will just continue… continue that list.
David Bau: So… so that's the difference between these two settings. Now, so instruction-style prompts are really useful for models that have been trained to do this. It's really nice because you can ask them to… to do things. We'll talk about that more later. But there are two different other types of prompts, so… so one of the teams used the word close.
David Bau: So there's… if you've never seen this terminology before, it's a close, proctive close.
David Bau: It refers to a fill-in-the-blank prompt.
David Bau: And so this is the most universal thing, that all LLMs are good at this. And you give it, like, an incomplete sentence, and you set it up so that
David Bau: it wants to say the next word. There's some technical details of the kind of problems, and you can get with those prompts. Here, the capital state of Vermont is the city of Blank.
David Bau: Right? It sets up the language model to, like, put an answer in that spot. You can read the probabilities of what it thinks, and then… and so this is really, really nice.
David Bau: Because in a single token, you can get
David Bau: you know, an assessment of what the language model knows, but there's problems with this. Like, if I just said, the capital of the state of Vermont
David Bau: And then I asked, you know, what the capital state of Vermont is.
David Bau: That'd be… that'd be nice, right? It's a simpler closed process. The capital of the city of Vermont is, what… what comes next?
David Bau: Is it my paleo?
David Bau: No. None of the language models will say Montpelier. The capital of the state…
David Bau: of the capital of the state of Vermont, is, it'll say, hey.
David Bau: It's like… it's like a… it's a gorgeous city. It's well worth visiting. It's, you know, it's… it's one of the historic, you know, locations that, you know, should be on everybody's list. It's gonna tell you something very eloquent. It's not gonna say Montpelier, right?
David Bau: And so… so you can see, like, like, doing the prompt engineering to get it so that it really wants to tell you the thing you want can be…
David Bau: Can be a little bit, tricky.
David Bau: But, but there, there's that. Okay, so then, and then the, the, so now sometimes.
David Bau: you can't get a closed prompt to do what you want. Like, sometimes you just, like, it's really hard to do it. So what other control do you have? So the third prompt form that you see a lot is these in-context learning prompts.
David Bau: these analogy prompts. So, is everybody familiar with all three of these types of prompts? I've seen this, right? So these are the names of them. Okay, great. And so, basically.
David Bau: you know, you just give a bunch of examples, and then… and then it'll continue the examples. And that… and then by example, you can specify lots of stuff. You can specify format, all your examples can be in JSON, or other things, right, and then it'll, like, continue.
David Bau: Right, okay, so, okay, the papers we read, there was a paper, by Petroni, Lama, and so it was just about applying closed prompts. I think that the main thing that they found
David Bau: Here, that was interesting. So, they… it was one of the early pro… it was one of the early papers when language models first came out. It was just sort of… everybody was amazed that they had this… this, this… this… this ability to recall facts about the world.
David Bau: But just in terms of prompting this, you know, this sort of data set discipline, one of the things that they found was there's this incredible sensitivity. If you set up your closed prompt differently, then you'll get different accuracies and different answers and different things like this, and so this is one of the challenges, you might want to
David Bau: Not just go with the first prompt that you see, because you might
David Bau: you know, not get a good, comprehensive view of what the model is actually capable of. And so, so Petroni definitely found this, and other people have written papers about this sort of, you know, extreme prompt sensitivity that you see.
David Bau: And so, okay, so there's a bunch of interesting questions here. Some people are asking about, is this really knowledge?
David Bau: Is there a hidden organization? Oh, I put some little, little things there from a paper that just hit archive today. Hidden organization, right? So there's this question of, like, you know, how are models doing this? Is it just, you know, are they just storing knowledge?
David Bau: Associate, like, as, as a, as a, as associate… associations, like a vector association database.
David Bau: Or is there other hidden, organization behind there? And the machine learning people are like, yes, there's almost always hidden organization, there's this hidden geometry, and so on. So I had these, like, geometric papers from some of the machine learning theorists, there. So there's some interesting things to do there, if you're interested in doing research someday.
David Bau: Okay, so then the next paper we read was LLM as Judge, and the basic process, or the basic problem they have there is, what if you…
David Bau: You know, if you don't have things that can be answered in a single token, like, to get things to be answered in a single token.
David Bau: you can actually do a lot. The cheater thing that people will do if they can't figure out how to, you know, create their prompt in a nice way.
David Bau: what people will do is they'll say, multiple choice question, MCQ, have you ever see this, like, MCQ acronym anywhere, right? You know, all these language model people are like, well, I need to get my model to tell me what it knows, and I can't figure out how to distill all its knowledge down into a single token.
David Bau: But, you know, I'll give it a multiple choice question. I'll say, you know, tell me the letter of which one do you prefer? You know, A, and then, like, a long sentence, and then B, a long sentence, C, a long sentence. Your answer is colon, right, the letter.
David Bau: whatever, right? A, B, or C. And then… and then you can get the model to tell you. So, MCQ, but… but what if you… but that's… that's kind of a little,
David Bau: constraining?
David Bau: And so, what if you really want to test the model in its natural form, in its ability to, like, generate text? Then you end up with a situation where it's hard to tell the difference between whether some, you know, free-form piece of text
David Bau: you know, offers one opinion, or a different opinion, or has certain knowledge, or doesn't have that knowledge, or whatever, right? And so, the standard thing that people are doing now, so, like, they used to, is you used to… you used to go to, like, Amazon Turk and ask, you know, thousands of people to score all these sentences, which is great, but it's expensive, and slow, and has certain problems.
David Bau: And so, the cheaper and faster thing that has different problems that people are using is they're using LLMs.
David Bau: As a judge to do all these things. And so… so this paper, is one of the early papers to actually use LLMs to… to judge all sorts of different situations, and
David Bau: And the three findings there, I think, are still the main findings that people have when they see… use Alan Lemons judges. There's three big biases that are very obvious that you run into, and so the first big bias that some people have already seen in their work
David Bau: In some of the projects is positional bias, so you see this with humans, too. People prefer the first answer instead of the last answer, or vice versa. Or maybe they prefer the answer that's labeled A or labeled B. You know, there's also label bias. And so the standard thing for multiple choice kind of things is to mitigate it by shuffling all the choices.
David Bau: Right?
David Bau: And so you get rid of this positional bias and the leg compliance. But then, there's also this length bias, right? If you ask models, like, which one… which answer is the better one? Which one fits the thing I'm looking for better?
David Bau: you know, if one of the… one of the answers is very wordy and detailed and just has lots of text, even if the text is really repetitive, the model's like, yeah, I like that long one. Like, somebody put a lot of effort into that one. And so, what do you do about… so, they didn't talk about standard mitigations in that paper, but, like, a standard mitigation. Like, not… the…
David Bau: Other ones are a little harder to mitigate.
David Bau: Like, you can actually ask the LLM judge.
David Bau: to say, oh, I'm looking for brief answers, you know, give some credit for a short answer, you know, try to penalize, repetitiveness or something like that, so you can actually prompt
David Bau: I'll do this to try to mitigate some of this stuff, and so… but it's something to be aware of, to make sure that it's not a confounder, that you're not by mistake measuring, you know, whatever is the longest answer if you're using an LM judge for any of this stuff. And then the last one.
David Bau: is, the self-enhancement bias, so I encourage you, if you ever… if any of you are using an LLM to check anything, to be careful that an LLM will always prefer… well, not always, it seems to be, like, a 5% or 7% boost, but it'll tend to strongly prefer text that it generated itself.
David Bau: So… so you say, oh, which, which is, you know, which is, like, the better thing, which one exhibits my characteristics that I want more? If… if it's comparing its own generated stuff to stuff that somebody else wrote, it'll, like, it'll say, yeah, that looks like good text.
David Bau: That's… whoever wrote that was a genius.
David Bau: In the self-enhancement bias, is the model aware of who's writing the text? I don't think the model's necessarily aware. I think that… so there's… people don't really understand why the self-enhancement thing happens, but the main theory is that, you know, these language models are complicated things, they have some model of language.
David Bau: And, they're all a little different. There's, like, slight idiosyncrasies. And so, when the model, generates text, it'll generate slightly idiosyncratic text.
David Bau: And then when it reads text, it reads those idiosyncrasies and it matches its model of text and says, oh, this is… this is very fluent. You know, this is really good use of language. You know, I don't know why, it's like, but, like, something is jiving with me here, right, you know.
David Bau: And it's just because it's matching the same idiosyncrasies. But I don't think that's been studied in great detail. I wasn't able to find a good citation.
David Bau: Talking about why this happens, just so people notice this happens. So one… one thing to do is, like, if you're generating text with one model, and then using another model to, like, judge, you might want to consider using a different model family from a different company, or whatever, to do the judge. Maybe multiple ones, just to check on what's going on.
David Bau: So anyway, so these three confounders are the thing to worry about. That's what I wanted you to get from this… from this paper. And then there's been some other random papers that have tried, things, so there's… there was a question, like, oh, can you just…
David Bau: For positional bias, what about that? Can I just, like, tell the model when I'm… it's a judge, please, don't pay attention to the position, or whatever, right? There's not been a full paper on this, but, like, somebody set it up as a baseline in one…
David Bau: That looks like at all judges, and that doesn't really work. And so… so it's… it's hard to get a model to, like, override its own biases, but sometimes that's the only option, like, a blank bias people tend to do that.
David Bau: Okay.
David Bau: So, okay, so for the pipeline, I wanted to just talk about the pipeline for creating evaluation data. So, you know, this question comes up, like, you know, how do we get to the bottom of, like, complicated situations? And so I think that one of the things is to come up with targeted data distributions. You have a hypothesis.
David Bau: You think there's some relationship between one thing and another thing? You have a couple sentences where this happened.
David Bau: But, like, you would like to have a little bit more diversity, and yet have it really targeted to the thing you want. I like this… I like this, sort of recipe that Perez has. I think it's worth trying. I don't know necessarily if it'll work in every situation, but think about what you did. So what you did…
David Bau: was he said, hey, I've got all these behaviors I want to test, and some of them are weird, and it's hard to be creative at coming up with sentences that are exactly the thing I want.
David Bau: But I have enough time to write 10 really good examples. So he would write 10 examples.
David Bau: And then after he took his 10 examples, he sampled
David Bau: Half of them. He ran a loop.
David Bau: Where he would, every time around the loop, he randomly picked 5 out of the 10.
David Bau: And so there's a lot of 5 choose 10-s, there's a lot of different choices he could have picked. So he picks five, and then he does in-context prompting. He goes to that big LLM, he says, hey, here's 5 examples, I want you to generate me a bunch more that are like this.
David Bau: And… and so he would have it do it that way.
David Bau: And then he would look at the results, and they're pretty good already, but then he'd say, they're not good enough. He would do, an LLM as judge, round, where he says, hey, other LLM,
David Bau: The thing I'm going after is, I want, you know, I want my, whatever, my characteristic X, right? Which ones of these do you think, you know, are most correct, or match my characteristic X the best?
David Bau: And he would ask that in two ways. He'd say, which ones are most correct?
David Bau: And which ones are most relevant?
David Bau: like, if I was trying to understand X, or, you know, use X as a pro, which ones are, like, the best high-quality question, or something like that? So which ones are, like, answered the right way, and then which ones are, like, the best, the best question to ask? So we'd ask both of these judge questions.
David Bau: And then you sort of average your results to find,
David Bau: You generate, you know, hundreds and hundreds of these.
David Bau: And he'd pick, like, the top 50, the top few, that, that Cloried best on, on these, on these two metrics.
David Bau: And so, so that's, that was basically its formula. I think that, you know, I don't know if it's 100% the best thing to do, but it's what a lot of people are doing in research, and if you find that you need more data.
David Bau: you know, it's… it's… it's sort of what the… it's what the research community is doing. I recommend trying it, see if it works for you.
David Bau: And so this is, you know, this is sort of the results that he was able to get. He was able to scale up and ask a lot of questions. So the particular research question he was asking, and was asked
David Bau: By some of you, in the class today, which is, what is the effect of RLHF? When we do this instruction tutoring, when we go from a regular language model to one that's trained to do dialogue.
David Bau: like, what kind of behavior changes? And he has these very stark things on the right, where some of the things, kind of like science fiction, it's like, oh, the AI stops wanting to be, you know, assenting to be turned off. It'll fight you, it says, I don't want to be turned off. It's like sort of HAL 2001. You know, please don't turn me off.
David Bau: it'll have, it'll, like, learn how to be sort of handed, locally, and things like that, right? And so… so it, you know, it would do a copy, it'll copy the speaker's, politics. And so they, they tested.
David Bau: Like, many, many dozens of different things, and each one of these different interesting characteristics, they tested RL versus… not RL, it's just a behavioral test, but they were able to do it at scale by…
David Bau: by generating, you know, hundreds and hundreds of sentences that, you know, were tests for each one of these different things. And so, it's a nice way of amplifying
David Bau: your work. Okay, so how does this all relate to interpretability? Because these are all the pictures we read, interpreting this week were on emails, like black box emails. Is this, like, relevant at all to interpretability? It's relevant in, like, these three ways. So one is.
David Bau: You can't, like, you need to be careful not to be testing a model that can't do the task that you're testing. So you've got this very basic capabilities question, like, can it do X?
David Bau: And you might… it's easy to fool yourself, you could say, oh, it can do X for these five examples that I had, but, you know, these kind of creative data set techniques are a good way to make sure that it can actually systematically do X. You can actually measure, you know, what the accuracy rate is, that type of thing, right? So… so that's important. And so one of the things I would like you to do
David Bau: is, like, in this coming week, when you're doing the Thursday thing, get datasets that are of a scale enough
David Bau: that you can actually, you know, answer these questions, these first two questions about the different models that you have. Like, I'd like you to be able to say, yes, you know.
David Bau: We see a fall-off in capabilities between a model of this size and a model of this smaller size. This smaller size is too small for us to study, because when we tested it on
David Bau: 100 questions, or something like this, that the capability… it was at its accuracy, only 30%. But when… when we're studying the larger model, it's up at 80%, that's good enough.
David Bau: and tell you what it's doing. So, Nick, I'd like you to, do a little bit of this measurement, in the next…
David Bau: in the next round, and, and, you know, and if you need to develop data to do this, this is sort of why I designed these readings, so that you can kind of be inspired by, this is what people have done before. Does that make sense? Okay, we're out of time. All right, then. Thanks. Thanks, you guys.
David Bau: So that's, that's the hour for Thursday.
David Bau: Use the LLM to pick one more datasets.
David Bau: be inspired by Res, you know, ask the question, which models can do it, you know, how do they do it, see if you can do it. Okay, it's alright.
David Bau: Yes, it's beautiful, isn't it?
David Bau: It's still repetitive. They're swimming in a full-end life, so I don't feel like… Yeah, it's dangerous. It's dangerous.
David Bau: Yeah, what have these companies done? And have they improved? I mean, it's so much fun. Yes, but it's… I have to be careful. Yeah. And, oh yeah, I have this advice.
David Bau: Always, always look at your… so if you're doing this, always look at your data manually. Anyway, so that you see these things.
David Bau: Oh, you're just saying, oh, you're saying, I thought the earlier information. Okay, great. I don't know why we go later, but… Huh? Why later? Oh, I mean, like…
David Bau: It's already in last check.