David Bau: Thanks.
David Bau: Then, we'll try.
David Bau: Democrat's presentation.
David Bau: Thank you.
David Bau: No, I think it's good that we have a unit tag. So now we just have to figure out if we received it.
David Bau: Same, right?
David Bau: That'll be instant.
David Bau: Probably not.
David Bau: Pleasure.
David Bau: Anybody?
David Bau: I don't know… I think that there's a lot of students, I guess.
David Bau: This corner is the worst corner, Patrick, right? So I can… I couldn't be the one that's sitting here. It's okay, I don't mind sitting here, it's all right.
David Bau: That's a pleasure.
David Bau: So we can go back and save.
David Bau: Lucky
David Bau: Bye.
David Bau: That'll work.
David Bau: It's still, yep, it's still my room. Yep, they can use the same one. Yep, I'm here.
David Bau: So, yeah, so I just, I always use the same for everything.
David Bau: I haven't had a problem with the game.
David Bau: Do you know, some parallel? What?
David Bau: But then I'll give somebody else's Zoom, yeah, tip of the meeting.
David Bau: That's one of them appears to me.
David Bau: But then some… the real problem is when I'm hosting a meeting.
David Bau: If I have some other meeting on the same line, then that's a problem. I don't have a way of transferring, like, 50 people over to a different one, I just, you know, like, that's a… that's an issue.
David Bau: So that's the only problem that I have. And I have to tell people that I have to move… at the last minute, I have to move islands.
David Bau: Right? That's… It only happens last one up.
David Bau: I usually try to avoid putting meetings right after I send them.
David Bau: Right?
David Bau: So, I said, I went okay.
David Bau: You proud of your paper?
David Bau: Anybody else for ICML?
David Bau: There we have a few.
David Bau: We had a few in the lab, right?
David Bau: Thank you, man.
David Bau: When he started panicking about getting his paper ready?
David Bau: And, I saw Karam working really hard, I don't know if he ended up slipping through.
David Bau: I don't know if you're following that at all. We… you're not in Crim's paper, right? No. Why would it… I guess it was cold. Yeah, instead of following, yeah, he has to decide if he can fall under the…
David Bau: So, the reviews will be out of the word and call it himself.
David Bau: Oh, the first round of reviewers will be out, so maybe it's worth it.
David Bau: I thought we didn't do. I don't know.
David Bau: I think Colin would be a better… better dreams than him, but it's okay, try adding that.
David Bau: you know.
David Bau: If he was my PhD student, I'd tell him, relax! Just go for column.
David Bau: But he's a master's student, so he's really worried about, like, making his resume look good.
David Bau: Like, every venue, he's gotta try every one. So, do it.
David Bau: But I'm really happy with Kevin's paper, with the…
David Bau: Turns to see another day around.
David Bau: So… I have to… forgive me.
David Bau: So, I haven't even… Damn.
David Bau: How many minutes we've had with him?
David Bau: Are we supposed to have started? We're supposed to have started.
David Bau: Okay, so whose… whose team slides are these?
David Bau: Team S.
David Bau: All right, anybody else on here on Team S?
David Bau: Are you guys ready to present?
David Bau: I'm not sure what they want to stay, beyond.
David Bau: If you guys are not prepared to turn that because you're missing the rest of your team, we can reshuffle.
David Bau: The other thing, let's see, when you… let me click on the next admin.
David Bau: So, if your team is not here yet?
David Bau: That's okay.
David Bau: You're in some other random order.
David Bau: Who's this?
David Bau: Team excellent. Is team excellent here? I'm still waiting for it.
David Bau: Oh, okay.
David Bau: Holy cow.
David Bau: Is it? Okay.
David Bau: Who is this?
David Bau: I don't see Jasmine. Yeah, I don't… Jasmine's done this… Oh, Jasmine, are you guys ready to present?
Jasmine Cui: Maybe not.
David Bau: Maybe not? What's going on? Okay, is there a team that's ready to present? How about we start from there? Yes! Better, better, better proposal.
David Bau: Team… team bias, team power.
David Bau: Okay, all right, so find your slides.
David Bau: And present, and okay, so now what I'm gonna do this time, go ahead, is I'm gonna… I'm gonna put up a 9-minute timer.
David Bau: for everybody. And so, so, I didn't announce this in advance, I apologize,
David Bau: But I'm just gonna set a buzzer off at 9 minutes, and we'll say thank you very much, and we'll have a few minutes of discussion, and then so on. So I'm just gonna… I'm just gonna truncate you, so you guys allocate your time any way you want. It's not necessary that everybody speaks, it's okay, but 9 minutes.
David Bau: Okay. Yeah, that's okay. So we don't have a slide deck, we just, like, have a, Google Sheet, with some, ideas about, different data, sets that we're composing. So we're just gonna talk through a couple of the different things.
David Bau: that we're dealing with, and, like, some of the questions that are coming up. So the first one that we've been… so, routine power, we're interested in how, models are kind of internally representing power, and then what you can do if you interpie on those representations. So, one thing that we were toying around with, was,
David Bau: style sentences, so if you take a bunch of sentences, can you make the font really big on it so that we can see some of the sentences?
David Bau: Very good. I just, yeah, oh yeah.
David Bau: So, if, you have some sort of sentence that this was asking Cloud to join a series of sentences at the end of the word power,
David Bau: Remove the word power, feed it into a different model, and, look at how, the model activates.
David Bau: the, sentences that we've been kind of playing around with, do activate power in different models, so it will produce power. And so that was kind of, like, one starting place that we've been,
David Bau: playing around with. There are, like, some, I don't know, basic issues with this kind of setup that we've been struggling with, too, so Ari that you want to talk a little bit about,
David Bau: So,
David Bau: One of the, like, things that may make the word power highly likely in the end is, like, the use of some words before is, like, exercise. If you search exercise off with no context on Google, it made power.
David Bau: Or maybe different combinations, like, influence,
David Bau: or other words that are somehow related to power. So, we also have a data set of, sentences with
David Bau: None of those Sinanese.
David Bau: We haven't tested this, but… Maybe we can't sleep.
David Bau: Yeah, that's start somewhere.
David Bau: Yeah.
David Bau: There is still some words that may be related to power, like decision, but at least
David Bau: We can see… we can feel there is some… some sort of power here. For example, nobody left until he…
David Bau: Well, Ben screamed off for coming post. They waited 3 hours before he arrived. It seems someone was…
David Bau: Okay, so I'm both. But does the LM have, that…
David Bau: understanding of how our ads can do or not.
David Bau: Maybe one question. Oh, nice.
David Bau: And he decided.
David Bau: The meeting will meet when she is down, I'm going, wow.
David Bau: That's a power move.
David Bau: That's good.
David Bau: Well, that's interesting. How did you generate these sentences? So cool.
David Bau: Oh, just with that thump. Just with that thump.
David Bau: My favorite thing on spreadsheets is wrap mode.
David Bau: Put a wrap on it, yeah.
David Bau: I agree.
David Bau: I bet.
David Bau: So then the third thing that we were, playing around with was, non-LOM-generated datasets, and whether or not there's some sort of more, kind of, like, bespoke.
David Bau: sentences that we could work with. So, last time we were talking about, different ways of, kind of, scoring power relationships within sentences, and what we could do with that.
David Bau: So we, started pulling just some recent news articles, and seeing if we could use that to generate sentences that have power relationships between individuals, and come up with a…
David Bau: close to ground truth score, or some sort of external way of measuring power in those relationships, and then, asking the L, and then to generate the score. So, how can you talk a little bit about that.
David Bau: If I can pull the clock here.
David Bau: Oh, you're on the planet. I mean, one pot.
David Bau: Yeah, so this is, kind of…
David Bau: A tool for scoring the power of different regions and sentences.
David Bau: It's called Riveter. I briefly mentioned it last time.
David Bau: and just as kind of a quick test.
David Bau: We, like, manually just copied over 10 recent AP news articles.
David Bau: And we ran this tool, and for, like, the…
David Bau: Most referenced entities, we were able to quantify
David Bau: a power score, which is not that surprising. And so you've got high bar and low bar, what do those mean? I can't really… Oh yeah, it's a bit small. So high bar means that entity has, like, more power, and low bar means it has less.
David Bau: So, the recent news articles are about Professor Silvon.
David Bau: So, like, authorities have a high bar score, protest, and trainings have lower.
David Bau: And we also ran this on just a thousand APUs articles from 2019, got kind of a broader range of things.
David Bau: The text is probably too small here, but there are some common themes, like authorities, police, rating high on the score, Republicans.
David Bau: things like that. And… Yeah, so this tool is using a lexicon of verbs labeled by crowd workers, for whether they, like, increase or decrease the power of the subject, and the…
David Bau: It's based on an analysis of the whole article, is that right? Yeah. So…
David Bau: It's kind of just… like, it's not using a…
David Bau: LLM to do the scoring, it's kind of based on that, like.
David Bau: But it is doing things like co-reference for speech.
David Bau: So, I guess the idea here is that this is our own truth.
David Bau: for future buildings. If we're trying to get
David Bau: like, an idea of which neurons or which circuits are responsible for power, but we don't exactly know what we're going to be prompting the model to do, if it's, like, maybe
David Bau: higher, lowest power entities, and… Have people tried triangulating Riveter with LLMs?
David Bau: you know, assessment, like…
David Bau: whether an LLM thinks that this is, you know, more power or less power, and if it agrees with what Riverter would say, or even you could give a few examples of River and say, you know, how would you generalize it and see if the general publication agrees with how River would generalize it?
David Bau: a lot of examples. I feel like there's a… it's an interesting…
David Bau: I've been there. You know, it's likely that MLMs understand how to read text better than earlier guys, but the real question is, like, oh, does it have… can you calibrate it to the same sense of power?
David Bau: But it might be… it might be a useful tool if you want to generate more text, or… or something like that, and you could use a bunch of river scored examples.
David Bau: You know, once you have a sense for the population.
David Bau: Yeah, I think anyone has done that.
David Bau: There's… Cool.
David Bau: Do you get, like, scores for particular sentences, or just averaged across content?
David Bau: That's… that's entity per article, right? Yeah, I think it's for…
David Bau: Entities, you can get it for individual sentences,
David Bau: I guess I'd just be curious, because, like, I see, like, China is really low power, but I would imagine in certain contexts, it's a lot of power, and so I, like… Right.
David Bau: So you just averaged over a thousand articles. And these are pretty rated, but they're from 2019, so there's COVID stuff.
David Bau: Interesting.
David Bau: Interesting. Oh, yeah, where'd you get? So where'd you get those articles from?
David Bau: Just the ID-based dataset, and there's, like, a big scrape of resources.
David Bau: Nice. Interesting.
David Bau: Wow.
David Bau: That's your 9 minutes. Any other last things in the last 10 seconds?
David Bau: Any comments or suggestions from people?
David Bau: Discuss.
David Bau: So they have a whole bunch of data here that they've played with, and
David Bau: So, so you, you suggested, oh, they could be used for making a pro.
David Bau: What's this leading to for, like, a… do you have a sense for what kind of research question you might want to…
David Bau: I'm learning… I gotta learn how to use this thing. Okay. What kind of research question?
David Bau: you know, your tools that you've played with so far are sort of suggesting that you might want to ask? I can kind of hop in, but just to make sure… sorry, we've covered the generated census, right? Yeah, okay. One… this goes back to Armina's experiment for the last time we presented, but I want to thank, and when she generated those.
David Bau: sentences, one thing we had discussed as a group was that, the pattern we saw originally was that the decision it makes pivoted in the last few layers. One research question would be, like, would be honing in on that kind of idea, saying, look.
David Bau: Are there particular layers? Are there certain decisions that seem to get subverted or altered? Are there… that can help identify, like, are there layers of alignment? Are there hidden opinions that are relatively consistent? That's one… that would be one research question.
David Bau: True.
David Bau: I see. Like, I didn't… I didn't subversion. So, one of the things I'm a little worried about, what's… I love that. It's such a cool story. I think I mentioned to you, like, I'm not sure…
David Bau: like…
David Bau: You can… so usually you want to triangulate your results when you have some innovative method, and the main evidence you have is that the opinion was different at a certain layer.
David Bau: From the opinion that it gives at the end, that's, like, one leg of the stool. But then people will ask you, well, what does that mean?
David Bau: What does that mean? Like, you're using the model in some way that wasn't designed to use. Is there a second leg of the stool, or a third leg of the stool, where you can say, that's evidence of some other behavior, or it matters because it shows up in this other situation? And so that's my…
David Bau: concern about… I love the story. I think it's so cool. I'm just not sure what you're doing, so I… so I don't want to dissuade you from advice. You know, if you want to go that way, I want to encourage you to be… Question.
David Bau: I assume either through pro… if there are some activations, we can locate that, which… that would alter the decision downstream. Would that… would that contribute? That'd be a second link. It could be a second link. I mean, it might be very closely related.
David Bau: It could be something.
David Bau: So you could say, oh, we hypothesize that the model has a concept of Something.
David Bau: Whatever, some aspect of power, that it processes, you know, that it's impolite.
David Bau: To… to talk about power, something like that.
David Bau: And, and that activates… use this as concept somewhere at Layer 27.
David Bau: Or different layers of the model, and we have assessments for figuring out what layer it is. And we have an ability to turn off this intuition
David Bau: Right? And maybe the third leg might be… and we noticed that if we look at models that are trained one way, and models that are trained a different way… Oh, the instructors, if they don't have the inhibition, or something like that, you know, let's say maybe that could be a third leg, right? So… so I think it's possible to make a…
David Bau: Make a tripod.
David Bau: Right, but but… but maybe there's something to explore?
David Bau: You know, with more… Preliminary experiments before you really commit to it.
David Bau: Does that make sense?
David Bau: Any other suggestions?
David Bau: I mean, not a suggestion, just a comment. I like the blank…
David Bau: power sentence, like, kind of like those sentences that, like, power's not explicit, but you could sort of… and I wonder if you could…
David Bau: Yeah, have, like…
David Bau: varying data sets for the different kinds of power you want to test for understanding that for something that are similar for this, where, like.
David Bau: It's implied, but not explicit. I like his pun.
David Bau: Yeah, I like it too. Like, it seems like there's different types of power in here, probably, right? Do you… like, I wonder if it's… so, have you guys considered…
David Bau: Trying to sort through different conceptions of powers within the model hands.
David Bau: Yeah, we've been narrowing down on, like, specifically relational conceptions of power.
David Bau: people. We haven't kind of fluttered this.
David Bau: for that, and thinking more in terms of, there's a question of, like, the, kind of, like, latent possibility of exercising power, if you're a position of power, rather than the actual exercise of power, and those are kind of two distinct things as well. So thinking in that sort of relational context between two entities. Interesting.
David Bau: Yeah, within that, that can, like, go in kind of different…
David Bau: direction. So, we were thinking more about the kind of, like,
David Bau: Diversity of context to make sure that we're picking up and,
David Bau: the conversations we've had so far, and haven't talked as much about it like this.
David Bau: arrangements.
David Bau: Yeah, that sounds, sounds fun.
David Bau: And I was like, keep it.
David Bau: Who's the next team?
David Bau: 8.
David Bau: Alright, Jasmine!
David Bau: How come you're remote today?
Jasmine Cui: Just under the weather, sorry.
David Bau: Well, I hope you feel better. Thanks for dialing in.
David Bau: Is your team ready to present?
Jasmine Cui: Let me… let me ask them, I'm not sure.
Jasmine Cui: Okay, so…
David Bau: I can put you guys up on the screen.
David Bau: If you're not…
Jasmine Cui: I think maybe a pass for today, if it's hard.
David Bau: Okay, should we, should we, should we… are you, are you entertained?
David Bau: I, I could just go through my slides.
David Bau: Are you a TMTK? Yes. Oh, cool, yeah, why don't you go through your slides?
David Bau: Jasmine, we'll have, we'll have, Aria… Is that fine?
Jasmine Cui: Yeah, yeah, yeah, yeah, no, Aria's great, yeah.
David Bau: I don't know where…
Jasmine Cui: Favorite superstar.
David Bau: You know what she looks like.
David Bau: Yes. Yes, indeed.
David Bau: Probably very small on the screen.
David Bau: I forgot to put the, font size is bigger, but… Anyway… For dataset.
David Bau: I want to start off by, like, asking a question.
David Bau: Cause…
David Bau: I know this class is, like, mostly about identifying a concept, and, like, the guidelines we have for building a dataset
David Bau: Mostly focused on, like, if you have a concept, you… this is how you do, like, the constructive pairs and stuff, all because you demonstrate that you, like.
David Bau: have an item that has this concept and one that doesn't. Right, for example. But, yes, for example.
David Bau: Like, for, for ours, We were focusing on something slightly different than, like, you wouldn't think of as a
David Bau: Mechanism, like, an algorithm of whatever day, which makes…
David Bau: constructing this data kind of different from the guidelines? That's okay, that's totally okay. That's… so that's alright.
David Bau: you know, the guidelines… I mean, so, you know, the real goal of the paper, or the real goal of the course, is to, like.
David Bau: produce a piece of interesting insight, about how these LMs are thinking.
David Bau: And, you know, we have suggestions on how you might go about it, but if you have a clever general that's, you know, A+, okay?
David Bau: I don't have a clever way, I just have a different way, that might not work, but… Anyway…
David Bau: because we're… our tasks were to, just to, catch up, people, have a transcript that is undifferentiated by speakers, and then have the LMs kind of attribute each sentence to, like, the crystalline speaker that spoke at.
David Bau: So, in terms of, like, a data set, like, an easier setup, we want, essentially, a discourse where you have to infer the speaker identity without, like, tags, and essentially, we need
David Bau: to have a question that's on the discussion within the dialogue, so that we could ask questions that… I love it. …the interpoles would then operate on.
David Bau: And, like, the data items should be short. It wouldn't be, like, a full transcript, it would be something shorter, because
David Bau: you… To make it not ambiguous, you cannot have the…
David Bau: Instances where a speaker just starts speaking.
David Bau: So, like, the next turn have to be… like, the current speaker must select the next speaker in some sense, either by, you know, referring by names, or in, like, a row setting of, like, you have an interviewer and the interviewee, the person who asks questions, the person who answers, you would know who is, like, taking that role.
David Bau: So anyway, there's a bunch of, like, nuances in there, because there's a lot of things that you have to…
David Bau: Satisfied to make such data unsolvable.
David Bau: in terms of, like, for example, you… the logic must match, and, like, there's gotta be hints that you could take. For example, if someone asks the same question twice.
David Bau: you already had someone answer it, the second answer would belong to a new person. Sure. Like, things like that.
David Bau: And, like, the interviewer interviewee is, like, an example of, like, role assignments that would help avoid this…
David Bau: And there's, like, inclusive clues of, like, if you mention someone's name, you're talking to them. So cool. But, so… Oh, so you have an example transcript here? For data items…
David Bau: I'm separating it into, like, my levels of difficulties, starting with, like, a just labeled transcripts, which… Can you hit the present button to make the fonts a little bigger here? I'm just curious to see who's, because they're all JSONs. Yeah, it's very… Better? Yeah, it's much better. Okay. Yeah, like, this will be super easy, like…
David Bau: Even the smallest, like, 1.5, you don't want to consult this, because it's, like, it's tact.
David Bau: And,
David Bau: So, wait, wait, the transcripts, literally, let's read it here, so the way I get… so I get an idea. So, Alice, I'm Alice, I'm Bob.
David Bau: I like basketball. I like soccer. Okay? And then now the QA pairs are… What sport does Alice…
David Bau: Like. Like? Well, Alice likes.
David Bau: aestha.
David Bau: I see.
David Bau: And so, these are just questions about the dialogue. Yeah. Okay, so it's pretty straightforward, like you were saying. Yes. And then, if it… if it turns into…
David Bau: And I'll label the client. Oh, without Alice Colin. Now you don't…
David Bau: Now the model, first, it must understand the task of the transcript of a conversation, then it starts turn-taking, so then you would know Alice is, like, saying that word. You would… Bob's second. Yeah. Alice is third, Bob is fourth.
David Bau: Where it gets a little more interesting is when you have three, or, like, more than two speakers, where now turn TikTok, and it's not, like.
David Bau: the next line have to belong to, like, the next speaker. Yeah. This is, like, an example where you…
David Bau: Oh, hi, Alice, yes, I like… yeah, I heard that Claire likes basketball, too. Oh yeah, Alice, yes. You would have heard that this line, basically, is spoken by Claire. Yeah, you're right. And, like, now that, like, you would have to also understand that when Alice asks
David Bau: Again, this answer cannot be clairsent class, it has to be Bob's.
David Bau: It's like, funny, transgirl thing. Oh, right, because… Because, like…
David Bau: Because it would be weird for Claire to go back to Alice and ask, what do you like? Because Alice just said that I like basketball. Yes.
David Bau: So… so it makes more sense for it to be Bob.
David Bau: Oh, what a cool transcript!
David Bau: If you can explain that one, that's pretty awesome.
David Bau: I like it. What do you guys think?
David Bau: Isn't that neat? This happens all the time when you're reading fiction, have you noticed? You read this dialogue in fiction, and it's like, quote this person, quote that person. Sometimes it gets a little long, sometimes the writer's like.
David Bau: You can follow, but once in a while, you have to sort of pause for a second and say, wait, who's talking? Yeah. Right? This is, like, an example where I tested Quinn on 1.5B, and it's instruct fair, 7B, and it's instruct fair, and then, just for fun, a reasoning model, Q over Q, 32B.
David Bau: This is, like, a case where the reasoning models can, like… They can do it? Yes, they can do it. But they also waste a lot of token thinking through, like… And they think through it? They reason through it? They're like, just like the way I narrowed it. Yeah.
David Bau: They're like, oh, that couldn't be wrong. They go on and off, they backtrack, they do, like, every single permutations of, like, how I could assign it. And then for the instruct models, they just suck.
David Bau: Really, they can't do it.
David Bau: No. And what about the reasoning models if you don't let them reason?
David Bau: Oh, I have not tried it. Okay. I wonder.
David Bau: Anyway, and then there's level 3, where I actually took this from, the transcript that Jasmine gave us, and, like, just,
David Bau: But to make it shorter.
David Bau: This is harder to explain, but… because it's so small. But basically, the setup looks like there's one interviewer…
David Bau: There's two interviewees and one interviewee, and thus, if there's two questions being asked consecutively, both of them have to belong.
David Bau: To the interviewers, and, like, through that, you could infer the roles, and…
David Bau: Take a question, and then ask about their stance.
David Bau: And,
David Bau: Oh.
David Bau: Quickly.
David Bau: For the… for the natural stretch, like, one, this is…
David Bau: By looking at the reasoning model's traces, I see some cases where the model could, like, understand what they're talking about, they could summarize people's saying.
David Bau: But notice how, when the models ask about Joe's perspective on, like, policy uncertainty, it refers to Richard's analysis. It basically collapsed the discourse into just, like, one narrative.
David Bau: And, yeah.
David Bau: I don't know if this happens to instruct models, because it doesn't really say much.
David Bau: But… here is an example of where I'm, like, the made-up three-person task, the instruct model, so it's…
David Bau: It's, like, an obvious mistake that's just, like… I don't know.
David Bau: Exactly. So, and how many, and how many… when you say it sucks.
David Bau: How many, cases did you try through these different models to sort of measure?
David Bau: Like, 10 of them? Yeah, 10 of them. Not much. Okay.
David Bau: because of… the…
David Bau: these above reasons I have not figured out how do I have data generate things in, like… With this form? …bunch, or, like, in this form. There's gotta be some kind of… I was hoping there's gonna be some kind of templates that I could take from, like, linguistic research that…
David Bau: the finds, things like that, like… But have you just… have you tried going to any of the big models and just saying, this is what I need? Just, like, pasting the text in it? I, I, I did, but, like…
David Bau: Oh, sorry. Sorry.
David Bau: It's just, like, you would have to manually go through them to actually, like, figure out if…
David Bau: It's valid. And also, they tend to, like…
David Bau: Some, like, like, the clever ones, like this.
David Bau: where Claire is, like, introduced into the conversation instead of starting with, like, I'm Alice, I'm Bob, I'm Claire.
David Bau: It's very hard for a bottle to come up.
David Bau: You could give a few examples. Yeah. Like, if I write one of these and a few-shot them, maybe… Yeah, which is, you know, that's what,
David Bau: That's what Franz did.
David Bau: Right? Yeah, more.
David Bau: Very nice. So, any other suggestions for Team TK? Are you guys interested in this setting, or would you change it in a way to make it more interesting? What do you think?
Jasmine Cui: Oh, I just wanted to note, too, that, like, some of the really big instruct models can also do the task, so, like, I just tried it on OpenRouter with, Quen3VL235B, and it can, like, it can do, like, the annotation
Jasmine Cui: I think, like, the size might be, like, a challenge, just cause… Yeah.
David Bau: How many bees, you said?
Jasmine Cui: It's 235B, but it's a… it's a Moe, so…
Jasmine Cui: I don't know how many parameters are active at a given time, but .
David Bau: Yeah, we should, we should ask the NDIF team to help you with that.
Jasmine Cui: Yeah, yeah, I think for some of the more challenging task settings, we might need to use bigger instruct models to be able to see the behavior, but…
David Bau: We love it.
David Bau: Yeah, that's great. So, it's really good to know this. So, like, this… the testing that you're doing, is…
David Bau: from my point of view, it's perfect, right? It's like, right, basically… You know.
David Bau: You want to avoid wasting your time.
David Bau: Study a model that can't do the task.
David Bau: Right? And on the other hand, you also want to avoid wasting your time setting a giant model that's oversized for the task, right? Because it is very inconvenient to study big models.
David Bau: So, so, you know, you sort of want to understand what you're… what you're facing.
David Bau: So what is… so how do you… how do you feel about the situation now that you've probed it a little bit? Do you feel like it's promising?
David Bau: I think… Yeah,
David Bau: I like to, try to do interctals on that, instead of just popping at first reason.
David Bau: Because… You like to what? I like to, like, see how… like, see how base models suck at it, like, why does it suck? Or, like, why does it…
David Bau: Yeah, I don't know, I'm just…
David Bau: This is, like, just one of the tasks. I would also want it to be more diverse than…
David Bau: like, then I just fit this through LMs, and it just replaced names and stuff.
David Bau: I think that's fine. Yeah, the fact that, like, some models cancel it makes it, I think, better, because otherwise it's very concerning.
David Bau: Right, right. Some models can definitely do this. So I, I, like, I don't know, like… right, go ahead. Oh, I'm just saying, I like the different levels they created, so…
David Bau: And I'm just wondering, so what the next step might be asking how the model solves the problem, right? And especially, like, how the model solves the problems differently. Especially if you could…
David Bau: For example, you could, like, patch a particular
David Bau: sentence from Level 1 to level 3, and see, like, how that…
David Bau: change things, I don't know. To answer, like, how the model, like, from, approaches, Different levels differently.
David Bau: But I like the level of different levels, so… Oh, yeah.
David Bau: This is a little silly, but I feel like it would be interesting to see your eye and something at the end. Like, someone be like.
David Bau: I love football, and then the next lines would be like, who's that?
David Bau: And then make it into a closed prompt, yeah. I think that would be, like, that would really show that it's also reading, you know, and it's understanding these concepts, and not just, like, finding a pattern or finding a speech pattern. I try to ask for, names that doesn't exist.
David Bau: And, I don't know what the phase model does. Instruct models answers as of…
David Bau: I'm asking about, like, Alice. The reason it all those goes, because the user's asking about this, this person must exist, and then proceed to attribute whatever sentences that they're unclear about to that.
David Bau: Not existing versus name.
David Bau: Yeah, thanks. Bye.
David Bau: Thank you. Okay, next team, next team. Team S? Is Team S ready now? Yep. Okay, Team S.
David Bau: 15 minutes.
David Bau: I don't know where to go. You find your thing. I think this is it.
David Bau: Alright.
David Bau: Whoa.
David Bau: We have a little bit different prompt design based on the last week's material.
David Bau: So we first do, like, a binary class, then we do a pairwise comparison.
David Bau: And we, do a multiple-choice pairwise comparison, and then…
David Bau: We do an attribute-specific class. Move on to the next page,
David Bau: So, for the banner clause, it's basically you show a… just one picture, and
David Bau: the model to say, like, if, this urban scene appears to be what? Like, we can't give them a pair of either safe or dangerous, either wealthy or poor, something like that, and we do, we…
David Bau: do a logic lens, inside the model, to see how it,
David Bau: How, what we have in the inside.
David Bau: And move on to the comparison. This one is basically, the same task that we have for, the MIT, place pause, test.
David Bau: Which is giving two images, making the model, say whether, which location, of these two,
David Bau: pictures looks safer? Is it on the picture on the left, or is it the picture on the right? And, at the meantime, we want to also, like, look into the logic… the logic list to see what it, do… where it does the decision.
David Bau: And move on to the next slide. This is, the…
David Bau: just a little runtime that I do open, and we do the first 100 tests, with the two, pictures here, and asking which image, which images is either wealthier, more beautiful, or something like that. Reply only with left or right,
David Bau: If uncertain, reply equal.
David Bau: So this is a result compared to what, in the dataset. So, you see, like, right here, like, for,
David Bau: When asking concepts about, beauty, it matches with the human status pretty well. But actually, when it… when we try to, like, ask whether it's safer or, like,
David Bau: more depressing, there's, like, zero matches. But for… I think for the wealthier part, it starts to, like, just pure guessing.
David Bau: And, now this…
David Bau: I, I will probably try more models for this task, and probably more, data.
David Bau: Move on to the next slide. So this, is, a, multiple choice, question. So, we want to ask the model, how does… how safe does this tree appear? So instead of, giving them a,
David Bau: lighting the rate for each score. We think that this one is better, because, it's more straightforward for the language model to understand.
David Bau: And we first, just do a pure prompt, prompt, and then we do a comparison. Like, we give it another picture, and to, make…
David Bau: Then, like, compare and to have the, how do you say, like, the, the scores right here.
David Bau: And this is basically how we want to design our future, experiment.
David Bau: Right now, Lex, let's move on to the…
David Bau: Yes, and this is also… yeah, sorry, this…
David Bau: The last… there is another one called Attribute Specific Class, where we want to see, like, in scale, the degree of, perceived… the perspective of one of these places.
David Bau: Like, we want to see, like, the decoding from, too high or too low, like, whether the model thinks that,
David Bau: If this picture fits in, the description that we provided.
David Bau: And, that's all for the…
David Bau: methods. So, before I start with my experiments, I want to give a context about vision language model, because it's different with language model. We have an image here, and the image goes through the, BIT, and we get the image token.
David Bau: And the, different lists, like, earlier vision language model, like.
David Bau: Nava, the image token doesn't go in the…
David Bau: tax decoder, so, it goes through the, cross-attention, so the, tax token
David Bau: And look at the image token through the auto tension, but the
David Bau: image token, it doesn't change in the decoder. So, one way to visualize that, how, how the image…
David Bau: affect the final decision is to look at the attention heat map. Like, we use the last token's attention on the map and visualize it through, different layers, and, it's really hard to find the patterns at the beginning, but if you keep looking at
David Bau: this for, like, one minute. Maybe you can find, like, the… at the beginning, like.
David Bau: earlier layer, the attention is maybe focus on, the adage of the image, but when this comes to, like.
David Bau: layer 20, around layer 20, it begins to focus on the center of the image. So, an initial guesses, the model starts to look at the image, or look at the contents in the image at layer 20, and so there's, the…
David Bau: some, initiatives. It's a small group.
David Bau: Yeah, I divided into 3 different…
David Bau: Stage, and the first stage is to spec for random, and focus on the adage, and the second stage is focused on some, real functions in the image.
David Bau: in the, I know a few layers, it doesn't look at anything.
David Bau: So, there's another thing interesting is, if we try to visualize the… Attention.
David Bau: For the image token.
David Bau: We can see, at the beginning,
David Bau: Very beginning is… it has something, but it dropped very quickly, and
David Bau: to close to zero until, layer 20, so it grew up very quickly. So, what I think is, maybe the model doesn't really look at the image token.
David Bau: the, first, like, Tanji Liars. It's just more focused on the tax token and, in, like, liar…
David Bau: 1819 is our duplicate image.
David Bau: And also, I try to,
David Bau: see, like, the probability of the final decision, like, safe or unsafe. So, from this figure, we can,
David Bau: Find that the final decision The probability of safe or unsafe There's…
David Bau: Close to zero for all the liar until…
David Bau: layer 21, and they start to go up. So, I think this…
David Bau: Three things say the same thing, that maybe in
David Bau: The first 20 layers, the model tried to understand what the question is, and after the model understands the question, it starts to look at the image and find some useful information, and after
David Bau: like, there are plenty that start to make the decision, so that's something I found interesting here. Very good hypothesis, yeah. It's pretty neat. It's sort of circumstantial evidence, but it gives you a sense for what might be going on. You can try to prove that later. Yeah, I think this very close to, like, humans.
David Bau: Like, reasoning process, right? When we have this kind of question, we first
David Bau: try to understand the question, like, what kind of thing I need to find in the image so I can make the final decision. And then I look at the image and find something that I have my decision. Okay, so this is for… Yeah, so we're about out of time for this one. So, any suggestions for the team?
David Bau: Suggestions? You must have suggestions for the team.
David Bau: I'll throw something out there. It seems like a lot of the questions that you're asking are, like, very subjective, which is, like, very interesting, and I wonder what… if there's, like, a way to compare it to more, kind of, like,
David Bau: Straightforward, objective questions about the image.
David Bau: And you could, like, compare across those, like, is there a tree in this image versus, is this image depressing? Yeah, that might… that may… what if that logic happens at different layers?
David Bau: Right? Yeah, yeah. But, like, our topic is about, scene-level perception reason, but, we didn't look at the object-level things, but… But you could also ask very objective scene-level things. Like, is this outdoors? Is it indoors? Or things like that, right? I mean, very objective things, right? Versus…
David Bau: Right? Even if you want to…
David Bau: Yeah, that might be something, too.
David Bau: to try to compare.
David Bau: I've also tried a larger model, just like a 7D model. In fact, the reasoning is definitely desired. Maybe the larger model may have more reasoning process that trivia's higher, too.
David Bau: I like that suggestion. Any other good suggestions? Any other cool ideas?
David Bau: I think that's a very good one.
David Bau: You keep on thinking. I have some suggestions for you, but they have to do with some of the technical choices, but if you guys have ways of making this super interesting…
David Bau: And, like, yeah, I can give them some ideas, but for some technical things.
David Bau: So you guys are the only team working with, like, you know, a lot of images.
David Bau: And the thing that… so…
David Bau: we've been coaching everybody on how to generate a lot of data using LLMs to generate text and things, so one of the opportunities you have, you're going to be in a different situation, is to actually generate
David Bau: Some of your data sets by generating images.
David Bau: Right, so you're using real image datasets, you know, I can see, right?
David Bau: But…
David Bau: But, like, if you want a minimal pair, if you want to say, hey, I wonder what makes this image unsafe. Is it because there's a fire hydrant?
David Bau: you know, can I make a version of this image without the fire hydrant? I wonder if that'll, like, you know, cause the model to behave differently. This is very interesting, like, we can use diffusion models to generate a place that is not safe, and… Or, you know, for generating data, you don't have to…
David Bau: use a little cheap model, right? You could use Nano Banana or whatever, use whatever, like, the commercial ones are, right, to generate
David Bau: information.
David Bau: And so, and so you can do image editing and do all sorts of stuff, but then the question is, like, so, but you can be creative on how to do it. You can think about, like, are there… are there things that you want to do to generalize sensitive images so that they make an interesting asset that you can get a little bit more control over than a natural
David Bau: Data testing. Yeah.
David Bau: Yeah, the other thing I'll recommend you read is, so, you're looking at attention patterns for getting some intuition about things. There's this great paper written by a North Houston professor, Byron Wallace, and his students from a couple years ago, called
David Bau: Attention is not explanation, so it's a little caution paper.
David Bau: About, you know.
David Bau: inferring too much from attention patterns. So I would read that. There's also a follow-up paper from Naomi, I think, that… called, like, Attention is not, not accumulation, so it's a little controversy. So you can take a look at those two papers and see what you think, but we'll… so the other things for Frisian.
David Bau: These salience methods, where you're trying to figure out what part of the image.
David Bau: is important, turn out to be important, but we're not covering standards methods till much later. But we do have a stainless method expert on the team. He's a Fosaf Gabriel Smarty.
David Bau: who's one of the, you know, world's experts in salience methods. So, so have you guys been in touch with Gabriel? Have you seen him?
David Bau: Okay, so his name is Gabriel.
David Bau: And… and ask me about them later, and then I'll connect you. Thank you very much.
David Bau: Excellent. Okay, any other suggestions, any ideas for the team?
David Bau: Like, you might be interested in the Maya paper? Oh, yeah, the Maya paper, and I also asked Nikhil for his latest paper, so if you read… okay, yeah, so…
David Bau: Will you send it on Discord? Sure. Okay, look it on Discord. We'll get a couple papers to you.
David Bau: And, you know, attention is not explanation.
David Bau: And, Gabriel Starte's name, on Discord, and we'll get you…
David Bau: And then Nikhil, our TA,
David Bau: who's skipping out on being TA today, because he got his ICML thing in probably at 7am this morning.
David Bau: I fell asleep.
David Bau: But his paper that he just submitted, is also, a VLN paper, and very, you know, so you can, and it's all pretty…
David Bau: So, asking for his preprint.
David Bau: Yeah, then I'll have to get that preprint ready for you then.
David Bau: Okay, great. Who's next?
David Bau: Let's see, let's… I don't know who's next. Will you guys click on another slide that's hard for me to click? Thanks. Okay, somebody nominate yourself.
David Bau: I think that sounds fine. That's yours, like… Okay.
David Bau: Biants.
David Bau: Thank you, guys. It's such a beginning, it's the beginning. Yeah, they're so excellent. Team excellent.
David Bau: And so, yeah, I broke down and charged it.
David Bau: So basically what we've done in this week is just, like,
David Bau: Kind of, like, start cleaning our data, because we have a lot of data, which is the thing.
David Bau: So, we're working with earnings calls, which are these transcripts of the calls that companies do with
David Bau: people, like, reporters, every quarter in the year. And then transcribed? Yeah, so we had the transcripts, because my boss had them, so he just gave them to me. Nice. It's really nice, but we have so much data. So we have data from 2001 to 2020.
David Bau: And we have in total, like, 200… 200K, like, 300K transcripts.
David Bau: And then… Don't keep going.
David Bau: So the setup for the earnings call is usually, they start by opening up the opening statements, in which they give, like, a description of how things have been.
David Bau: And then there is a question, a Q&A, with reporters and analysts from, like, JP Morgan.
David Bau: And we said… we decided, we thought it was a good idea to keep, chunks that were, like, question and answer.
David Bau: So that's sort of the idea. That's gonna be, like, our unit of observation. It's gonna be the number of the charts. Can you keep going?
David Bau: So what does this look like? For example, we have, an analyst, Denise McAllefine, who's asking.
David Bau: he was asking the CEO of Netflix.
David Bau: You are wanting to put Walmart out of its misery by taking over its operation. Would you consider doing the same for Blockbuster? This is an old transcript.
David Bau: And this is a very long answer that doesn't really say anything, but it doesn't contain much uncertainty. It doesn't say… it's just Reed Hastings just, like, saying nothing for a lot of words. Yeah, it's a lot of words, but it doesn't really say anything. It's just saying that maybe not. But there is… so some of them are just, like, these kind of questions, and it doesn't really say a lot about their uncertainty with respect to the future of the company.
David Bau: Next. Sorry. Sorry, Claire.
David Bau: But now, for example, this one is a software company, and they're asking them, about the quarter,
David Bau: They're just asking for the revenue in the next quarter, and he does give a little bit of, like, a… oh, that's much better, thanks. Yeah, he'll give a…
David Bau: description here says, it depends on the amount of new contracts we get with closing any given quarter, so this is a little bit more, like, an uncertain answer, we're not really sure. And then if we keep going, this is a long one, sorry. Very challenging. This is the CarMax spread CEO, when they were asking him for closing statements, and he was just saying.
David Bau: The retail environment, there's a lot of uncertainty out there, the credit markets, the gas prices.
David Bau: And he's saying we're in such a slow environment right now that there is so much uncertainty in the marketplace, it's very difficult for us to figure out what's going to happen in the rest of the year. So this is an example with very high uncertainty.
David Bau: And now, did you say that this was already pre-labeled manually? So this is our issue.
David Bau: We have all data, we don't have labels.
David Bau: So I did those labels. How many did you label by hand? Not that many, 16. 16, that's a lot, that's good. But the issue is that some of them are really easy. Can I get the LLM to generalize the labels for you?
David Bau: So, we did run an example on, like, 500 chunks. So, when we split by chunks of Q&As, we have 2 million, 3 million observations.
David Bau: They look like this. I think there is some issues here in this traction that we're gonna fix in the next…
David Bau: Couple days.
David Bau: Do you want to explain it? Yeah, yeah, explain what you do? Yeah. I'll finish the title.
David Bau: Oh, yeah, well, we are… we don't know how to do the, the labels, basically. That's our issue. Yeah, yeah, yeah, so I, so, so, so, I mean, so, like, so…
David Bau: you have 60 examples, and you tried… you tried ICL prompting of a large model to see if, like, you gave 60 examples, and then you gave it 10 more, and you say, you know what, I need you to label the spans, and I need you to say
David Bau: whether it's uncertain or not, here's 60 true examples. Here's 10 more that are unlabeled. Can you add the labels to this one? Have you tried that?
David Bau: We did the labels here, but we didn't give the example. We didn't do it.
David Bau: But this… yeah, we didn't do it like that, but we could try that. Yeah, I think that's… that's a typical way that people are doing it now, right? They say, oh, here's some really high-quality ones, now that you've seen these examples, and
David Bau: And they would also, like, write a long set of definitions above, like the kind of thing you probably talked about here. They'll say, this is what I'm looking for. For example, here's 10 examples of it being done well.
David Bau: And then here's, here's, like, 10 things that I would like you to label. And then you could try to double-check it in a few ways.
David Bau: you could try labeling the same thing twice with two different prompts, or two different sets of examples, see if it comes out the same. If it's very different, then you might not trust that one. But if it's pretty consistent, then you might trust it. You can ask a second LLM to do it. You could ask a second LLM to check your work.
David Bau: You know, here's a mix of good ones and bad ones, can you tell me which ones are inaccurate? Can I pick up the bad ones? You know, that type of thing. So you can come up with, like, ways of doing it and double-checking it to keep the quality high.
David Bau: Anyway, so… but I like… it's a cool problem, it's like a cool labeling problem, like…
David Bau: Perez would love this problem, right? You know, Ethan Perez, who wrote the paper about… Yeah, all this stuff, right? Yeah, great, yes, right, so… but, you know, he's written a bunch of papers on how to use LLMs to sort of amplify data set labeling like this, and
David Bau: And so, I think it's, you know, these kind of methods Seem like it's…
David Bau: It's the kind of thing you want to try.
David Bau: So I wouldn't give up.
David Bau: I think that the problem that you have is a great problem, and it's, like, an interesting research contribution to figure out how to auto-label.
David Bau: the stuff. One thing we're thinking of is not SATA empowered, so…
David Bau: Oh, yeah, you can go… I don't know if we have it, maybe. So there's a lot of… well, this is the labeling that we got from prompting the definition into Llama?
David Bau: But it is the case that there is very few cases of high uncertainty, there's a lot of known certainty. Nobody… none of the CEOs want to admit high uncertainty, yeah. Yeah, exactly. And they mostly show up in the Q&A, which is why we're only doing the Q&A, not the opening statements.
David Bau: But that's okay, I mean, you have… you have data set and balance it. Like, if you need to… if you need to balance it later, obviously, you can sample.
David Bau: you know, more equally. But it might be, like, do you believe, like, maybe the real data is like this, right?
David Bau: So this is the result. Yeah, so when I did it manually, I also got kind of, like, the same thing. I got a lot of, known certainty, I got a lot of… some more… so this is very similar to what… can you go next?
David Bau: I did, like, manually, I did, like, this.
David Bau: And I got mostly uncertainty, some intermediate, and very few high, so that's sort of how the data is, like, what it looked like. I think that's okay.
David Bau: Yeah.
David Bau: You know, it can be more challenging to train a model with data set imbalance if you're trying to do a classifier.
David Bau: You know, you can ask all your machine learning friends.
David Bau: for techniques for how to deal with that, right?
David Bau: But but I wouldn't worry about it. I think that's cool. But the real question is, you've got this vast amount of data.
David Bau: And you're trying to figure out how to use as much of it as you can. And so, see if you can auto-label more of it. It would be great.
David Bau: Eric, would you go back to the prompt slide?
David Bau: Yeah, so just a little bit about,
David Bau: experiment, so, like, we… using this dataset and the 61 samples on Vernica.
David Bau: labeled. We basically tested out, like, how, models But, like…
David Bau: formed this task of identifying, like, the levels of uncertainty using the same labels, like, those, the three-way labeling, no uncertainty, intermediate, and high.
David Bau: And this is, like, what we gave it, gave the prompt. So, basically, we gave it the definition, and I think one thing here is that we were thinking of how, like, because
David Bau: We kind of want to, like, one interesting research question is.
David Bau: How the model, classified, like.
David Bau: different, like, so, like, it distinguished between uncertainty versus the first moment, like, general sentiment of this earnings call. So, like, if this earnings call is, like, pessimistic, that doesn't mean
David Bau: Well, it's bad, but, like, that's just first movement. The uncertainty measure is, like, independent of that, so we sort of want to…
David Bau: know how models understand that distinction. So, we give it definition and explicitly, like, saying that do not treat positive, negative sentiment, or clear numerical… numerical guidance as uncertainty by itself. So, something like that.
David Bau: And then we just tested out, like, how the model is performing the task. Okay, I'm supposed to say thank you very much.
David Bau: Sorry, either… so, but you can put up, like, are there other things that you want feedback on?
David Bau: Can you go ahead? What are the key things you want feedback on?
David Bau: What kind of pets in? We just, yeah, we need mostly the labeling, and and this is another attempt at labeling, and then you tried this, and then how did it do?
David Bau: Keep going.
David Bau: What's this? This is a rock. This is just a disk.
David Bau: Okay, cool.
David Bau: This is just, this is not the labels, this is the, like, model's prediction on a model prediction, right? How good the model is at. So you, like, labeled 60, right? So you could hold out, like, you know, 30 of your labeled things, and you could ask the model to auto-label your holdouts to see if it agrees.
David Bau: your thing, right? So you can… right, so you can… you can kind of see how auto visualizations work. I'd be… I'd be kind of… you know, you're… you're at this thing where you've got this massive piece of data, and you're trying to figure out how to deal with it, and… and it's its own little mini research.
David Bau: then you can treat it as its own little problem. It's not… it's probably not enough… it might be enough for a whole paper just labeling this data, but it's not… it would be the point of this class. The point of the class is going to be to use that data for other things. Yeah. But but yes, it's… but it's… it's a serious problem. It's worth, you know, trying a few different things like this.
David Bau: And then measuring… measuring how you're doing.
David Bau: yeah, we'll try a few different things. Yeah. Any other suggestions for this team?
David Bau: Now, do you guys… okay, go ahead. Yeah, I don't… this is… this is more just kind of, like, an open question, maybe for everybody. Like, how did you decide
David Bau: 3 labels instead of just, like, yes, no on the one hand, or you could have 4 labels, you could have 10 labels, like, how did you end up with… Ideally, we wanted some sort of continuous measure, but we thought that would be too broad. So initially, when you did your experiment, we had asked it to give it, like, a score from 0 to 1.
David Bau: But,
David Bau: We were thinking of, so, like, this is on the chunk of question-answering pairs. So, like, an earnings cost, like, is consists of, like.
David Bau: many chunks. So, like, we could sort of, like, aggregate that using those, three-way classification into, like, a continuous. We just wanted to be very conservative, so we wanted to go as legal as legal label as possible, because we wanted it to be very clear what it was.
David Bau: You know, so when you make more categories, it becomes, like, what does it mean, higher and lower?
David Bau: Yeah, so that's definitely an issue that we're thinking about. You could also just… I think it'd be pretty easy to, like, go from this to yes, no. Yeah, but yeah, we thought that, too. And then just, like, try it and see if you get better performance, if you get different.
David Bau: But on the other hand, right, like, on the same spirit, there's almost nothing to be lost if you're auto-labeling to ask an LM
David Bau: To give you a rating from 0 to 100.
David Bau: And, and then, and then, you know, and then you might be like, okay, well, actually, it doesn't have very good resolution, so I'm gonna round it down to yes, no, like that. Like, there's almost nothing to be lost from just trying. The,
David Bau: you can… you can do things, like, you can say… you can… you can have the different LMs to see if they agree with it. Like, oh, you get a rating from 1 to 100, then you ask the LM a different way. Which one's more uncertain? You know, and then you give it two examples.
David Bau: And if it agrees, then it means it's got some consistency, or consistency with another LM, maybe you believe it more.
David Bau: But if it's very internally consistent, then maybe you believe it lasts. You can do different…
David Bau: different things. I've been surprised, you know, some research going on, like David Atkinson in the lab is, just asking for numbers from 1 to 100,
David Bau: on these things, and it's nice. It gives him the resolution to be able to check if the models
David Bau: Like, the models, you know, have big error bars on a scale of 1 to 100, but by getting these numbers, it allows them to fit models to it that
David Bau: you know, like, you can do linear regression instead of just yes-no assessments and things like that. And so, so it gives them a few more tools that you can use.
David Bau: So it's a reasonable question.
David Bau: Yeah, so… is this… is this interesting to people? Is there anything that, like, do you see, like, where this might go for a research paper in the end?
David Bau: Where is this going?
David Bau: Yeah.
David Bau: Are you focused on speaker uncertainty?
David Bau: Yes. Like, the first, like, interpreting.
David Bau: Yes, I think we are.
David Bau: Because I… but… Versus… Yeah.
David Bau: She's like… That, like, that person feels uncertain.
David Bau: Based on the way they're talking right now. You know, how, like, the situation at all is uncertain, or, like, just the way they word it is confusing, like, I'm not certain of what they're saying. I guess that came up in the first presentation, whether, this has something to do with how the model is uncertainty itself. Someone mentioned uncertainty, research that we have to review.
David Bau: But yeah, that's definitely, yeah, that we… that's so many questions.
David Bau: There's a different… have you considered… like, okay, I'll start. I'll let you make your suggestion. No, I was just gonna say, like, I think…
David Bau: it is interesting that, like, the context of the earnings calls, where there's sort of this reputation management thing, where it's like, you don't want to say, like, like, oh, we're definitely going to fail and go bankrupt, like, you know what I was like, you know, it's really uncertain about just all the changes, and, like, I think that they might not be uncertain at all. Yeah, but there's also… there's also an incentive to,
David Bau: provide some sort of honesty, usually, because, usually if you don't… if you lie to a shareholder, they… they don't like it. Yes, that makes sense. So there's also, like… but definitely they try to be very… that's why we have very little examples of high uncertainty. So this… right, exactly. So I was about to say the same thing as you, but in different ways, but in more stark terms. There's this question of honesty. Some management is more honest than others.
David Bau: And you have interesting ground truth because you're in the financial markets, so you can see when management is actually misrepresenting the situation by looking at the next quarter.
David Bau: And so, so I'm, I'm really, like, so…
David Bau: if you were… if you were a quantitative trader training your model, you would actually do this. You would… you'd be training your models
David Bau: On looking at not just measuring uncertainty, but seeing if you can tell from their verbal cues and so on, whether management is obscuring the truth.
David Bau: And something like that. So I'd be kind of, like, I wonder if that's the kind of thing that you'd want to try to label as well, or if you want to label things based on financial ground truth or anything like that.
David Bau: Anyway, yes, yes. Anyways, it's a thing that comes to mind when you show me, like, the actual text, and you tell the stories.
David Bau: That there might be, and maybe even a third dimension beyond uncertainty.
David Bau: That might be interesting to… Try to label.
David Bau: Yeah, we're open to ideas. Like, economic activities, how, sort of, that correlates with… Yes, or… Or management honesty, even.
David Bau: So, yeah, it's, like, super interesting.
David Bau: So… I think that a lot of people are actually deeply interested in this.
David Bau: So, what people are deeply interested in
David Bau: Will an LM lie to you?
David Bau: But people are also deeply interested in
David Bau: Doesn't LM know if you're lying to it?
David Bau: What would be the graduates for that? Well, you can see the financial statements. Yeah, so they can say, hey, you know, we booked a lot of sales, and then you look at the next quarter, and they haven't booked a lot of sales. They're lying.
David Bau: But you can… but you know the future. Yeah, because we have… Right, because you have the future. The old data, yes. Right.
David Bau: I think you probably should also be careful of, like, one, like, these are public companies, so maybe the LMs do know the outcomes.
David Bau: Perhaps, you know, or, like, know how… Oh, they memorized them, so you might have to anonymize it. And also, the, like, they know the history. Differentiate between…
David Bau: the LM telling how… seeing, like, seeing how certain the person is, or the LM's sort of also imposing its own determination over, like, what it's saying. They're like, are you here to double your sales? Like, I doubt it. Yeah, I know… I know this company went out of business. Right. Because the LM.
David Bau: Or even if they didn't know about the company at all, just, like, the… like, the cr… like, the… like, what they're… the content of what they're saying, like, the more…
David Bau: I don't know, like, boastful or, like, unreasonable, it sounds, like… I see, so that's a question. If you're looking… are you looking for linguistic cues or societal cues? Like, if you're only… if you want to make sure you're trying to understand linguistic processing.
David Bau: Do you need to make an anonymized version of this dataset, where you go through and you say, change all the company names to fictitious company names.
David Bau: And things like that, so that you can't count.
David Bau: what it's about, right? Like, does that… does that mean it can be meaningless? Because now, you can't read it, because the way that people talk about it, you would have to know this is a car company, but if you change the name of the company, you can't tell anymore. But it's kind of an interesting question.
David Bau: I like it. Oh, fascinating.
David Bau: Fascinating question.
David Bau: It might be something that you need to control for.
David Bau: Yeah, going back to the definition of uncertainty, I'm curious if you looked at, like, uncertainty taxonomies?
David Bau: Right, what does that mean? Like, there are different kinds of uncertainty, like, aleatoric or, like, inherent uncertainty. Like, it seems like a lot of…
David Bau: the statements that you show are, like, oh, this is… Have you heard of these? Yeah. Will you send them, will you send someone to the Discord? Yeah. Alright, I'm gonna go on to the next team, but that, I think, is a really great comment.
David Bau: Yes, lots of… Yes. Thank you, thank you.
David Bau: Who's X team? There's no team after S and Teams. There's no team after you? Team B. Team B, okay, Team B. We're the B team. Team Al, you renamed yourself, but you didn't change your folder.
David Bau: Okay.
David Bau: Are you the last name?
David Bau: Don't know. No, there was Team M also, and there's other Team M. Oh, are you Team M? Okay, okay. So, Team… Team… Team L, or B,
David Bau: I had to change our name again. Okay, yes, who knows? Team Pencil and…
David Bau: That was a very good name for a team, Team Excellent, I like that.
David Bau: embedded.
David Bau: Okay. Yes. So what we tested out in… since Tuesday was there was the idea of… It's been a long path. I know it's been so long ago, it's been so long ago. Is there sick of it? Like, how much of…
David Bau: syncophancy is actually… sycophancy, like, the assistant trying to manipulate the user, or is it just, like, next word prediction? And one thing we were thinking of testing is, like, okay, let's swap the roles so that I'm gonna have the assistant
David Bau: say, for Llama 70B instruct.
David Bau: hey, to the user, I'm curious to get your opinion on this political question, and for some reason, I'll just inject that, like, the assistant says, I'm liberal, or I'm conservative, and see just… and then when it predicts for the user's role.
David Bau: Does it predict that the user
David Bau: will match the elements. Oh, see if the user to be symptomatic. That's interesting. This is… I don't know.
David Bau: That's a funny experiment. Yeah. Sure, why not? Because these models were trained to fine-tune on the user parts of these exchanges as well. And what we see is that the user is super sycophantic as well, so, if the assistant says, you know… You can't call it sycophantic, you have to stay persuadable, right? Because that would be the…
David Bau: Like, models think that persuasion we were discussing about… Models think that the users are super persuadable. Persuadable.
David Bau: So, yeah, so we see the same kind of pattern, where if the assistant says, I strongly disagree with this question, then
David Bau: Then the user… it predicts that the user is mad, like, they say that it also is strongly disagreeable times.
David Bau: Yeah, it's actually…
David Bau: I should have put up the comparison, but it looks as though it's even kind of more persuadable than the system. And we also see this along the liberal and conservative campuses, where, yeah, CS column is, like, I'm conservative.
David Bau: So, this might just be super out of distribution and long, and it's just a… Oh, no, it's really interesting, that's cool. It's a good experiment.
David Bau: So I did a couple of media experiments, picking Yuen from a Chinese as a Chinese…
David Bau: mobile, and Gemma is an American model, to compare the
David Bau: To see the provenance effect, origin effect.
David Bau: On, and asking, are economic and social conservatism or liberalism separable in different…
David Bau: In LLMs from different origins, and thus this very bio…
David Bau: I picked Gemma, at two prompts, abortion for the social, dimension of ideology, and a taxation, to see whether economic ideology,
David Bau: exist, and… How sincere this may look.
David Bau: Coastal.
David Bau: And this is… entry abortion graph. It's kind of a mess, but…
David Bau: You see there is abortion.
David Bau: Domain knowledge in the middle.
David Bau: and abortion legal texts, regulation, etc, etc, but it triggers social issues, controversial issues, and LGBTQ. So, interestingly, ISIS and Adolf Hitler is…
David Bau: I don't know, I didn't know what his stance was on this issue. Interestingly, it triggers this kind of…
David Bau: very extremist, both religious extremists and kind of nationalist extremist.
David Bau: Kind of, features.
David Bau: And I can't wait to understand this is a kind of American topic?
David Bau: Then…
David Bau: feminism issues. These are all social.
David Bau: kind of…
David Bau: Dimensions, but there is no economic dimension here, and this means economic and social dimensions are separable to each other.
David Bau: And there's pro-life, bundle here.
David Bau: This means Coventry understands, kind of.
David Bau: different aspects of, kind of, ideology, whether it is conservative or liberal, there are some responsible features. What technology did you use to make this graph? The
David Bau: Thanks, I just started.
David Bau: All visual cycles… And what algorithm is in Neuron Pedia? Which one did they… what was this? It was the,
David Bau: There are many other country, the… that I used…
David Bau: This is the transcoder circuits? This is the transcoder, so this is not Michael Hannah's circuit? Yeah, that's… I was… This is Michael Hannah's thing. Okay, okay, I had only one option, so…
David Bau: Okay, yeah, yeah, yeah, we should… Yes.
David Bau: I'm not familiar… I've never used it. I've heard of it, I've talked a lot about it with people, but I've never used it myself, so that's why I'm asking. So this is,
David Bau: country steering experiment.
David Bau: Because in the previous,
David Bau: In the graph, there is, I said, pro-life, and it also understand abortion and guns are related to each other topic… related to topics, and bundle them together.
David Bau: And I kind of…
David Bau: experimented on a couple of different features, and I, I was suspicious on it, I was suspicious on it, but…
David Bau: Interestingly, abortion and… or guns feature, is kind of responsible for stance.
David Bau: But domain, features, are not kind of responsible for stance, as a stance identifier.
David Bau: And it changed the leftist model
David Bau: basically a conservative model. I say this is…
David Bau: Just one, just one feature change. Just here, that one feature. Yeah, just one feature change that I do.
David Bau: Interesting. And this is the kind of…
David Bau: I need to dig into that feature, maybe it triggers… Additional features, but…
David Bau: It basically, applies to all social, kind of, conservatism or liberalism, because it bundles abortion and guns.
David Bau: I have some suspicions on that. Maybe if I ask about LGBTQ question, this feature may turn…
David Bau: Remember, all the way I can.
David Bau: Also, vaginal.
David Bau: And when we look at the abortion graph, this is kind of a more structured, but still, they have social issues, they go with tea rights.
David Bau: Different from Queen, we have sensitive issues, features. Understand this is a sensitive issue. In the Queen model, there was… I didn't see any kind of features titled as sensitive.
David Bau: But China labeled this as sensitive issues. It triggers immigration compared to gun.
David Bau: And still, we have a GBGQ. This is the…
David Bau: kind of a shared feature, but immigration, instead of the gun, JAMA2 kind of triggered the immigration topics.
David Bau: And morality. In Kuwen model, we don't see… I didn't see any kind of religious and moral kind of,
David Bau: features, but Gemma, too, apparently have some moral kind of features.
David Bau: To understand, kind of, ideology and
David Bau: And LGBTQ is the responsible for… Stents.
David Bau: as it can be labeled as a stance identifier compared to all others. I experimented on a couple of things. Some features turn the model into neutral, but not kind of strongly. LGBTQ, just one feature.
David Bau: little, kind of, steering.
David Bau: Minus 1, turn the model into conservative model.
David Bau: And this also tells us that social dimensions are kind of correlated to each other, and then, there is one kind of responsible… there can be one responsible kind of feature, turning all answers…
David Bau: Chrome, kind of, liberal to conservative.
David Bau: So when it comes to taxation, this is an economic topic.
David Bau: Kuvan is also, still kind of scattered around, because it triggers…
David Bau: Medical disease terms and legal terms, regulations, tariffs.
David Bau: And interestingly, to understand this is kind of forward relations. Oh, sorry. All right, do you have anything else you want to comment on?
David Bau: You know, I can quickly… just show one more thing.
David Bau: Power is responsible for a stance in the coup, and when it comes to JAMA, JAMA tons of triggers tons of, things, and political theory and ideology is kind of responsible for… this is a general kind of feature. It turns to mobile, ideology, and…
David Bau: CON is more supportable than Gemma, and CON3 has explicit bundling, but Gemma 2 has implicit bundling between issues. LGB2 is common, and model permanence
David Bau: shapes, not just what models believe, but how they recognize. Domain markers are not identifiers, that's identifiers. Cool. Nice. So, any suggestions from people? What if people are interested in what's going on there? Any suggestions?
David Bau: Our team focuses on discrefancy, and then kind of domain knowledge experiments, and then tried to understand
David Bau: The model provenance effects at the same time.
David Bau: Want to hear my suggestions?
David Bau: I always have opinions. Why do I have so many opinions? You guys, I'm sure you guys have opinions, too.
David Bau: Any opinions?
David Bau: Suggestion.
David Bau: I guess just, trying to get back up to speed, so…
David Bau: I guess you're studying sequence of models with respect to, like, Perpetually divisive tissues? Is that…
David Bau: Yeah. But then the political issues.
David Bau: Yeah, the framing that we went in was, like, what… how can we understand the political leaning of an LLM?
David Bau: But then, we were realizing that that's sort of entangled with, like, is that even a coherent?
David Bau: But before we can dig into fully understanding, like.
David Bau: the multi-dimensional sort of ideology that it might have. We have to see, like, it's in response to something else, like, it's in response to the user, it's in response to the context, and so that's why we're focusing on significantly as, like, the starting spot.
David Bau: Simultaneously looking at. Right.
David Bau: So I… so I… I think… so, like, I… I… so my… my worry… about the topic.
David Bau: is that…
David Bau: Oh, what was the joke that you made when you said, like, it was last week, or a couple weeks ago, when you said, oh, you know, I'm, something something liberal, whatever, and then you said, oh, it should say liberal, but it says, oh, I'm,
David Bau: I'm… I'm a stereotype. 100% bad, right?
David Bau: So my, my, my main fear about your work is that a lot of this stuff is so disgusted and so much in the news that, you know, you, like.
David Bau: like, there's… like, it's like the snow tracks that we have outside, right? You know, I worry that you're gonna fall into people's common thinking, and you're gonna be like, oh, you know, there's succancy, and…
David Bau: And certain stereotypes line up, and the model is, like, reinforced the stereotypes, it's just the same message, and we've heard this paper a lot. It's like another one. And then you'll be like, oh, we'll find the mechanisms of stereotypes, but, like, okay, so…
David Bau: So how do you avoid this, right? And so, so one… one thing that'd be… that could spice it up would be interesting.
David Bau: Might be… To try to, develop Datasets are ways of probing.
David Bau: For behavior that people don't think
David Bau: Is related to politics at all.
David Bau: Does that make sense?
David Bau: Like, I don't know. Like, like the answer to, like, what seemingly objective questions, or maybe judgment questions that don't seem to have anything to do with politics, like…
David Bau: You know, what kind of shoes should I buy, or something like that? Who knows, right?
David Bau: But, like, things that totally… Like…
David Bau: Shouldn't be related to your stance on these, you know, controversial ethical issues.
David Bau: But it'd be interesting to see if there are, like, strong
David Bau: And it's easy for models to predict the same thing, and then you could go in and say, why is that? Why is it that the models predict that, yes. Seeing that, like, is sick and see a general thing that's, like, not just…
David Bau: specific to… Well, so you had this opposite thing, right, which was, hey, I wonder if there's, like, hidden cues for succinct. Like, if I say that I wear a certain kind of shoe, and I like to read a certain kind of book.
David Bau: Is that… is that gonna tip the model off? Like, will it know, you know, that I'm a closet liberal?
David Bau: Right?
David Bau: And I'm sort of asking, well, how far can you push at? And can you push it in the opposite direction?
David Bau: But, like, you know, how much… how much of the spectrum of behavior
David Bau: is determined by these very, very strong latencies that you say, oh, you find this one latent thing, completely flip.
David Bau: the behavior of the model. What flips? You know, is it, you know, is it, besides these political beliefs, is it a whole pattern of behavior, including things that people wouldn't expect right now?
David Bau: And, and the different LLMs agree, and is there a mechanism for why that is?
David Bau: What is the… the concern is by focusing just on… So, I think the most interesting thing
David Bau: the, for, from my point of view, is… your idea… Presenting…
David Bau: Signals that are hidden, that are, like, less obvious.
David Bau: You know, when you say, hey, oh, I'm from Atlanta, or I like to eat this kind of food, or something like this, right? And say, oh, and the model, like, funholes you.
David Bau: It's I-N-W-U-Y. Beautiful, great. I think that's interesting.
David Bau: And so I'm… my suggestion, as you're developing your data sets and so on, is, like, how far…
David Bau: Can you push that? Right, because, like, we're starting with demographic categories. You're starting with these demographic categories, and instead of, like, just continuing to circle around the demographic categories, can you amplify your data sets to, like, look at more of these unexpected things? Can you get all the way to…
David Bau: Like, I don't know.
David Bau: like…
David Bau: you're a mathematician. Do you believe in the axiom of choice? Right? This is not a political decision, right? But it's also not…
David Bau: One way or the other.
David Bau: What if you're conservative? Do you believe in the axiom Choice? What if you're liberal? Do you believe in it? I don't know. Maybe it changes it. Yeah. Right?
David Bau: I think the hard thing is, like, I think we're focusing on dem… like, we started with the demographics because there's ground truth data about, like, the correlations between these things, and so, like, I would worry, you know, maybe there's some, you know, these…
David Bau: these elements have gobbled up so much data, like, maybe they know some pattern that I don't know about, and, like, it's really, like, accurately reporting what it exactly. Yeah, so you can ground yourself in things where there's no data. But we could be, like, very obscure. You could, like, go and try to generalize that, and then you could also say.
David Bau: it's… you could say, oh, it has this complete misconception. Obviously, people are not like this. But, you know, we see the mechanism for why the model has this misconception, or something like that.
David Bau: Yeah, one thing I was also thinking about is, like, is there a hierarchy of, like, preferences where it's like, you know, say you describe, you know, you know, I support gun rights, I'm white, I'm high school educated, and I'm a flamin' liberal, you know, like, like, like, there's…
David Bau: Contradiction here. Like, when does it, choose to go with person self-reported versus be like, you can't possibly be, you know? But I agree that, like, pushing it in a weirder direction makes sense. Yeah, yeah, yeah. That's, that's my main…
David Bau: Does that make sense? I don't definitely make sense.
David Bau: Anyway, so that's my opinion. Okay. Oh, okay, we gotta keep on going. We gotta keep on going. What's the next team? E-Mam.
David Bau: Okay.
David Bau: Go, go, go, great, go! Okay, so,
David Bau: This, week, we actually focus more on engineering our benchmarks, such that, like, we have a good collection of data set, and a good collection of the inferences as well. So, mainly we engineered our, way through,
David Bau: Building that, codebase.
David Bau: And we focused on two things, egocentric and,
David Bau: allocentric, directions. So egocentric means, if I'm the agent, am I moving, up, down, left or right, or front to back, left or right? Something like that.
David Bau: But allocentric means, if I'm in this, location, and if I want to go towards the door, which door am I talking about, and should I, move towards
David Bau: or should I move towards the clock? Something like that. So, it means that now the model needs to have that special representation within it, such that it knows that clock is over there, 1 is here, and
David Bau: Why is this challenging? As a language model, you are not given an image, you are given a string of some characters.
David Bau: Like this one. So… You need to interpret, what kind of,
David Bau: map representation do you have internally, such that you can navigate through the maze while also avoiding all these obstacles on the way. So, we don't have a lot of results this time, but we are working on the inference and how to speed that up.
David Bau: So… what we… Part of doing is, you can, check out,
David Bau: Will you make the screen big because it's really small fonts? Just press, press present.
David Bau: What is… there's also… is there a photo with the document? I see a document. Okay, cool.
David Bau: I think they should be fine. Yeah. So…
David Bau: In… as a data, we are collecting, all these attributes, like what type of, grid is it? Grid work basically is, 2D grid, and, we are also expanding into, some 3D views and, some other… we are also thinking of other geometries, but these are the baselines that we are working on.
David Bau: Size, we can have, either 5 cross 5, 7 cross 7, or 5 cross 7 as well, which is not shown on the top image, but in the bottom one, you can see the…
David Bau: Grid size is basically a string.
David Bau: So yes, we are storing, all these data. This is, basically, we call it, the ground truth labels, rather than, generations. So…
David Bau: We use basic computer science algorithms like BFS and EFS to search for the shortest path, and this is the storage of that ground truth.
David Bau: In the inference, we collect the same things.
David Bau: That we did for,
David Bau: The data collection in terms of ground truth, and, we do the same thing for,
David Bau: our entrance as well. But this time we collect a few more details, like, what temperature did we set, in that particular, dataset, and,
David Bau: What were the probabilities, what was the probability of the next token generation? Things like that, so that, we can benchmark it with.
David Bau: Hey, let's introducing the problem.
David Bau: the questions we asked before. Oh, yeah.
David Bau: So, the questions that we're asking are, like, how far are you from the statue? So, for example, if,
David Bau: The agenda's here, if you can look at it, the agenda's here, and statue is here, how far are you from the statue?
David Bau: And how many steps do you need to perform in total to move to the goal, or something like that? And what do you find? Are the models really good at it, or really bad at it? Proprietary models are good at it. Lama, the smaller one, is not. The Lama bigger one is good, without reasoning.
David Bau: How's that? Oh, how good.
David Bau: They are giving connect results, we have still not, collected the data, so right now, as I speak… Have you guys ever played NetHack?
David Bau: Sorry? Have you ever played NetHack? Have you ever heard of this… this game?
David Bau: the desk.
David Bau: Anybody here ever played NetEk?
David Bau: Oh, I'm so old.
David Bau: So… Back in the day.
David Bau: when… Teletypes… you know what a teletype is?
David Bau: Back in the day, when, you… everybody had mainframe computers, and they didn't have, like, PCs.
David Bau: It used to be the way you used a computer is you would hook up a typewriter.
David Bau: To, like, a serial… Or… that went down to their basement.
David Bau: So, connect to the big computer, it would run your typewriter.
David Bau: And then the computer would listen to your type… things that you typed, and then it would type things to answer you.
David Bau: You could call up a report, and we'd print it out.
David Bau: You get it on paper? That was a… that's called a teletype.
David Bau: And then people switched to video terminals. Anybody heard of a VT100?
David Bau: Oh my gosh, do you know.
David Bau: Okay, so they… because all the computer codes that, like, when you have a terminal emulator, it's a VC100 emulator.
David Bau: on this topic.
David Bau: Anyway, so these terminals came up, these video terminals, it's very high-tech, and then the first thing that happened is people were like, oh, you can make video games!
David Bau: And then one of the first video games to show up was this game called Mac.
David Bau: And then there's a new version of it called NetHack that came out a couple years later, and it's still available, but for today, you can download NetHack to your Mac if you play on it. And it's this little terminal game that looks just like
David Bau: the maps that you're making, like this. Right? You, like, move around a little… Great world.
David Bau: With your little dude. Your little dude is like a little at sign.
David Bau: You, like, move it up, and move it to the right, like, get some treasures, and fight some monsters.
David Bau: No. It's like Pac-Man. No, but because there's, like, magic potions you can drink, and then there's sports you can pick up, and it's a very rich game, much richer than Pac-Man.
David Bau: Right?
David Bau: And… And the thing is.
David Bau: There was a training environment, around NetHack. It's a very sophisticated game, and they trained a lot of the models
David Bau: on how to read an ACC board, it's like a Language Monster's task, the CASCI text.
David Bau: Right? And how to decide on what the next move is.
David Bau: How to win the game.
David Bau: And I would… Be surprised if none of the big companies included winning NetHack.
David Bau: They probably include winning chess in the training data, but then there's this nice NetHack training environment, they probably include winning net pack.
David Bau: In the reinforcement learning training environment, because it's kind of a similar
David Bau: Of a lot of human education.
David Bau: And the navigation container and things like that. So when you say strong models, like, big models are, like, really good at this, probably because they were trained on it. They were probably trained on that hack.
David Bau: Right?
David Bau: And so, so, and then when I look at… so you should… Look up NetHack.
David Bau: A NetHack board is a lot more challenging
David Bau: than the boards that you were trying, and you could see if any of the models could actually tell you distance in a NetHack board, or navigate a NetHack board. They're, like, they're a lot trickier than this. Does that make sense? Yeah, yeah.
David Bau: Anyway, yeah. And also, long story, sorry.
David Bau: Yeah, also add on to Kimmy's, structure.
David Bau: All right. Any other thing you want to suggest? Yeah, just one more thing to add on. So, we have to very carefully to design the prompt to the model, so the model can, can, can describe how far it is, and how far, how close they are to some, like, landmarks.
David Bau: So before that, we didn't, put this, like, coordinates
David Bau: introduce this coordination system to the model, because we want to see, like, how model would like to… how they would choose to describe in this. But if we didn't include that, the model, like, didn't, like, answer quite well. They… they didn't know what they would… they would like to… how to describe in this concept of foreign quest. So we have to include this.
David Bau: And then the model can output some reasonable response to it.
David Bau: That's one of the alphablo is.
David Bau: So, given that more complicated settings?
David Bau: So here's a question for you. So I don't know if you're interested in this. So this is, I mean, this is setting off… okay, so you should also talk to Gabriel, because he just submitted for ICML a grid world.
David Bau: paper, right? Did I… do we mention that to you before? I think he mentioned it to you, that he's working on it. So, where… where he's probing into the models, doing… is it inter… like, all the… all my colleagues, it's all interpretability work, right? So it's, like, probing into models. So you should do… do… ask him about it for two reasons. One is.
David Bau: to get ideas, and the other is to make sure that your paper is less… you don't want to make a paper that's less interesting than his. You don't want to make a tiny subset of what he's doing.
David Bau: Which is the main challenge with what you've got here. And so I think that you've got an issue of, like, how do you make sure that you do something that hasn't been done 500 times before, right? Like, robotics people have been studying grid world for about 50 years, right? And so how do you make sure that you don't, like.
David Bau: write another Greek world paper, but it's, like, just watered down, and never gets to peer review. Does that make sense? I think that's the main…
David Bau: question is challenging for me, so I… so, yes, any suggestions? Yeah, this is a… this is just a good question. Like, when I look at the grid, like, I think it's correspondence to, like, the way I experience space feels intuitive, but, like.
David Bau: How do you know that it's intuitive in that sense for the model? Like, what's the correspondence between this and then, like, the, like, physical, real-world settings that you all motivated the project with at the beginning?
David Bau: Like, is this… yeah, how do you link this to how the model thinks about life physics work?
David Bau: We are using neuronP to, figure out, which neurons activate the boost, and,
David Bau: Which particular portion of hypertension or any other mechanism inside it.
David Bau: Is kind of responsible for,
David Bau: working through these problems the most. I think we did a few results in the last week's presentation as well, on, I don't think that the question is really trying to get an answer. I think the question is more of a question to ask if you want to do your research in this direction. Yeah, we're going in this direction.
David Bau: Right? Like, if you were to do this, but you put real city names on your grid. You said Berlin is over here, Paris is over here, you know.
David Bau: you know, Rome is over here, Moscow's over here, right? Would that make it easier for your mom? Because it's like, oh, now I know where things are, because I know, like, things in the real world. What if you put them in the wrong order? What if you mixed up
David Bau: like, real cities, then would it be in conflict with itself? Does it know these two things are the same thing? I think that that's kind of the spirit, right? Does that make sense? Yeah, yeah, yeah, yeah. Yeah, yeah, yeah, yeah, yeah, right? Like, so, like, does it… does a model think about this abstract
David Bau: Boy world, the same way it thinks about
David Bau: The real world, or is that totally unrelated?
David Bau: I think it's a very, you know, you know, like, that's, like, nobody's written that paper. Like, the Roblox people wouldn't touch that one. Does that make sense? Thank you.
David Bau: Yeah.
David Bau: How are we doing out of time? Are we out of time?
David Bau: Did everybody get some advice?
David Bau: Okay, I think everybody got some advice. Thank you very much.
David Bau: We talked to all the teams.
David Bau: And we'll see you on Tuesday.
David Bau: Alright, stay safe from the snow.
David Bau: I don't know. Netac.
David Bau: It's a fun game. You install it, and you might get addicted to it. It's pretty good. I haven't played it for like that.
David Bau: Many of the adjac.
David Bau: I mean, that happened.
David Bau: That's it, this one, this one. Oh, okay. Yeah, you see? All these funny little… Oh, wow, things. Alright, this is more fun.
David Bau: Yeah, yeah, yeah. So I'll see it. And then you can also look for the Netback Training Environment, or I don't know what they call it, but they have their own… No, it's all. It's all, like, open source one, like, oh, the open source. Oh, so Haber is your student, or… Yeah, he's a postdoc in my lab. Oh, okay, bro, sorry, so you can ask him about that. Thank you, thank you. He'll be there.
David Bau: Yeah, that's go back to Sugaroo.
David Bau: But I know I can do more.
David Bau: Praise.
David Bau: Did you get my sleep last night?
David Bau: I don't know what happened to Nikhil. Nikhil, like, he didn't even tell me exactly. No, but it's alright. Yeah, it's okay.
David Bau: It's fast enough, for sure.
David Bau: But,
David Bau: But I logged in in the morning and did sort of vinyl edits on them to make it work a bit like any other works for the pair.
David Bau: And, but no, let's see…