00:39:19	Chris Wendler:	are observables just statistics that help you write down conditions for distinguishing the two different minima?
00:40:13	Chris Wendler:	cool ty!
00:40:41	Chris Wendler:	it’s an interesting concept
00:43:34	Chris Wendler:	labelling this x-axis as sample complexity if it really refers to to the number of samples used is confusing
00:45:11	Chris Wendler:	so observables are kind of like probes but on the model weights. sth that’s very hard to set up for big NNs afaik (unless they are finetunes of the same model then it’s very easy — eliahu’s work).
00:46:30	Chris Wendler:	i guess maybe looking at attention only makes things easier but the complexity comes from permutation-symmetry in MLPs
00:50:32	Chris Wendler:	i love these plots
00:50:38	Chris Wendler:	toy models are great
00:51:26	Chris Wendler:	so this is “easier” without softmax
00:52:00	Chris Wendler:	the blue area is bigger in the bottom row
00:52:29	Chris Wendler:	i added all cols together
00:52:52	Chris Wendler:	yes
01:01:41	Chris Wendler:	the relation-based counting i did not fully grok but looks interesting. is the idea to attach this counting direction everywhere (to the values) so then when you match the token A with a query, you will pull in 3x the counting direction, which can be shared across token identities?

in the inventory based approach: is (A at position 1) distinct from (A at position 3) for the MLP? or does the MLP figure out “we are pointing 2x into direction A”
01:01:54	Chris Wendler:	(david feel free to ignore if this breaks the flow)
01:11:37	Chris Wendler:	when you are in these toy tasks you can imagine creating an automated mechint pipeline that figures out what the implemented mechanisms are.

e.g. sticking probes everywhere
01:17:38	Chris Wendler:	i wonder whether it would be insightful to probe for all token identities and for the counts after the attention layer and before the MLP 

and the same after the MLP.
01:19:57	Chris Wendler:	@michael: you can multiply both observables with a constant and nothing changes so there are infinitely many