hmm, i'd thought of lemon markets ruining basic economic activities in modern life, and i'd also thought of urbanization being the root cause of social isolation, and i've even thought it was better socially when people had economic excuses to form communities, but i've never made the particular connection written about here (that functionally, this makes modern socializing a lemon market). thanks!

I often like to cite [https://www.youtube.com/watch?v=Njk2YAgNMnE](this music video) as an example of something that was made possible by AI, and used it as just a building block in a complex artistic process (for my part, I couldn't imagine how I would auto-generate a video like this, or even encode the movement of the camera as a constraint (without some substantial effort), and it was made in 2022!)

forgive my ignorance, but is there any reason that you can't have multi-layer sparse autoencoders, even those that are interpretably compatible with the linear representation hypothesis? like what would their drawbacks be (other than more required compute)?

no matter how it is that you're computing the latents, 0) you still have a reconstruction loss;

1. the final layer is still a decoder that computes a linear transformation to the thing you're reconstructing;
2. you can still have a sparsity penalty on the last set of activations.

it seems to me like this still constructs a set of latents that sparsely activate, and which are linearly represented in activation space

FYI, I heard that Oliver Sacks fabricated/embellished a lot of the anecdotal accounts in his books. This was a fairly public controversy, so evidence for it can be found on Google.

i would like to know where this question leads, since i in principle like children and animals and yet have no idea what to do with them

I think I see the logic. Were you thinking of making the model good at answering questions whose correct answer depend on the model itself, like "When asked a question of the form x, what proportion of the time would you tend to answer y?"

The previous remark about being a microscope into its dataset seemed benign to me, e.g, if the model were already good at answering questions like "What proportion of datapoints satisfying predicates X satisfy predicate Y?"

But perhaps you also argue that the latter induces some small amount of self-awareness -> situational awareness?

While the particulars of your argument seem to me to have some holes, I actually very much agree with your observation we don't know what the upper limit of properly orchestrated Claude instances are, and that targeted engineering of Claude-compatible cognitive tools could vastly increase its capabilities.

One idea I've been playing with for a really long time is that the Claudes aren't the actual agents, but instead just small nodes or subprocesses in a higher-functioning mind. If I loosely imagine a hierarchy of Claudes, each corresponding roughly to system-1 or subconscious deliberative processes, with the ability to write and read to files as a form of "long term memory/processing space" for the whole system, and I imagine that by some magical oracle process they coordinate/delegate as well as Claudes possibly can, subject to a vague notion of "how smart Claude itself is", I see no reason a system like this can't already be an AGI, and cannot in principle be engineered into existence using contemporary LLMs.

(However, I will say that this thing sounds pretty hard to actually engineer, i.e, it being "just an engineering problem" doesn't mean it would happen soon, but OTOH maybe it could if people would try the right approach hard enough. I can't imagine a clean way of applying optimization pressure to the Claudes in any such setup that isn't an extremely expensive and reward-sparse form of RL.)

(paraphrasing would be a markov kernel here, and with the transitivity property I mentioned earlier, I'm asking that achieves its stationary distribution in one iteration)

for this condition, if you also want symmetricity, this is a very strong condition; you'd only accept "lossless paraphrasings". i think not only are you achieving the stationary distribution in one iteration but the distribution cannot change, so this is either a markov kernel for every semantically different phrase, or not-markov.

There is some danger in this suggestion: it can improve the situational awareness of the LLM.

Why? 

i think compute and networking speeds are honestly enough that most people struggle to take advantage of more of those things (streaming video is about the most data-intensive thing a lot of people do, and what's above that is mostly actual computational tasks), so it would take (significant) additional innovations in figuring out how to convert these things into better experiences in order for this to be tenable. it seems a lot of the time that the line is usually drawn somewhere around gaming enthusiasts (e.g there is a cohort of people who will buy a more powerful smartphone so it can render graphics better so they can game on their phones more enjoyably, same for the display). this could be because economic incentives towards innovations in compute still favor commoditizable things, since compute is more generally useful (for the amount of work you could employ to make phones better for a small contingent of people who would buy them, you could just make some similarly advanced/complex system better for some industrial/trad-tech purpose and make way more money)