Posts
Wiki Contributions
Comments
How do you feel about Bayeslord's description of Jhana meditation being a positive form of prediction error, creating a sort of feedback loop of bliss?
Now this is effective altruism.
1I've seen some convincing arguments that water is not wet.
This isn't related to the post directly, but do you think that public transportation being free would be a good or bad decision for any reasonably large city (Chicago, Boston, New York, etc)?
Good meaning 'good for people, good for the city's local economy generally (via other benefits besides income from fares)'
This is a weird and stupid question, but did you used to be an admin on Hellmoo?
It's really interesting to hear that people go this far in this regard. I had thought maybe I was overthinking it, but it seems like some people like yourself find a lot of value in cataloguing these things beyond just bookmarking them on the site or vaguely remembering the concepts and searching when they need them.
This is really interesting and useful.
Particularly, the two things you linked are just interesting on their own, but also although I don't think my brain works in the same way yours does, I appreciate your perspective and how you tend to work with regards to these things. I think that I need something like a reference or a bookmark because these concepts don't stick quite as strongly in my mind without lots of repeated exposure. I tend to be a 'ground-up' learner (if that's even a thing) as opposed to someone who can keep lots of disparate concepts separately in my mind. Jargon and acroynms seem to fall out of my head like a sieve. I've confused the terms 'anosmia' and 'aphasia' for years. I just had to look up 'word for not being able to remember words' in order to remember the word aphasia. Ironic, right? Shiri's Scissor/sort by controversial is an article I already read once in the past, but completely forgot until you linked it, I clicked it, and I read four paragraphs of it.
I think you might be right. For example, any of the logo changes I described is going to necessarily be related to making the company more attractive to investors by seeming more 'modern', and a lot of these changes are probably not simply decided upon by the designers themselves, but are also incentivized and meddled with by higher-ups who want things to look more like another, more popular and profitable app.
I assume you live in the US or Canada. The fact that you feel the need to give the 9-year-old a kid license (the tile is smart!) I think points to societal issues to do with norms and structure that lead to the sort of effects described in the OP.
US and Canadian cities (and much of Europe and the developing world that designed their cities by the West's example) are generally not designed in a way that is friendly towards kids exploring and existing in the world safely.
I don't mean 'safely' as in 'they might fall down and scrape their knee or get lost', I mean 'safely' as in 'they might get struck by a driver going 40mph while staring at their phone as they barrel down a stroad' or 'they need to walk 3 miles to get to the nearest convenience store or park'.
It's easy to find a number of examples of parents being disciplined or even arrested for allowing their children to walk to school, the store, or the park. To allow a child outside without guidance is considered gravely irresponsible by western society at large in a way that really isn't healthy or helpful for promoting independence, in my opinion.
https://reason.com/2023/01/30/dunkin-donuts-parents-arrested-kids-cops-freedom/
https://www.cnn.com/2014/07/31/living/florida-mom-arrested-son-park/index.html
In Japan there's a cultural rite of passage (usually in smaller towns, it seems) where children sometimes as young as 3 or 4 are sent on an errand, usually to go to the store and pick up a few things, or visit a family friend and retrieve something. There's a Netflix series documenting a slightly more staged version of this, called 'Old Enough!'. It's very cute.
Here's another potentially interesting article regarding this, from NPR, about playground safety:
I hope one day we can organize our society in a way in which kids can experience safe amounts of risk and develop into capable human beings. Thanks for doing your part.
The following is text from Claude Opus 3. I generally find people just dumping answers from LLMs to be kind of cringe, but in this case, as I was using it to try to understand the post and your comments better, I came across some really genuinely insightful-feeling stuff and felt as though Claude's input might be helpful to you in some way, and that it would be remiss not to share it. I'm sorry if it's nonsensical, I'm not informed enough on the topic to know.
"Regarding the relationship between belief states and "features", I think there are a few important connections:
Belief states can be thought of as the "true features" that optimally compress the information needed for prediction. In the simplex visualizations, each point corresponds to a unique belief state - a particular probabilistic weighting of the possible hidden states of the generator at that point in the sequence. These belief states are the most parsimonious representation of the relevant information for predicting the future.
The "features" often discussed in mech interp (e.g. as visualized in attribution maps or extracted by probing classifiers) can be seen as approximations or projections of these true belief states. When we probe an LM's activations and find that certain neurons or directions correspond to interpretable features like "noun-ness" or "sentiment", we're effectively discovering linear projections that approximate some aspect of the true belief state.
The topology and geometry of the belief state simplex can give us intuitions about the structure of these feature spaces. For example, the fact that belief states are constrained to a simplex (rather than an unbounded space) suggests that there may be inherent trade-offs or dependencies between different features. The trajectories of belief states through the simplex as a sequence evolves could also inform us about the dynamics of how features interact and compose.
So in a sense, belief states provide a principled foundation for thinking about what the "ideal features" for a given predictive task would be, and mech interp techniques can be seen as ways to approximate or probe these ideal features in real models.
Regarding toy models and information-theoretic aspects, some elements that could bridge between formal systems and natural language are:
Temporal dependencies and long-range correlations: Natural language exhibits complex temporal structure at multiple scales (e.g. within sentences, across paragraphs, across documents). Toy sequence models with tunable temporal correlations could help us understand how belief states track and compress these dependencies.
Compositionality and recursion: Language exhibits rich compositional structure, where smaller units (morphemes, words, phrases) combine in systematic ways to produce complex meanings. Toy generative grammars or rule systems could help us study how belief states navigate these combinatorial spaces.
Stochasticity and entropy: Language is inherently stochastic and exhibits varying levels of predictability (e.g. some words are much more predictable from context than others). Toy models with tunable entropy could help us understand how belief states adapt to different levels of predictability and how this impacts the geometry of the feature space.
Hierarchical structure: Language operates at multiple levels of abstraction (e.g. phonology, morphology, syntax, semantics, pragmatics). Toy models with explicit hierarchical structure could illuminate how belief states at different levels interact and compose.
The key idea would be to start with minimally complex toy systems that capture some core information-theoretic property of language, fully characterize the optimal belief states in that system, and then test whether the key signatures (e.g. the topology of the belief state space, the trajectory of dynamics) can be found in real language models trained on natural data.
This could provide a roadmap for building up more and more realistic models while maintaining a principled understanding of the underlying information-theoretic structures. The goal would be to eventually bridge the gap between our understanding of toy systems and the much more complex but often qualitatively similar structures found in real language models.
Of course, this is a highly ambitious research program and there are many challenges to contend with. But I believe this kind of cross-pollination between formal methods like Computational Mechanics and more empirical mech interp work could be very fruitful."