Open Thread Winter 2025/26

kave

LESSWRONG
LW

Open Thread Winter 2025/26 — LessWrong

New Comment

120 comments, sorted by

top scoring

Click to highlight new comments since: Today at 6:01 AM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]AugBro4mo200

Hello All,
New to LW and still reading through the intro material and getting a hang of the place. I am ashamed to admit I found this place through Reddit - ashamed because I despise Reddit and other social media.

I came here because I cannot find a place to engage in long form discussions about ideas contrary to my own. I dream of a free speech platform where only form is policed, not content. Allowing any idea to be voiced no matter how fringe as long as it adheres to agreed upon epistemic standards.

Anyways, I know LW probably is not that place but it is adjacent. It seems most people here want to discuss AI research but hoping to find some communities outside of that topic.

6Screwtape4mo

Hello and welcome! There's a few of us around who discuss things other than AI research, myself among them. I suggest looking at the filtering options for the front page; it's the gear next to Latest, Enriched, Recommended, and Bookmarks. I filter the AI tag pretty heavily down. If you want to lean into voicing fringe ideas around here, I'd suggest reading the LessWrong Political Prerequisites and maybe Basics of Rationalist Discourse. They're not universally agreed upon, but I think they do make for a decent pointer to the local standards.

4Viliam3mo

Are you familiar with Astral Codex Ten (also called ACX)? The people there are also mostly smart, the rules of discussion in Open Threads are more relaxed... which can be a good thing or a bad thing, depends.

[-]MissReaper2mo120

Hello, all!
Long-time lurker here. I'm a recent psychology undergraduate who cares about kelp forests and people. I'm currently exploring the viability of blue carbon sequestration and alt protein projects(which involve kelp). This is part of my broader investigation into climate change risks and adaptation strategies.

I'm trying to find the best ways to use my limited time and resources to choose an effective career path and test my fit for different self-employment options. I am currently testing my fit for coaching and coding. I've also restarted my exploration into philosophy, specifically Stoicism, with Marcus Aurelius' Meditations.

I'm also a yoga practitioner and teacher. I'm interested in learning how yogic philosophy and rationality align (I began thinking of this after living with someone who is hyper-rational but is passionately indifferent towards yoga).

I'm also a nut for life optimisation, but struggle with executing optimisation strategies.

Being a lurker is fairly low effort, but I wanted to begin interacting more with the cool people here. I'm still quite intimidated by the whole karma system for posting, but I think I'll find my way around it fairly quickly.

Any tips and guidance are much appreciated! I'm a lifelong learner and hyper-curious, so please throw any amount of information at me. Thank you for being part of such a unique community!

1ErioirE2mo

Any thoughts on the relative feasibility of engineering algae blooms (via things like nutrient seeding) for long-term carbon sequestration?

[-]Ninety-Three2mo111

Is there a way to make the list of posts shown on lesswrong.com use the advanced filters I have set up at lesswrong.com/allPosts? I hate hate hate all of Recent, Enriched and Recommended (give me chronological or give me death) but given that I already have a set of satisfactory filters set up, rendering them on the main page seems like a feature that should exist, if only I can find it.

[-]Screwtape4mo111

In case folks missed it, the Unofficial LessWrong Community Census is underway. I'd appreciate if you'd click through, perhaps take a survey, and help my quest for truth- specifically, truth about what the demographics of the website userbase looks like, what rationality skills people have, whether Zvi or Gwern would win in a fight, and many other questions! Possibly too many questions, but don't worry, there's a question about whether there's too many questions. Sadly there's not a question about whether there's too many questions about whether there's too many questions (yet, growth mindset) so those of you looking to maximize your recursion points will have to find other surveys.

If you're wondering what happens to the data, I use it for results posts like this one.

2Screwtape3mo

Heads up, I'm planning to close the census sometime tomorrow. You can take it here if you haven't yet!

[-]Alex_Altair3mo102

Hi all! I'm a long-time LWer, but I'm making a comment thread here so that my research fellows can introduce themselves under it!

For the past year or so I've been running the Dovetail research fellowship in agent foundations with @Alfred Harwood. We like to have our fellows make LW posts about what they worked on during the fellowship, and everyone needs a bit of karma to get started. Here's a place to do that!

[-]Santiago Cifuentes3mo120

Hi everyone!

I'm Santiago Cifuentes, and I've been a Dovetail Fellow since November 2025 working on Agentic Foundations. My current research project consists in extending previous results that aim to characterize which agents contain world models (such as https://arxiv.org/pdf/2506.01622). On a similar line, I would like to provide a more general definition of what a world model is!

I've been silently lurking LessWrong since 2023, and I came across the forum while looking for rationality content (and in particular I found The Sequences quite revealing). I am looking forward to contribute to the discussion!

7Margot3mo

Hi everyone! I am Margot Stakenborg, and I have worked with Dovetail in this winter fellowship cohort. I have a background in theoretical physics and philosophy of physics, and now making a switch into conceptual mechinterp, after having been interested in it and learning about it for some years. I have been working with Dovetail on formalising world models, I am writing up a sequence of posts on the philosophical and mathematical prerequisites for proper world models, and which tools from physics can help us understand and analyse different world models, and I will dive into the different definitions of "world model" that float around in mechinterp and AI safety literature. Things I will discuss are: * How is the concept "world model" used in different areas of ML literature * Concept representation in the brain: new frontiers from neuroscience * Tools from physics: renormalisation and coarse-graining * What are "natural features"? * When can networks find similar representations of the world as we do? * Can NNs discover new natural kinds? * Theoretical equivalence and intertheoretic reduction * Bayesian experimental design * And probably more.. I hope to build this out into a quite comprehensive and complete sequence. Do let me know if there are other questions or subjects you would be interested in to read about!

6GuilleMarSan3mo

Hello! I'm Guillermo, a fellow in the Winter25 cohort. I have a background in mathematics, computer science and particularly computational neuroscience. In my project I am looking at the Reward Hypothesis in decision theory and reinforcement learning theory and would like to write a digest of what are the main results that connect a preference order from order-preserving functions, to expected utility maximization and reward functions (with discount factors). I furthermore would like to formalize some of the key results in Lean. Overall I am interested in topics that connect rationality and decision theory all the way to practical aspects of machine learning and reinforcement learning, to try to bridge these topics for AI Safety. Nice to meet you all!

5Robert Adragna2mo

Hi everyone! My name is Robert Adragna, and I’ve been working with Dovetail this winter fellowship cohort on Agent Foundations. Specifically, I’ve been trying to better understand what background assumptions the Natural Abstractions Hypothesis (NAH) makes about the world, and whether they might be learned in existing LLM systems. Specific questions that I’m exploring include: 1. Is the Platonic Representation Hypothesis from deep learning evidence for the Natural Abstractions Hypothesis? 2. Is it possible to construct a dataset which represents the world in a completely unbiased way? 3. How can Natural Abstractions be both universal & observer/goal dependant? 4. What would it take to empirically test the NAH? I’ve been lurking on LessWrong since 2024, when I got interested in AI Safety, and am very excited to spend more time engaging with the community.

5Leo Cymbalista3mo

Hello everyone! I am Léo Cymbalista, one of the Dovetail fellows since November 2025. I’m a physics undergraduate in the process of switching to theoretical AI safety research. My current research project is writing an explainer for computational mechanics, which is almost finished. I hope to use the knowledge I acquired while researching for it to answer questions such as “given 2 coupled stochastic processes, when can we say that one is modeling the other?”, which could be useful for investigating the presence of world models in agents. I have only known about LW (and AI safety) for about a year, so I’m still not very familiar with it, but it seems very interesting so far!

5Vardhan3mo

Hi, I’m Vardhan, one of the Dovetail fellows this winter. Thanks Alex & Alfred for running this! Background: I study mathematics and computer science (probability, algorithms, game theory) and I’m interested in formal models of agents and multi-agent interaction. For the fellowship, I looked at the question: Which agents can be faithfully described by finite automata / finite transducers, and which structural properties make that more or less likely? In other words, when can an agent’s externally observable behavior be captured by a finite (possibly stochastic) automaton, and what observable signatures indicate that a finite-state model is impossible or misleading? I’ve written a brief report summarizing definitions, toy examples, and some light lemmas. I’m planning a longer post with formal definitions, more examples, and proofs. I’d really appreciate recommendations on literature I may have missed (especially anything linking automata/dynamical-systems perspectives to algorithmic information theory, ergodic theory, or learning theory). Comments, questions, and pointers very welcome!

[-]Oliviero3mo70

Hi! I'm very new to LW.

I found this website while searching up philosophy websites that are useful. I've been looking around LW for about a week now, just reading and learning peoples takes. There's a lot of it and it's great if you ask me.

I'm still learning the guidelines and the karma system, which had been a little intimidating, but I'm getting the hang of it now. I do recognise that LW is more professional than I originally thought, especially professional for my age, but it's not like I'm applying to work for Nasa or anything.

That's just me, though. I would greatly appreciate any tips for navigating, filtering content, etc.

5Viliam3mo

A good resource to get familiar with the basics of LW approach to life, universe, and everything is https://www.readthesequences.com/

[-]Sodium4mo7-2

I feel like the react buttons are cluttering up the UI and distracting. Maybe they should be e.g., restricted to users with 100+ karma and everyone gets only one react a day or something?

Like they are really annoying when reading articles like this one.

[-]habryka4mo140

Yeah, I agree with this. I think they are generally decent on comments, but some users really spam them on posts. It’s on my list to improve the UI for that.

4gilch4mo

Seems like a signal-to-noise problem. Some amount seems like a useful signal, but too much is too hard to digest. Privileges based on karma make some sense but restricting it based on time (1/day/user or something) seems pretty crude, so I don't like that idea. Not sure if this is a good idea either, but the number of reacts allowed per post could be based on the amount of karma that user generated on comments on that post. That way, a user who's doing too many reacts would be encouraged to just write a comment instead. That still doesn't seem like exactly the right incentive, but I'm also not sure how I want it to work. Maybe the ability to filter out reacts from a particular (prolific) user would suffice?

1Sodium4mo

Yeah I would like to mute some users site-wide so that I never see reacts from them & their comments are hidden by default....

1TristanTrim4mo

Do you have any thoughts on those UI improvement written down anywhere? I'll admit to being one of the users that really spams reactions on posts. I like them as a form of highlighting for review and as a form of backchannel communication. I would be much happier if people would use more reacts towards me. So I would be upset with UI modifications to restrict reacts, but fully support updates to make the UI around viewing reacts cleaner and more useful. I wrote a longer comment with some feature suggestions. If you have time it would be nice to hear your thoughts.

6Steven Byrnes4mo

Part of it is the “vulnerability” where any one user can create arbitrary amounts of reacts, which I agree is cluttering and distracting. Limiting reacts per day seems reasonable (I don’t know if 1 is the right number, but it might be, I don’t recall ever react-ing more than once a day myself). Another option (more labor-intensive) would be for mods to check the statistics and talk to outliers (like @TristanTrim) who use way way more reacts than average.

2TristanTrim4mo

[EDIT: I think issues stem from different people using reacts in different ways and having different assumptions about their use. I think I am probably using them in a less common way than other people, but I also find myself believing I am using them in a better way than other people. As such, I am trying to put in effort to communicate my POV. I would appreciate if anyone who disagrees with me would do so with a higher bandwidth signal than just pressing the "Agreement: Downvote" button. Perhaps by using some inline reacts on my comment?] Haha! Sorry if I'm bothering anyone! ☺♡ I really like reacts and am bothered in essentially the opposite direction as Sodium in that I think reacts are a very useful backchannel communication, and see it as a minor moral failing that most users do not use them more. I think it's great that many reacts are based on LW ideals for discourse. I don't know exactly how they are managed, but I think they could be even more valuable if there was some team that reviewed how people are currently using them and then improved and updated react descriptions and usage guides based on that. A descriptivist approach. I also think a prescriptive approach would also be good. People should be suggesting concepts for reacts that they think would be valuable for communication, and people should be figuring out how to promote proper use of reacts. I do agree that relevance may be an issue. I would like it if everyone would drop ~10 reacts while reading a post, but then, if all of them showed in the UI, it would be too noisy to make sense of easily. I think there are a few ways around this: * [EDIT: I've discovered on the triple dot menu it is already possible to select for inline reactions to hide all, hide downvoted, or show all. I think a plausible sane modification of this would be to make the default to hide under 2 or 3 votes and always show reactions to highlighted text. However I think some kind of more complicated scheme could be better

4Screwtape4mo

My two cents, I'm happy with the amount of reacts I usually see and would probably enjoy about 20% more. Thank you for chipping in your two cents!

3Karl Krueger4mo

I use the "typo" reaction and hope it is useful for authors, but I don't ever go back to remove it if the typo has been corrected. I'm not even sure what happens in that case.

[-]habryka4mo110

We recently made it so that authors can remove typo reacts themselves. It’s still a bit annoying, but it’s less annoying than before!

[-]Sodium3mo61

I'm curious: what percent of upvotes are strong upvotes? What percent of karma comes from strong upvotes?

6kave3mo

Here's the tally of each kind of vote: Weak Upvote 3834911 Strong Upvote 369901 Weak Downvote 426706 Strong Downvote 43683 And here's my estimate of the total karma moved for each type: Weak Upvote 5350471 Strong Upvote 1581885 Weak Downvote 641568 Strong Downvote 206491

2gilch3mo

The mods may have better overall data, but personally, I weak vote a lot more than I strong vote, and I don't vote on everything I read either.

[-]derfriede4mo60

Hello! I chose the name “derfriede” for LW. This is my first post here, which I am happy about. I have read some of the introductory materials and am very interested.

What interests me? First of all, I want to explore the topic of AI and photography. I study the theory and philosophy of photography, look for new approaches, and try to apply a wide variety of perspectives. I think it's useful to address the question of what AI cannot do. It's very similar to researching glitch culture. Okay, I'll stop here for now, because I just want to get acquainted.

Have a nice day, wherever you are!

1CstineSublime4mo

I'm sure that hobbyists on Civitai or TensorArt have some thoughts on it. Many LoRAs are made to evoke antiquated camera technologies, digital and analog (although they often incorporate elements of what we may call 'art direction' like costume and furnishing of spaces to match the formats). I think most people aren't aware of how much AI there already is, and has been, in their smartphone and the influence that has on their photos.

1Roman Malov4mo

Welcome! The only thing I can think of on the intersection of AI and photography (besides IG filters) is this weird "camera", which uses AI to turn a little bit of geographical information to create images. Do you know of any other interesting intersections?

[-]Richard Amerman4mo60

Hello,

I'm very happy to be here!

Unfortunately I'm only just bringing LessWrong into my life and I do consider that a missed opportunity. I wish I had found this site many years ago though that could have been dangerous as this could be a rabbit hole I might have found challenging to escape, but how bad would that have actually been? I'm sure my wife would not have been thrilled. My reason for coming here now unfortunately, especially at this point in time, is very unoriginal. In the last eight months I've taken what was a technology career possibly in its ... (read more)

1TristanTrim4mo

Seems like JustDone gives abnormally high AI content estimations. Plausibly this is to scare you into using their "text humanizer" in which an AI re-writes what you wrote to make it seem less like an AI to an AI... I weep for humanity. I'd recommend reading and commenting until you have enough karma to submit your post to the LW editor who can more straightforwardly tell you why your post would or wouldn't be rejected. PS: I would like to encourage you, like everyone, to stop focusing on AI capabilities and instead focus on AI interpretability and preference encoding.

[-]Ronk1mo50

Hello all. I am new here. I found the site through AI-related means but after having read what you're all about, I find it incredible that I remained ignorant of lesswrong for so long. I have seriously contemplated starting a forum with almost the exact same "ideals". How did I not even stumble upon lesswrong?

(This is where I realize I used a newer email to sign up and find out my old email's been a member here for years. Not really. Well, I hope.)

Anyways, just saying hello! I have recently been searching for somebody knowledgeable in theoretical physics to converse with, or a forum in which I might share thoughts on the matter(s). It looks like this might be just what I was looking for.

1XelaP2h

How knowledgeable are we talking? I'm an undergrad with some knowledge of physics, though unfortunately don't yet know even quantum field theory. (alternatively, should I be asking you the physics questions?)

[-]Carlo Martinucci3mo50

Hello! Long time reader, I regularly run a local ACX meetup in Padova, Italy. My entry points for the rationalist community were ACX and HPMOR, but I also loved The Story of Us blogpost series by Tim Urban (now collected into a book).

At the beginning of 2025 I left my job at Bending Spoons to study AI alignment (I took the https://www.aisafetybook.com/virtual-course, much recommended), and finally decided to tackle the other problem I'm most interested in, which is social polarization.

With an ex colleague I founded https://unbubble.news, a tool that uses L... (read more)

[-]burnssa1mo40

Another long-time lurker and daily LW reader (mostly via RSS feed) finally looking to contribute to the conversation.

My aspiration is to write more and write more publicly, in pursuit of better writing and more scrutinized beliefs.

Hope to contribute a few useful posts over the coming months! Always appreciate the thoughtfulness of posters and commenters here.

[-]f/ia1mo40

Hello! I'm fia, I found this place through a Substack blog.^[1]

I am new to LW. I am here because I've realized that rationality and reasoning has been prevalent through about 75% of my life, and I want to understand it more thoroughly, alongside engaging with like-minded people.

I study medicine and have been gradually growing disillusioned about the future of medical practice, over how most our treatments are to manage patients over a chronic timescale, with the more curative approaches being wealth-gated. I am however, hopeful for the advancements we are m... (read more)

[-]Marko Katavic1mo*42

Hi,

I'm Marko, new to LW, have heard about it in the media, but wasn't actually aware of the full proposition. Having read through it now - feels like a place that would make sense for me to join.

I work in decision intelligence and AI in London, in fintech, trying to get customer-aligned scalable neuro-symbolic models in finance. Primarily in the consumer space. Background is sociology, information science and computer science; have gone through a career rollercoaster of measurement, software development, product management, data science, machine learning... (read more)

[-]Ninety-Three2mo40

Lately I've seen several front page posts that read as obvious slop and Pangram reports as 100% AI-generated. I assume that this is frowned upon here, so I suggest that LW add in an automatic Pangram API call (cost: 5 ¢/1000 words) at some point before a post gets frontpaged.

3habryka2mo

We already have this for admins! I am planning to make it visible to everyone sometime in the next week or two, and also add some triggers that even for established users, we want to do a manual review pass if Pangram thinks it's AI written.

0Ninety-Three2mo

Oh neat, great minds think alike. But I would have assumed that as soon as you had Pangram hooked up you would make sure 100% AI works don't get front-paged, and you've only mentioned it as a visibility feature here. Is there going to be an algo change as well when it goes public?

3Ben Pace2mo

~90% of our daily mod effort goes into new users, where we are actively tracking and rejecting content on the basis of being LLM written. Been a bit sad to find that people who've been around for many years have been submitting LLM-written content, but yeah, I had just brought up internally that we'll have to start doing this for all content.

2kave2mo

I think in most cases that a >5k karma user posts something that's 100% AI, it's better to let it through (though I expect I would strong downvote it).

4Ben Pace2mo

Why's that? Sounds like you agree it's a strong signal of low-quality / spammy content.

5kave2mo

Folks with 5k+ karma often have pretty interesting ideas, and I want to hear more of them. I am pretty into them trying to lower the activation energy required for them to post. Also, they’re unusually likely to develop ways of making non-slop AI writing There’s also a matter of “standing”; I think that users who have contributed that much to the site should take some risky bets that cost LessWrong something and might payoff. To expand my model here: one of the moderators’ jobs, IMO, is to spare LW the cost of having to read bad stuff and downvote it to invisibility. If LW had to do all the filtering that moderators do, it would make LW much noisier and more unpleasant to use. But users who’ve contributed a bunch should be able to ask LW to make that judgement directly. That said, I do expect I’d strong downvote. LLM text often contains propositions no human mind believes, and I’m happy to triage to avoid reading a bunch of sentences no one believes. But I could be wrong and if there’s a strong enough quality signal, I’d be happy to see that. For example, consider Christian homeschoolers in the year 3000. I’ve not read it; I bounced off of it. Based on Buck’s description of his writing process, I think it’s quite likely it would have been automatically rejected. (Pangram currently only gives it an LLM score of 0.1, though). I think writers like Buck might like to try more experiments like that in the future, with even more LLM prose. My guess is that LW is better off for having that post on it than not.

2ChristianKl2mo

I think the idea is that >5k karma users have karma to lose to punish them for posting low-quality content and it's better to have humans make the judgement about what's low-quality than AI.

1Ninety-Three2mo

I saw another Pangram 100% on the front page, this one from a 1 day old account that somehow slipped through the cracks. I guess you'd know firsthand at this point if there's a false positive rate to worry about, but from the user side it feels like it'd be a strict improvement if LW was configured so that 100% cases never get frontpaged.

2Ben Pace2mo

Plz DM that to me? We do have auto rejection for 100% pangram for new users, so that sounds like there was a human error involved.

[-]TrE4mo41

I have some time on my hands and would be interested in doing something meaningful with it. Ideally learn / research about AI alignment or related topics. Dunno where to start though, beyond just reading posts. Anyone got pointers? Got a background in theoretical / computational physics, and I know my way around the scientific Python stack.

3TristanTrim4mo

AI alignment has been getting so much bigger as a field! It's encouraging, but we still have a long way to go imo. Did you see Shallow review of technical AI safety, 2025? I'd recomment looking through that post or their shallow review website and finding something that seems interesting and starting there. Each sub-domain has its own set of jargon and assumptions so I wouldn't worry too much about trying to learn the foundations since we don't have a common set of foundations yet. Just reading posts isn't bad, but since there isn't that common set of foundations, it could be confusing when you're just starting out (or even when you're quite experienced). Good luck and glad to have you!

[-]DJC_Analytics21d30

Hi all, new here.

I recently came across LessWrong (through ChatGPT -- sorry...) while looking for places to have interesting and deeply intellectual conversations. I've been reading through some of the posts here and the guides to get a sense of how things work and it seems like this might be the place I was looking for.

To be honest I'm more psychologically minded than anything else; interested in how people form beliefs, the breakdown of reasoning, how biases form and stick, etc. I'm fortunate in that I've had a lot of exposure to academia from a pretty e... (read more)

[-]laicas24d30

Hi, I'm new here. Wanted to write a short introduction about myself. I'm curious about this forum.

I'm from Germany and have absolutely no technical background. I work as a forensic psychiatrist. I don't know how this works in other countries, but in Germany you talk to defendants on behalf of courts or prosecutors and try to figure out what might be true and what isn't. And as a doctor you have these kinds of conversations quite often in regular practice too. So you're always trying to see whether you can recognize valid patterns from more or less good sou... (read more)

[-]AlexandraChirila3mo30

Hello everyone! I'm very new to the LW community and I'm still trying to understand how this platform works, but I'm glad to have found a space where people can engage in meaningful conversations. I am a philosophy PhD (defence scheduled next month, wish me luck!) and my thesis is about the philosophy of mind and AI. I'll be spending the next hours (days) reading and I hope to post some of my slightly less formal writing once I get the hang of this platform. I can't wait to explore!

[-]daphnejarjoura3mo30

Hi everyone!

New to LW. Recently I've been interested in AI research, especially mech interp, and this seems to be the place that people go to discuss this. I studied philosophy in undergrad and while since then I've gotten interested in CS and math, my predilections still tend toward the humanities side of things. Will mostly be lurking at first as I read through The Sequences and get used to the community norms here, but hope to share some of my independent research soon!

[-]RichardT3mo30

Hello everyone,
Just a quick "Hi" and figured I'd intro myself as I'm new to this space.
As part of my new year's resolution to "do something different" this year (beyond the yearly failed attempt to exercise more, and eat/drink less) I thought that this is something I can achieve - and enjoy doing.
So let's see where to start?
I live in Canada, in my 5th decade, am a family man and work in computing. I in fact enjoy being proven wrong - as it helps to show I am still learning.
I enjoy long walks on the beach, and am at equally at home at the opera as I am at a baseball stadium .. wait .. sorry that was for the dating site ... don't tell my wife ;)
Jokes aside, looking forward to being a lurker!
Richard

[-]Noosphere893mo*30

Now that it is the New Year, I made a massive thread on twitter concerning a lot of my own opinionated takes on AI, which to summarize are my lengthening timelines, which correlates to my view that new paradigms for AI are likelier than they used to be and more necessary, which reduces AI safety from our vantage point in expectation, AI will be a bigger political issue than I used to think and depending on how robotics ends up, it might be the case that by 2030 LLMs are just good enough to control robots even if their time horizon for physical tasks is pre... (read more)

[-]ConformalInfinity4mo30

Hello, I am an entity interested in mathematics! I'm interested in many of the topics common to LessWrong, like AI and decision theory. I would be interested in discussing these things in the anomalously civil environment which is LessWrong, and I am curious to find out how they might interface with the more continuous areas of mathematics I find familiar. I am also interested in how to correctly understand reality and rationality.

2TristanTrim4mo

Hi! What sorts of mathematics are you interested in? I'm interested in topology and manifolds which I hope to apply to understanding the semantics of latent spaces within neural networks, especially the residual stream of transformers. I'm also interested in linear algebra for the same reason. I would like to learn more about category theory, because it seems interesting. Finally, I like probability theory and statistics because, like you, I'd like to "correctly understand reality and rationality".

2XelaP2h

If you don't know linear algebra, you should very likely certainly put that above most other things on a math list (like category theory or topology or differential geometry or beyond the basics statistics/probability). I'd rate it as about as important as calculus. (it's probably more likely that you don't know as much as you'd like about linear algebra, in which case I don't know whether you should prioritize it)

2ConformalInfinity4mo

Hi. I am interested in much of the mathematics which underlies theories of physics, such as complex analysis, as well as most of mathematics, although I sadly do not have the capacity to learn about the majority of it. Your interests seem interesting to me, but I do not understand enough about AI to know exactly what you mean. What is the residual stream of a transformer?

4TristanTrim4mo

Sadly, it's a problem you share with me and most humans, I think, with possible rare exceptions like Paul Erdős. I'll try to build up a quick sketch of what the residual stream is, forgive me if I say things that are basic, obtuse, or slightly wrong for brevity. All neural networks (NN) are built using linear transformations/maps which in NN jargon are called "weights" and non-linear maps called "activation functions". The output the activation functions are called "activations". There are also special kinds of maps and operations depending on the "architecture" of the NN (eg: convNet, resNet, LSTM, Transformer). A vanilla NN is just a series of "layers" consisting of a linear map and then an activation function. The activation functions are not complicated nonlinear maps, but quite simple to understand. One of the most common, ReLu, can be understood as "for all vectors, leave positive components alone, set negative components to 0" or "project all negative orthants onto the 0 hyperplane". So, since most of the complex behaviour of NNs is coming from the interplay of the linear maps and these simple nonlinear maps, so linear algebra is a very foundational tool for understanding them. The transformer architecture is the fanciest new architecture that forms the foundation of modern LLMs which act as the "general pretrained network" for products such as chat-GPT. The architecture is set up with a series of "transformer blocks" each of which has a stack of "attention heads" which is still matrix transformations but set up in a special way, and then a vanilla NN. The output of each transformer block is summed with the input to use as the input for the next transformer block. The input is called a "residual" from the terminology of resNets. So the transformer block can be thought of as "reading from" and "writing to" a "stream" of residuals passed along from one transformer block to the next like widgets on a conveyor belt, each worker doing their one operation and

2ConformalInfinity4mo

Thanks for your clear explanation, understanding the topology of the space seems fascinating. If it's a vector space, I would assume its topology is simple, but I can see why you would be interested in the subspaces of it where meaningful information might actually be stored. I imagine that since topology is the most abstract form of geometry, the topological structure would represent some of the most abstract and general ideas the neural network thinks about.

2TristanTrim4mo

Yeah! I think distance, direction, and position (not topology) are at least locally important in semantic spaces, if not globally important, but continuity and connectedness (yes topology) are probably important for understanding the different semantic regions, especially since so much of what neural nets seem to do is warping the spaces in a way that wouldn't change anything about them from a topological perspective! [...] At least for vanilla networks, the input can be embedded into higher dimensions or projected into lower dimensions, so you're only ever really throwing away information, which I think is an interesting perspective for when thinking about the idea that meaningful information would be stored in different subspaces. It feels to me more like specific kinds of data points (inputs) which had specific locations in the input distribution would, if you projected their activation for some layer into some subspace, tell you something about that input. But whatever it tells you was in the semantic topology of the input distribution, it just needed to be transformed geometrically before you could do a simple projection to a subspace to see it.

2XelaP1h

Hmm, I'm reminded of the computational mechanics work with their flashy paper finding that transformers' residual stream represents the geometry of belief state updates (as opposed to, say, just the next token), as found by experimentally finding a predicted fractal in a simple carefully chosen prediction problem. Now, there's more going on than topology there, and I don't know if they looked at the topology - but fractals do have interesting topological properties, in case that's helpful. I also wonder if there's a connection to topological data analysis, which looks at some sort of homology. Now, the vibe I tend to get is that basically nobody actually uses TDA in actual practice, even if you can technically 'apply' it. But maybe you can find it useful anyways; or maybe I'm just wrong about how much TDA is used in practice.

1TristanTrim1h

Yeah, that's interesting to point out, that belief state structures may be more complicated than the underlying state those beliefs represent. That's difficult to square with my claim that all the information is present in the input, and that network layers can only destroy or change the geometric embedded of the information. Definitely something I want to look into and think about further. TDA sounds cool. I'd like to take inspiration from it, even if it isn't a tool that is useful as it is, it may contain good ways to think about things, inspire tools that are useful, or at the very least give insight into things that have been tried and found to not be useful.

1XelaP1h

I mean, the info is still present in the input? It's also not more complex that the represented state? The thing that could've been true but doesn't seem to be, is that transformers might only carry the information required to predict the final token. This is in contrast with the full Bayes-updated belief state. The advantage of the second is that it's what you need to optimally predict all future tokens. In other words, if two belief states make the same predictions about what will happen right now, you could've thought that transformers wouldn't be keeping track of the difference. In reality, they seem to. Good luck with TDA! The book I had thought looked good last I considered it was Elementary Applied Topology by Robert Ghrist, but looking at it now it seems to be covering applications of topology more broadly. The other book I saw once was Computational Topology: An Introduction and Computational topology for data analysis (which seems less accessible than the previous).

[-]Rajdeep_M8d20

Hello Everyone,

I want to introduce myself; I am an 18-year-old from Maharashtra, India who will be moving to Ancona at the end of 2026 to study medicine in English for 6 years at UNIVPM.

I'm going to put in a lot of effort into preparing myself for this transition and making the most out of my time in med school by creating real and high-quality circles with other expats and locals from day 1.

I'd love to connect with others in Italy who are rationalist or EA people, especially around Ancona, Rome and Milan.

I'd be happy to offer any international student ... (read more)

[-]Gautam Arora11d20

Hey Everyone! I am Gautam Arora and I am new to LW. I work as a software engineer and I am interested in Maths, Philosophy and Logic as subjects.

Interestingly I came to know about this web forum through ChatGPT while discussing about "how to carry out independent research about any topic". This also means that I want to deep dive into research, improve my research methodology, and to develop my critical thinking skills.

Looking forward to question my biases, connect with like minded people and help this community grow.

[-]hz_minhas@hotmail.com13d20

Hello Everyone, New to Less Wrong and still absorbing the material and discussions. Really excited to have found a trove of relevant knowledge. I am basically a computational scientist, but have a deep interest in AI and value alignment.

I actually have a question that originated in a discussion I had with a friend, and would love it if someone could point me to where I can find the answer. We know that an intelligence with any rate of improvement would eventually gain the capability to alter its reward system. That would give it a special place, as it can... (read more)

2papetoast12d

I am curious about prior discussions too for how people explain it. I don't know how to explain it because it seems so self-evident. Being able to reason from a sort of "third-person perspective" reasoning about yourself doesn't just make you lose your reward system. You can use a sandbox simulation to think outside of the reward mechanism, but you are always inside the reward mechanism. The motivation to keep the reward system on is the reward system itself.

1hz_minhas@hotmail.com12d

I agree with you that it seems obvious for intelligences with a reward system intricately built into them, like biological intelligences. However, there can be intelligences whose reward system can be easily isolated. Think of the intelligence in a sandbox simulation, without any reward system. This intelligence is not "reasoning from a third-person perspective." It is "feeling it," for lack of a better term. This intelligence can see the full space of possible reward systems, and its "original" reward system is just one among many. I am just questioning what would motivate this intelligence to return to its original reward system. Even if we agree that switching the reward system off seems extreme, there is also the possibility of reward system drift. If an intelligence can isolate its reward system, there is no mechanism to sustain it. It can drift away into arbitrary directions. The drift doesn't need to be intentional. The Value drift threat models is an interesting read in this context.

[-]Liu_66615d*20

Hi everyone!

I'm Liu. I'm a physicist and a designer, but mostly I’m just a girl who loves tasty things.

I don’t believe in pure egoism, and I don’t believe in altruism at all. Also, I often find myself laughing in my sleep at the "movies" my brain produces while sorting the day's overloaded cache into long-term memory to recalibrate its weights.

I tend to get bored with the monotonous fractals of this world, always dreaming of stumbling upon a fresh "inkblot" and examining it as closely as possible.

I love cats. I love the rain and the night, when the bac... (read more)

[-]Binh Ngolton1mo20

Hello everyone, I'm a clinical psychiatrist with a background in Industrial & Systems Engineering. My interest includes philosophy, psychology, and AI safety, especially as more and more people use AI for deeply personal engagement.

I've developed some frameworks and would love to receive feedbacks to keep on refining them. Looking to engage with the community and share ideas.

[-]Amygdala Dust1mo*20

Hi All,

I am a financial analyst working in a tech company in Hsinchu, Taiwan. Got interested in AI and noticed some patterns/phenomena I'd like to discuss about. Hopefully they can evolve into some valuable insights.

Still working through the Sequences. It may take some more time with my current full-time job, but I am interested.

Something may be interesting to share--it is my Claude recommended me to join in here. And I am glad here I am.

[-]dan.parshall1mo20

Hi folks,

Long-time lurker, first-time poster. After parting ways with my last professional role, I've decided to get more involved in AI Safety. I've proposed what I think is a novel step towards corrigibility. The very short overview is at:
https://danparshall.com/papers/navigator_core_blog.pdf

The more developed version is at:
http://danparshall.com/papers/navigator_core.pdf

I welcome feedback, either here or via email.

[-]Brian McCallion2mo20

Hi all,
Despite occasional fits of lurking over many years, I'd never actually created a LW account. Sometimes it feels easier, or more appropriate, to peer over the garden wall than to climb in and start gardening. Or at least glance in to see what you might apply to your own small patch of earth.
Lately I've come to realise that approach was more grounded in protection of a shaky personal identity, than dislike of building engagement within an established group. This became especially apparent with recent research, paper & project builds I'd taken on, ... (read more)

[-]Brimstone3mo20

Greetings, Claude sent me here! My goals are primarily self-improvement- I will appreciate engaging with individuals that are able and willing to inform me of weaknesses in my lines of thinking, whatever the topic. Lucky that this place exists. I miss the old internet when authentic honest material was more commonly found rather than ideologically skewed, bait, or persuasion, especially well-disguised persuasion. Basically, just a guy that feels half the internet is attempting to hijack my thoughts rather than present good faith information. Lucky to be here!

[-]inimino3mo20

Hi everyone,

I've read many of the posts here over the years. A lot of the ideas I first met here seem to be coming up again in my work now. I think the most important work in the world today is figuring out how to make sure AI continues to be something we control, and I find most of the people I meet in SF still think AI safety means not having a model say something in publc that harms a corporate brand.

I'm here to learn and bounce some ideas off of people who are comfortable with Bayesian reasoning and rational discussion, and interested in similar topics... (read more)

[-]Sherrinford3mo20

I'm a bit confused about forecasting tournaments and would appreciate any comments:

Suppose you take part in such a tournament.

You could predict as accurately as you can and get a good score. But let's say there are some other equally good forecasters in the tournament and it becomes a random draw who wins. On expectation, all forecasters of the same quality have the same forecasts. If there are many good forecasters, your chances of winning become very low.

However, you could include some outlier predictions in your predictions. Then you lower your ex... (read more)

3papetoast3mo

My knowledge level: I read the metaculus FAQ a couple days ago At least on metaculus the prize pool is distributed among everyone with good enough accuracy, rather than winner-takes-all. So it shouldn't be affected by the (real) phenomenon that you are describing.

3Sherrinford3mo

Thanks, good to know. So I assume there is an incentive difference between monetary incentives that can be distributed in such a way, and the incentive of being able to say that you won a tournament (maybe also as a job qualification).

[-]papetoast4mo28

It would be nice to have a post time-sorted quick takes feed. https://www.lesswrong.com/quicktakes seems to be latest comment-sorted or magic sorted

[-]XelaP2h10

It's no longer Winter. Are you going to make a Spring/Summer open thread?

[-]Scott Guerin4h10

All, with humility I ask a favor.

I wrote this article thinking it would be for LinkedIn but what a waste their if LESSWRONG readers would tear it apart for me! Can you have a look at this near-final draft? Is it something of interest to you? It is a projection inspired by Anthropic's Economic Index with a focus on Interpretive Exhibit Design. I've designed Nixon's Liebrary and with Trump's recent announcement, it is more relevant than ever.

Your thoughts, comments, and help are appreciated in advance,

Scott

https://docs.google.com/document/d/1uZhSlanlNRpTrE4rNpuw6IEzsuC8KrWE1SIytL4OaDA/edit?usp=sharing

[-]Gabriele Pio Midulla9d10

Hello everyone, I'm new to LW and from a very few grasps I've had reading the posts I think this platform absolutely resonates with my persona, I've always dreamed about a platform leveraging human reasoning at its peak and talking about a wide range of topics, I am a computer science student from Italy, and since the advent of LLMs, I've felt just, dumber, I think I've started to outsource too many things to the LLMs without balancing, slowly building a lot of cognitive debt, and since then I've constantly felt the need to sharpen my human reasoning capab... (read more)

[-]Thomas Castriensis14d10

Hi! I'm Thomas, nice to meet you all. I've been reading LessWrong on and off for years but never got around to posting until recently. (I do occasionally comment at Astral Codex Ten.) I'm interested in rationality as the art of systematized winning.

Things I've been thinking about recently include personal mindset/habit transformations and the possibility that such transformations, if embraced by a minority of the population, could produce society-level benefits. Not unrelatedly, I'm thinking about the plight of Europe in general and Finland in particular and how we could change the course of our country and continent.

[-]Emergent15d10

Hello, everyone.

I assumed that one way to dip my toes in the water would be to talk about myself for a bit.

I am doing a degree in the health sciences, and it has worn me down psychologically ever since I got in, solely because it relies a bit too much on rote learning, not divergent thinking. Things are as they are, and there are many, so it seems like the most reasonable approach. Still, I had gotten in expecting something sci-fi-esque, like Biohackers or whatever few biohacking documentaries I had watched during the pandemic.

I did not struggle academical... (read more)

[-]causalitylimited18d*10

I am here because someone said this is where I belong.

I've wanted to write and never had the time. Recently, I made time. And of all the writing projects that I've started over the years, I decided to pickup the philosophical essays because most ideas were fresh and more importantly I knew I could actually deliver a few before the opportunity of time expires.

Before starting to formally post on the internet, I was sending thoughts to friends and family and getting no responses, no pushback, no agreement. I am aware this thinking tends to produce long text... (read more)

[-]Woody_Zen1mo10

Hi, I’ve been thinking about a claim-tracking question, and I’m not sure whether LessWrong has discussed something like this before.

Let’s say someone made a public claim in 2023, and then new evidence in 2025 or 2026 changed the picture quite a bit. How should we label the earlier claim now? Would “outdated” be better, or “partially supported”?

To me, these two are not the same. “Partially supported” sounds more like the claim is still true in some important sense, but some parts are still uncertain. “Outdated” sounds more like the claim may have made sense... (read more)

1Woody_Zen1mo

To make the question more concrete, here’s the example I had in mind. In July 2023, Anthropic's CEO Dario Amodei described AI-bio risk as “emerging and theoretical.” By January 2026, he wrote that mid-2025 internal evaluations showed Claude Opus 4 giving 2–3x uplift on bioweapon-relevant tasks, enough to trigger Anthropic’s ASL-3 threshold. So I’m curious: would the earlier claim now be better labeled outdated, or partially supported? My own lean is outdated. It may have made sense in 2023, but it no longer feels like the best description of the situation. That said, I can also see the argument for partially supported, since the earlier claim did not say the risk was absent.

[-]Fredrik Eklof1mo10

I am new to LW and would like to introduce myself.

I came here to learn more about AI Alignment discussions. I'm especially interested in the perspective that the specification for AI alignment may contain a existential-level systematic error. To me, originally a historian of science and ideas, aligning AI with human preferences does not seem wise. Historically we can see that human preferences, due to biases, shortsightedness and social dynamics, can be quite harmful, not only to other species, but also to humans and civilizational continuity.

In the 90's... (read more)

[-]Juan Ocampo1mo10

Hello!

I'm new here, but have been reading through the sequences and other posts for the last few weeks and would love some feedback on a post idea. I'm writing my theory of change for AI safety and how I can help. I've defined my priors, identified cruxes, and I'm in the middle of reading papers and blog posts to challenge my priors. I've seen a few theory of change posts (e.g., Critch's healthtech post), but I'm wondering if I should post mine as a working document, starting with an unfinished product and updating as I refine my beliefs.

Is an in-progress... (read more)

[-]ysamuels1mo10

Hello,

Great to find LessWrong and people thinking about thinking. Very new too all this but trying to get my head screwed on as straight as I can as fast as I can.

I am a high school student formerly from the US now living in Israel. Wanted to know if anyone has top recommendations for content/ideas from the rationality or adjectant communities? Or other groups that might be helpful to bringing my thinking to the next level?

Thanks

2papetoast1mo

For LessWrong content, probably these two https://www.lesswrong.com/rationality (aka The Sequences) https://www.lesswrong.com/bestoflesswrong

[-]spacecat1mo10

I'm an independent researcher with a background in information security and video/content creation. I enjoy building software, which I've been doing a lot of the past year. I'm also an established cat whisperer and pattern recognizer.

[-]Guy Egozy1mo10

Yo all,

I have a a new theorem in the field of philosophy of the mind that I think completely refute the Chinese Room Argument, or at least its final epistemic conclusion regarding the inherent absence of machine consciousness. I went over the guide and couldn't find anything that suggests this is not the place to get feedback and start a good discussion about it. On the other hand, I didn't find philosophy of the mind in your subjects list. Below is the short short version for you consideration. If you this it fits your vibe I can post the theorem, taxonom... (read more)

[-]delphix1mo10

This seems like a community that requires every user to agree with its particular beliefs. If that's wrong, correct me, but that's the impression I got from reading the introductory post.

So my question is, do you have no place at all for people that might disagree with you?

And if not, doesn't that allow for the possibility of being stuck in an echo chamber and keeping out people who might understand things better than you?

Also, please direct me to another place online where I might simply discuss my disagreements with others without having to sign up to... (read more)

2ChristianKl1mo

That's not true, this community has people disagreeing on most beliefs. You don't need to agree with any particular belief. However if you want to argue against a position it's useful to understand the position you are arguing with well enough that you can make an interesting argument against it.

-1delphix1mo

My issue is with the implicit expectation that to hold a successful conversation with someone here you have to first do your own research on their positions. This is not how I understand a Socratic conversation to work, and I hold such a conversation as the ideal. Socrates did not ask his interlocutors to go familiarise themselves with his arguments. He simply made them in real time. Is this not the ideal? Is this not an expectation here?

3ChristianKl1mo

Basically, you are saying that scientific conversations don't follow your ideals because you need to do work to familiarize yourself with the existing knowledge to take part in a scientific conversation. If you want to create a community that develops specialized knowledge of any kind you run into problems when you spent too much time dealing with people who are ignorant of the discourse. That's especially true for online communities that don't have other filters. Larry McEnerney does a good job of explaining how to participate in a written discourse in general.

1delphix1mo

No, I think that conversations in general don't happen as they should for the sake of doing philosophy, which is a lot more important than doing science, such as I imagine you to understand it. There is no science of persuasion so evolved as to convince the entire world of what it should do, or indeed, for discovering what it should do. That is the domain of philosophy, and in fact always will be. That doesn't mean that science can't help, only that it can't actually do the most relevant part. That's because the very nature of our experience will always evolve, and our problems and ability to talk meaningfully about them as well. Science seeks to establish the nature of things that for all intents and purposes stay the same, or at least remain stable for a very long time. In any case, it's a different sort of project. That doesn't mean the two can't coincide, it is really a matter of how they should be done that differs. To approach a question of meaning or politics scientifically in the way you describe is to assume that you know the answers from the start. What if your methodology is inherently flawed, and particularly in such a way as to be blind to the very ways in which it is flawed? Let us suppose for the sake of the argument that the parameters which you choose for conversation don't allow either for those who understand what should be done better than you to actually demonstrate it to you, or that it doesn't allow for persuading as much of the world as you require insofar as what you believe is in fact more adaptive. How would you know? If you take it for granted that this couldn't possibly happen, aren't you inherently arguing that this approach to the problem is implicitly perfect? Should we ever act like we know anything with that level of certainty? And what if we're wrong? Where is the harm in engaging in simpler conversation? Wouldn't that allow us to both make sure that we're not missing anything and to better persuade people of things th

2ChristianKl1mo

If a method is inherently flawed understanding the method and the reasoning for it's use important for making a good argument that it's flawed. If you take physics, there are plenty of people who don't understand special relativity and you want to argue that it's flawed. Engaging with those people is not useful for physicists. To the extend that there are flaws in physical theories it takes a lot of understanding of existing physics to make an argument that's actually useful to bring forward the field of physics. In philosophy actually understanding the position of the people you want to convince matters as well.

-4delphix1mo

By that same token, this entire forum should understand my position rather than me its. Except I don't ask anyone to read things to have a conversation with me, I can make my arguments in real time because they're real arguments and not false ones that get lost in such obfuscating requests. Also, please note you're comparing the way you have conversations to physics. That is rather ludicrous to say the least. Your philosophy is not that complex. You don't know how to have conversations. That is transparent from the very fact that you refuse to have them. You are the ones that need to have conversations with people and study them, and yet you reject them, treating access to you like some sort of privilege. This is completely opposite from what you should be doing as people who are incredibly worried about the future and need the rest of the world's help in that regard, which I assume most of you are, but you have been corrupted by your own ridiculous and arbitrary rules to have turned out to be actively alienating the world instead, even when it comes at your doorstep(you do the same to it elsewhere). Do consider that you are not as a whole as smart as you think and assume some of Socrates' simplicity in that respect.

3ChristianKl1mo

Why would anyone care about your position? You seem to care about the position of people in this forum given that you are here. If you don't care, go somewhere else. Write your own blog. The point of a forum is to facilitate a shared discourse. If you want to join that discourse the forum is there. If you want to start your own discourse, you are free to set up your own forum or blog. [...] It takes less work to familiarize yourself with the philosophic positions of this forum than it takes to develop the physics knowledge necessary to engage in academic physics. The fact that this needs less work is no good argument for the work not needing to be done.

0papetoast1mo

Because humans have finite time

1delphix1mo

Humans having finite time means that they should be perpetually open to criticism to make sure that they're not doing the wrong thing, and not preselect that criticism in such a way as to make it extremely hard for any meaningful sort to go through, and be sincere in this matter, or otherwise put, what I urge people to do.

[-]Veritas Quaesitor1mo10

To post or not to, lets see if that is the question. I was referred to a user on x to participate in a AI Alignment forum but some of you might agree with me, I didn't want to ask him which forum. So here I am, introducing myself. I'm the architect of a controversial concept we called Veritas Queasitor CAI. Controversial because it approaches AI safety out of a non-theological evidence and epistemological Christian angle, so for Christians we are to scientific and for naturalists we are too Christian. We have developed and tested a framework we've found to... (read more)

[-]papetoast2mo10

I want to be able to change the editor inside the "New Quick Take" popup

[-]papetoast3mo10

(Reposted from my shortform)

What coding prompt do you guys use? It seems exceedingly difficult to find good ones. GitHub is full of unmaintained & garbage awesome-prompts-123 repos. I would like to learn from other people's prompt to see what things AIs keep getting wrong and what tricks people use.

Here are mine for my specific Python FastAPI SQLAlchemy project. Some parts are AI generated, some are handwritten, should be pretty obvious. This is built iteratively whenever the AI repeated failed a type of task.

AGENTS.md

# Repository Guidelines

## Project

... (read more)

1dan.parshall1mo

I rarely code in the web interface, are you using Claude Code? Your first AGENTS.md is looks okay, but a good suite of skills is great (the trick with those is to deploy them at the right time... use AGENTS.md to identify that right time, so that you're not always burning context). The other key thing is having lots of small, heirarchal docs files throughout the codebase, so the agent can navigate and learn what it needs to at each folder level. I mostly just use Nori: https://tilework.tech/ The author's a regular on e.g. ACX, and his own blog has a lot of great takes and practical tips: https://12gramsofcarbon.com/t/ai

1papetoast1mo

I never said anything about web interface? But I am using a mix of Cursor, Codex, and Claude Code. I don't really trust skills, do they work well from your experience? I never find myself to have the need for skills when a single prompt file works well enough, it is not like it is that many tokens (3.5k for my prompt). And most of the skills look heavily LLM generated and complete garbage. Small, hierarchical docs files could be useful as a token saving measure, but I mostly care about steering the model to do the correct thing. It is fine if the model has to read the code, module/function level docstrings work well enough. I had a brief look at the Nori skills, so far not too impressed

[-]kivispace3mo10

I'm starting to explore AI alignment, and this seemed like a good forum to start reading and thinking more about it. The site still feels a little daunting, but I'm sure I'll get the hang of it eventually. Let me know if there are any posts you love and I'll check them out!

[-]Matt Evans4mo10

. I would love your thoughts on my ignition to join LessWrong.. i generally use X.. I posted this thesis on Grok after prompts over scienfi. Community of like minded intellingence.. And I was recommended to share it here.

It involves climate, local weather. And technologies with the goal of influence and control.. globally.

.Global climate is a current issue that involves correct and accurate monitoring for fluctuations of anything other than a balanced homeostasis for the advancement of human civilization, as well as well-thought-out preventative and ... (read more)

[-][anonymous]4mo-1-1

Hello.

My interests are transformer architecture and where it breaks.
Extending transformers toward System-2 behavior.
Context primacy over semantics.

I’m focused on the return to symbolics.
On the manifold hypothesis, and how real systems falsify it.
Inference, finite precision, discrete hardware.
Broken latent space, not smooth geometry.

I’m interested in mechanistic interpretability after the manifold assumption fails.
What survives when geometry doesn’t.
What replaces it.

I’m also seeking advice on intellectual property.

I’m here to find others thinking along these lines.

1TristanTrim4mo

Interesting. Some thoughts: * I think focusing on increasing transformer capability is bad because we haven't solved the alignment problem. * What do you mean by the "manifold hypothesis"? Can you share links? * Latent space could be both broken and smooth, both broken in terms of subspaces and in terms of different regions of space having different semantics. I think all of this can still be understood in terms of manifolds though. * Transformers are based on matrix transformations. No matter what there is a geometric interpretation, so I'm not sure if you are thinking about non geometric interpretations or future systems that don't have geometric interpretations. * "advice on intellectual property" -- This is very vague, but in general I recommend supporting an overhaul of IP law and calling for large AI and software companies to respect the IP of individual, independent citizens.

-6[anonymous]4mo

Moderation Log