the gears to ascenscion's Shortform

[edit: why does this have so many more upvotes than my actually useful shortform posts]

Someone mentioned maybe I should write this publicly somewhere, so that it is better known. I've mentioned it before but here it is again:

I deeply regret cofounding vast and generally feel it has almost entirely done harm, not least by empowering the other cofounder, who I believe to be barely better than e/acc folk due to his lack of interest in attempting to achieve an ought that differs from is. I had a very different perspective on safety then and did not update in time to not do very bad thing. I expect that if you and someone else are both going to build something like vast, and theirs takes three weeks longer to get to the same place, it's better to save the world those three weeks without the improved software. Spend your effort on things like lining up the problems with QACI and cannibalizing its parts to build a v2, possibly using ideas from boundaries/membranes, or generally other things relevant to understanding the desires, impulses, goals, wants, needs, objectives, constraints, developmental learning, limit behavior, robustness, guarantees, etc etc of mostly-pure-RL curious-robotics... (read more)

[edit: pinned to profile]

I feel like most AI safety work today doesn't engage sufficiently with the idea that social media recommenders are the central example of a misaligned AI: a reinforcement learner with a bad objective with some form of ~online learning (most recommenders do some sort of nightly batch weight update). we can align language models all we want, but if companies don't care and proceed to deploy language models or anything else for the purpose of maximizing engagement and with an online learning system to match, none of this will matter. we need to be able to say to the world, "here is a type of machine we all can make that will reliably defend everyone against anyone who attempts to maximize something terrible". anything less than a switchover to a cooperative dynamic as a result of reliable omnidirectional mutual defense seems like a near guaranteed failure due to the global interaction/conflict/trade network system's incentives. you can't just say oh, hooray, we solved some technical problem about doing what the boss wants. the boss wants to manipulate customers, and will themselves be a target of the system they're asking to build, just like sundar pichai has to use self-discipline to avoid being addicted by the youtube recommender same as anyone else.

This will be my last comment on lesswrong until it is not possible for post authors to undelete comments. [edit: since it's planned to be fixed, nevermind!]

originally posted by a post author:

This comment had been apparently deleted by the commenter (the comment display box having a "deleted because it was a little rude, sorry" deletion note in lieu of the comment itself), but the ⋮-menu in the upper-right gave me the option to undelete it, which I did because I don't think my critics are obligated to be polite to me. (I'm surprised that post authors have that power!) I'm sorry you didn't like the post.

some youtube channels I recommend for those interested in understanding current capability trends; separate comments for votability. Please open each one synchronously as it catches your eye, then come back and vote on it. downvote means not mission critical, plenty of good stuff down there too.

I'm subscribed to every single channel on this list (this is actually about 10% of my youtube subscription list), and I mostly find videos from these channels by letting the youtube recommender give them to me and pushing myself to watch them at least somewhat to give the cute little obsessive recommender the reward it seeks for showing me stuff. definitely I'd recommend subscribing to everything.

Let me know which if any of these are useful, and please forward the good ones to folks - this short form thread won't get seen by that many people!

edit: some folks have posted some youtube playlists for ai safety as well.

things upvotes conflates:

• agreement (here, separated - sort of)
• respect
• visibility
• should it have been posted in the first place
• should it be removed retroactively
• is it readably argued
• is it checkably precise
• is it honorably argued
• is it honorable to have uttered at all
• is it argued in a truth seeking way overall, combining dimensions
• have its predictions held up
• is it unfair (may be unexpectedly different from others on this list)

(list written by my own thumb, no autocomplete)

these things and their inversions sometimes have multiple components, and ma... (read more)

reply to a general theme of recent discussion - the idea that uploads are even theoretically a useful solution for safety:

• the first brain uploads are likely to have accuracy issues that amplify unsafety already in a human.
• humans are not reliably in the safety basin - not even (most?) of the ones seeking safety. in particular, many safety community members seem to have large blindspots that they defend as being important to their views on safety; it is my view that yudkowsky has given himself an anxiety disorder and that his ongoing insights are not as h

I have the sense that it's not possible to make public speech non-political, and in order to debate things in a way that doesn't require thinking about how everyone who reads them might consider them, one has to simply write things where they'll only be considered by those you know well. That's not to say I think writing things publicly is bad; but I think tools for understanding what meaning will be taken by different people from a phrase would help people communicate the things they actually mean.

Would love if strong votes came with strong encouragement to explain your vote. It has been proposed before that explanation be required, which seems terrible to me, but I do think it should be very strongly encouraged by the UI that votes come with explanations. Reviewer #2: "downvote" would be an unusually annoying review even for reviewer #2!

random thought: are the most useful posts typically karma approximately 10, and 40 votes to get there? what if it was possible to sort by controversial? maybe only for some users or something? what sorts of sort constraints are interesting in terms of incentivizing discussion vs agreement? blah blah etc

[posted to shortform due to incomplete draft]

I saw this paper and wanted to get really excited about it at y'all. I want more of a chatty atmosphere here, I have lots to say and want to debate many papers. some thoughts :

seems to me that there are true shapes to the behaviors of physical reality[1]. we can in fact find ways to verify assertions about them[2]; it's going to be hard, though. we need to be able to scale interpretability to the point that we can check for implementation bugs automatically and reliably. in order to get more interpretable sparsi... (read more)

my shortform's epistemic status: downvote stuff you disagree with, comment why. also, hey lw team, any chance we could get the data migration where I have agreement points in my shortform posts?

I hate to be That Guy, but are you aware that the usual spelling is "ascension"?

most satisficers should work together to defeat most maximizers most of the way

[edit: intended tone: humorously imprecise]

Connor Leahy interviews are getting worse and worse public response, and I think it's because he's a bad person to be doing it. I want to see Andrew critch or John Wentworth as the one in debates.

https://atlas.nomic.ai/map/01ff9510-d771-47db-b6a0-2108c9fe8ad1/3ceb455b-7971-4495-bb81-8291dc2d8f37 map of submissions to iclr

"What's new in machine learning?" - youtube - summary (via summarize.tech):

• 00:00:00 The video showcases a map of 5,000 recent machine learning papers, revealing topics such as protein sequencing, adversarial attacks, and multi-agent reinforcement learning.
• 00:05:00 The YouTube video "What's New In Machine Learning?" introduces various new developments in machine learning, including energy-based predictive representation, human le

They very much can be dramatically more intelligent than us in a way that makes them dangerous, but it doesn't look how was expected - it's dramatically more like teaching a human kid than was anticipated.

Now, to be clear, there's still an adversarial examples problem: current models are many orders of magnitude too trusting, and so it's surprisingly easy to get them into subspaces of behavior where they are eagerly doing whatever it is you asked without regard to exactly why they should care.

Current models have a really intense yes-and problem: they'll ha... (read more)

a comment thread of mostly ai generated summaries of lesswrong posts so I can save them in a slightly public place for future copypasting but not show up in the comments of the posts themselves

too many dang databases that look shiny. which of these are good? worth trying? idk. decision paralysis.

• https://www.edgedb.com/docs - main-db-focused graph db, postgres core
• https://terminusdb.com/products/terminusdb/ - main-db-focused graph db, prolog core (wat)
• https://surrealdb.com/ - main-db-focused graph db, realtime functionality,
• https://milvus.io/ - vector
• https://weaviate.io/developers/weaviate - vector, sleek and easy to use, might not scale as well as milvus but I guess I should just not care
• https://clientdb.dev/ - embedded, to compensate if

oh man I just got one downvote on a whole bunch of different comments in quick succession, apparently I lost right around 67 karma to this, from 1209 to 1143! how interesting, I wonder if someone's trying to tell me something... so hard to infer intent from number changes