by lc1 min read19th Mar 202081 comments
80 comments, sorted by Click to highlight new comments since: Today at 5:05 AM
New Comment
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings


<3. Thanks for letting us know.

That's very brave
6Evan R. Murphy2mo
What happened to lc? They contributed to some good discussions on here, but look to have suddenly disappeared.

To the LW devs - just want to mention that this website is probably now the most well designed forum I have ever used. The UX is almost addictively good and I've been loving all of the little improvements over the past year or so.

Ditto here; kudos to everyone involved in creating such an excellent forum design!
I find [] hard to use (because I have yet to find a way to disable the mouseovers, which quickly deplete my orienting response, about which I can explain more if asked) but LW is better than most sites in that alternative interfaces can be created. In particular, I use [] as my interface and am pretty satisfied with it (though it was slow for a lot of the last 2 months). But I strongly upvoted parent because it is good reminder to me of the cognitive diversity in the human population.
I wrote somewhere that this is the only forum that looks to me the result of Intelligent Design, and not an accident. It's the only one that looks like I AM trying to intelligently design the forum MYSELF, including going back in time after discovering problems and fixing them (or just thinking in advance for five minutes on each aspect "how can I hack this / what are the vulnerabilities of this system of rules /how trolls can use it"). The point is not only that, unlike many other sites, I don’t think every five minutes “why can’t X be here”, the point is that I look somewhere and see in advance that something is provided that I don’t I had time to think, and some kind of protection was made against the exploitation of vulnerabilities in the system of rules or even involuntary errors in human psychology.
I agree completely. The last N weeks or so there have been performance problems, but all of the little things... Version history on posts, strong upvotes/downvotes, restoration of comments... They make writing things just fun.
If we still talk about shortcomings, then I would still be able to name 4, I wrote about the first two in the questions: lack of arrows between sequences; useless SEQ RERUNs for the sake of comments and problems with missing nested answers to questions in old comments; lately performance problems (which turned out to be lesswrong problems, not mine, so I didn't count them before); the fact that from time to time they vote for you once in the minus for completely incomprehensible reasons and then this value does not return to the plus (but as far as I understand, setting the need to indicate the reason for a bad vote will be either harmful or not very useful measure). But considering that for all this time I have found the number of minuses that can be counted on the fingers of one hand, while on any other site they literally pour from every element every second like from a cornucopia and instead of eliminating them, monthly useless graphic updates are made .. All in all, this is just a surprisingly good result, although (there is no limit to perfection) I hope at least three of them will be fixed in the coming months (how about the last one I do not know and there seems to be reasons why it is not fixed. But just in case I I note this minus, otherwise, as in the case of glitches, it turns out recently that everyone simply did not report it).

The problem with trade agreements as a tool for maintaining peace is that they only provide an intellectual and economic reason for maintaining good relations between countries, not an emotional once. People's opinions on war rarely stem from economic self interest. Policymakers know about the benefits and (sometimes) take them into account, but important trade doesn't make regular Americans grateful to the Chinese for providing them with so many cheap goods - much the opposite, in fact. The number of people who end up interacting with Chinese people or intuitively understanding the benefits firsthand as a result of expanded business opportunities is very small.

On the other hand, video games, social media, and the internet have probably done more to make Americans feel aligned with the other NATO countries than any trade agreement ever. The YouTubers and Twitch streamers I have pseudosocial relationships with are something like 35% Europeans. I thought Canadians spoke Canadian and Canada was basically some big hippie commune right up until my minecraft server got populated with them. In some weird alternate universe where people are suggesting we invade Canada, my first instinctual... (read more)

This doesn't seem like an either-or question. Freer trade and more individual interactions seem complementary to me.
I should note that I'm also pro free trade, because I like money and helping people. I'm just not pro free trade because I think it promotes peace.

Does anybody here have any strong reason to believe that the ML research community norm of "not taking AGI discussion seriously" stems from a different place than the oil industry's norm of "not taking carbon dioxide emission discussion seriously"?

I'm genuinely split. I can think of one or two other reasons there'd be a consensus position of dismissiveness (preventing bikeshedding, for example), but at this point I'm not sure, and it affects how I talk to ML researchers.

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it!" Upton Sinclair []
I'm not sure the "ML Research Community" is cohesive enough (nor, in fact, well-defined enough) to have very strong norms about this. Further, it's not clear that there needs to be a "consensus reasoning" even if there is a norm - different members could have different reasons for not bringing it up, and once it's established, it can be self-propagating: people don't bring it up because their peers don't bring it up. I think if you're looking for ways to talk to ML researchers, start small, and see what those particular researchers think and how they react to different approaches. If you find some that work, then expand it to more scalable talks to groups of researchers.
1Purged Deviator2mo
I don't expect AI researchers to achieve AGI before they find one or more horrible uses for non-general AI tools, which may divert resources, or change priorities, or do something else which prevents true AGI from ever being developed.
Because of it's low chance of existential risk or a singularity utopia. Here's the thing, technologies are adopted first at a low level and at early adopters, then it becomes cheaper and better, than it more or less becomes very popular. No technology ever had the asymptotic growth or singularity that ML/AI advocates claim to have happened. So we should be very skeptical about any claims of existential risks. On climate change, we both know it will be serious and that it is not an existential risk or a civilization collapse disaster.
1Michaël Trazzi2mo
I think best way to look at it is climate change way before it was mainstream

Hey [anonymous]. I see you deactivated your account. Hope you're okay! Happy to chat if you want on Signal at five one oh, nine nine eight, four seven seven one (also a +1 at the front for US country code).

(Follow-up: [anonymous] reached out, is doing fine.)

So apparently in 2015 Sam Altman said:

“AI will probably most likely lead to the end of the world, but in the meantime, there’ll be great companies.”

Serious question: Is he a comic book supervillain? Is this world actually real? Why does this quote not garner an emotive reaction out of anybody but me?

I was surprised by this quote. On following the link, the sentence by itself seems noticeably out of context; here's the next part:

LessWrong as a website has gotten much more buggy for me lately. 6 months ago it worked like clockwork, but recently I'm noticing that refreshes on my profile page take something like 18 seconds to complete, or even 504 (!). I'm trying to edit my old "pessimistic alignment" post now and the interface is just not letting me; the site just freezes for a while and then refuses to put the content in the text box for me to edit.

Marvelous. I didn’t talk about this because I thought that the problem was not on the side of LessWrong, since in my country a lot of things have been slowing down, blocking, denying access, and so on, and at least from three sides at the same time: the state / providers, others countries / companies and those who do not want problems.
In order to synchronize against the illusion of transparency, I will write specific errors that I myself see: bad gateway (seems to be somehow related to following links within the site and back); "Error: NotFoundError: Failed to execute removeChild on Node: The node to be removed is not a child of this node." (red, replaces the entire page, sometimes appears when you click "submit"); long page loading at the beginning; long loading of the remaining page after the update of the profile karma indicator and new messages has loaded; when double-clicking (on the phone), the voice is not amplified, but reset.
The performance problems have also been annoying me, though I don't think it's been 6 months since they've gotten worse (I think it's been more like 4 weeks based on my read of the logs, which have sadly overlapped with some time period where it's been hard for me or others to focus on fixing them). I've really hated it, and if I didn't have COVID right now, would probably be trying to fix them right now. Not sure what's up about the editor. I don't think I've experienced many additional problems here, though we have been rolling out a new editor, so new bugs aren't that surprising. A bug report via Intercom would be greatly appreciated.
Sounds very likely upon reflection that I could be misremembering them to that far out; I just picked a date upon which the site definitively worked fast.
Have you reported this to the staff?

Noticed something recently. As an alien, you could read pretty much everything Wikipedia has on celebrities, both on individual people and the general articles about celebrity as a concept... And never learn that celebrities tend to be extraordinarily attractive. I'm not talking about an accurate or even attempted explanation for the tendency, I'm talking about the existence of the tendency at all. I've tried to find something on wikipedia that states it, but that information just doesn't exist (except, of course, implicitly through photographs).

It's quite odd, and I'm sure it's not alone. "Celebrities are attractive" is one obvious piece of some broader set of truisms that seem to be completely missing from the world's most complete database of factual information.

Analyzing or talking about status factors is low-status. You do see information about awards for beauty, much like you can see some information about fiances, but not much about their expenditures or lifestyle.
4G Gordon Worley III3mo
Part of the issue is like that celebrity, as wikipedia approaches the word, is broader than just modern TV, film, etc. celebrity and instead includes a wide variety of people who are not likely to be exceptionally attractive but are well known in some other way. There's individual preferences in terms of who they think are attractive, but many politicians, authors, radio personalities, famous scientists, etc. are not conventionally attractive in the way movie stars are attractive and yet these people are still celebrities in a broad sense. However, I've not dug into the depths of wikipedia to see if, for example, this gap you see holds up if looking at pages that more directly talk about the qualities of film stars, for example.
I think there's also a "it's obvious to everyone, so archaeologists of the future won't find any mention of it because no one has had to explain it to anyone" factor. (I heard that archaeologists and historians know much less about everyday life than about significant events, although the former was obviously encountered much more often)

Made an opinionated "update" for the anti-kibitzer mode script; it works for current LessWrong with its agree/disagree votes and all that jazz, fixes some longstanding bugs that break the formatting of the site and allow you to see votes in certain places, and doesn't indent usernames anymore. Install Tampermonkey and browse to this link if you'd like to use it. 

Semi-related, I am instituting a Reign Of Terror policy for my poasts/shortform, which I will update my moderation policy with. The general goal of these policies is to reduce the amount of ti... (read more)

I have no clue whether any of my previous comments on your posts will qualify me for perma-ban, but if so, please do so now, to save the trouble of future annoyance since I have no intention of changing anything. I am generally respectful, but I don't expect to fully understand these rules, let alone follow them. I have no authority over this, but I'd hope the mods choose not to frontpage anything that has a particularly odd and restrictive comment policy, or a surprisingly-large ban list.
I think it's better to annoy commenters than to annoy post authors, so actually allowing serious Reign of Terror is better than meaningfully discouraging it. That's the whole point of Reign of Terror, and as the name suggests it shouldn't be guaranteed to be comfortable for its subjects. One problem with how it's currently used is authors placing Reign of Terror policy for their own comfort in a motte/bailey way, without any actual harsh moderation activity, inflating the category into the territory of expected comfort for the commenters. There should be weak incentive for authors to not do this if they don't actually care.
For a lot of posts, the value is pretty evenly distributed among the post and the comments. For frontpage-worthy ones, it's probably weighted more to posts, granted. I fully agree that "reign of terror" is not sufficient reason to keep something off frontpage. I was reacting more to the very detailed rules that don't (to me) match my intuitions of good commenting on LW, and the declaration of perma-bans with fairly small provocation. A lot will depend on implementation - how many comments lc allows, and how many commenters get banned. Mostly, I really hope LW doesn't become a publishing medium rather than a discussion space.
There's practically no reason on a rationality forum for you to assert your identity or personal status over another commenter. I agree the rules I've given are very detailed. I don't agree that any of the vast majority of valuable comments on LessWrong are somehow bannable by my standard. The reason I'm stringent about doing this, is because the status asserting comments literally ruin it for everybody else, even when the majority of everybody else is not interested in such competitions. They make people like me, who are jealous and insecure, review everything they've ever written in the light that they might be judged. I don't come here because I want to engage in yet another status tournament. I come here because I want to become a better thinker and learn new and interesting things about the world. I also come here because I like being able to presume that most of the other commenters are using the forum like I am. In this sense it's worth it to me if this policy prevents one person from trying to social climb even if I have to prevent four other comments that wouldn't otherwise be a problem.
As I said, obviously this is not a retroactively applying policy, I have not followed it until now, and I will not ban anybody for commenting differently on my posts. I'm not going to ban you pre-emptively or judge you harshly for not following all of my ridiculously complicated rules. Feel free to continue commenting on my posts as you please and just let me eventually ban you; that's honestly fine by me and you should not feel bad about it. I personally hope they would not refuse to frontpage my posts from now on for having a restrictive comment policy when it's not obviously censoring criticism of the post itself, but I have already forfeited arbitrarily large amounts of exposure and the mods can do what they wish.

Computer hacking is not a particularly significant medium of communication between prominent AI research labs, nonprofits, or academic researchers. Much more often than leaked trade secrets, ML people will just use insights found in this online repository called arxiv, where many of them openly and intentionally publish their findings. Nor (as far as I am aware) are stolen trade secrets a significant source of foundational insights for researchers making capabilities gains, local to their institution or otherwise. 

I don't see this changing on its own,... (read more)

Do we have a good idea of how prominent AI research labs compare to the resources that go into Five Eyes AI models for intelligence analysis and for Chinese government pursuits?
I've forgotten at this point who they are, but I will ask some of my friends later to give me some of the public URLs of the "big players" working in this space so you can partly see for yourself. Their marketing is really impressive because government contractors, but I encourage you to actually look at the product on a technical level. Largely: the NSA and its military-industrial partners don't come up with new innovations, except as applies to handling the massive amounts of data they have and their interesting information security requirements. They just apply technologies and insights from companies like OpenAI or DeepMind. They're certainly using things like large language models to scan your emails now, but that's because OpenAI did the hard work already. More importantly, when they do come up with innovations, they don't publish them on the internet, so they don't burn much of the "commons", as it were. I can't give much insight on china, sadly.
There was a large amount of time when the NSA did come up with cryptography-related math innovations in secret and did not share that information publically. The NSA does see itself []as the leading employer of mathematicians in the United States. To the extent that those employees come up with groundbreaking insights, those are likely classified and you won't find them in the marketing materials of government contractors.

Every once in a while I'm getting bad gateway errors on Lesswrong. Thought I should mention it for the devs.

Today, I also got errors.

In hindsight, it is literally "based theorem". It's a theorem about exactly how much to be based.

What theorem?
Bayes’ Theorem, presumably.
Especially to non-native speakers, it's not at all obvious that Bayes' Theorem and Based Theorem sound almost the same since d, which reads like t, merges with th.
I think we should reward admitting-of-ignorance, or at the very least not punish it.

Would it be a good idea to get [OP] stickers on comments by the author of the post?

Based on Victoria Nuland's recent senate testimony, I'm registering a 66% prediction that those U.S. administered biological weapons facilities in Ukraine actually do indeed exist, and are not Russian propaganda. 

Of course I don't think this is why they invaded, but the media is painting this as a crazy conspiracy theory, when they have very little reason to know either way. 

4Purged Deviator5mo
Here's an analysis by Dr. Robert Malone about the Ukraine biolabs, which I found enlightening: [] I glean that "biolab" is actually an extremely vague term, and doesn't specify the facility's exact capabilities at all. They could very well have had an innocuous purpose, but Russia would've had to treat them as a potential threat to national security, in the same way that Russian or Chinese "biolabs" in Mexico might sound bad to the US, except Russia is even more paranoid.
It seems like "biological weapons facility" is a quite subjective term. The US position is that their own army labs that produced anthrax that was used after 9/11 are not a "biological weapons facility" because while they do produce anthrax that could be used militarily, it's not produced with the intent of military use. Based on those definitions it's plausible that the Ukrainian labs produce viruses that can be weaponized but that the US just doesn't see them as a "biological weapons facility" because they believe the intent for offensive use isn't there. Glenn Greenwalds reporting is good on this. [] is the freely accessible video version, there's also a written version on his substack behind a paywall. If you make exact predictions like that you should define what you mean with your terms. It's like Fauci's dance saying that there's no gain-of-function research in the paper he mailed around with gain-of-function in the filename. The US government doesn't use commonsense definitions for words when it comes to biosafety.
I'm registering a 90% predicition those facilities do not exists, as in "how the hell would the US have been dumb enough to plant biological weapons facilities in a remote country outside their sphere of influence and where Russia has (used to have until recently) a lot of weight..."
Did you remember what weapons the US gave Iraq? How is arming Ukraine with such weapons less insane than it was with Iraq?
Wouldn't be the dumbest thing they've ever done.

Within the next fifteen years AI is going to briefly seem like it's solving computer security (50% chance) and then it's going to enhance attacker capabilities to the point that it causes severe economic damage (50% chance).

Does "seem like it's solving computer security" look like helping develop better passively secure systems, or like actively monitoring and noticing bad actions, or both or something else?
My thoughts are mostly about the latter, although better code scanning will be a big help too. A majority of financially impactful corporate breaches are due to a compromised active directory network, and a majority of security spending by non-tech companies is used to prevent those from happening. The obvious application for the next generation of ML is extremely effective EDR and active monitoring. No more lateral movement/privilege escalation on a corporate domain means no more domain wide compromise, which generally means no more e.g. big ransomware scares. The problem comes if/when people then start teaching computers to do social engineering, competently fuzz applications, and perform that lateral movement intelligently and in a way that bypasses the above, after we have largely deemed it a solved problem.

How long does it usually take for mods to decide whether or not your post is frontpage-worthy?

2Matt Goldenberg8mo
It varies but usually not long. My uninformed guess is that your recent post was deliberately not frontpaged because it's a political topic that could attract non-rationalists to comment and flame in an unproductive manner.

Two caveats to efficient markets in finance that I've just considered, but don't see mentioned a lot in discussions of bubbles like the one we just experienced, at least as a non-economist:

First: Irrational people are constantly entering the market, often in ways that can't necessarily be predicted. The idea that people who make bad trades will eventually lose all of their money and be swamped by the better investors is only valid inasmuch as the actors currently participating in the market stay the same. This means that it's perfectly possible for either ... (read more)

I need a LW feature equivalent to stop-loss where if I post something risky and it goes below -3 or -5 it self-destructs.

> be me

> start watching first episode of twin peaks, at recommendation of friends

> become subjected to the worst f(acting, dialogue) possible within first 10 mins

This makes me sad. Season one and the first 2/3 of season 2 were transformative and amazing for me and my nerdy college-age-at-the-time peer group. The end of that season and the followup movies were rather less so. I intellectually understand that it's no longer innovative or particularly interesting, and it hasn't aged very well either in terms of investigative technology nor mountain-town isolation and creepiness. Being local and contemporary likely helped a whole lot as well. Still, I visit Snoqualmie Falls and have brunch at the lodge there a few times a year, and the connection to twin peaks makes me smile a bit wider than just the beauty and power of nature would. Anyway, I look forward to hearing a review from your perspective if you decide to stick with it.

IMO: Microservices and "siloing" in general is a strategy for solving principal-agent problems inside large technology companies. It is not a tool for solving technical problems and is generally strictly inferior to monoliths otherwise, especially when working on a startup where the requirements for your application are changing all of the time.

The first three episodes of Narcos: Mexico, Season 3, is some of the best television I have ever seen. The rest of the "Narcos" series is middling to bad and I barely tolerate it. So far I would encourage you to skip to this season.

Spoilered, semi-nsfw extremely dumb question

If you've already had sex with a woman, what's the correct way to go about arranging sex again? How indirect should I actually be about it?

7French Marty20d
That is a very context dependent question. Your safest bet is to just arrange meeting her in a context where sex is a possibility (for example: "hey, do you want to go for coffee then stop at your place afterwards sometime?"). The desire to have sex isn't something you can forecast far in advance, it can quickly change just like the weather. You can have sexual conversation and establish the general desire for her to have you as a sexual partner. Essentially like saying she likes a particular restaurant but doesn't schedule going there days or even hours in advance, she's just open to going there when and if she feels the desire. As far as how to be good at sexual talk in general, unfortunately it takes careful practice. You just have to risk being akward or turning her off (within reasonable limits, don't immediately test saying something too crazy). Trial and error within reasonable bounds.
thanks fren

Lost a bunch of huge edits to one of my draft posts because my battery ran out. Just realizing that happened and now I can't remember all the edits I made, just that they were good. :(

Happened to someone else once :) []

I wish there were a way I could spend money/resources to promote question posts in a way that counterbalanced the negative fact that they were already mostly shown by the algorithm to the optimal number of people.

If you simply want to people to invest more into answering a question post, putting out a bounty for the best answer would be a way to go about it.

Interesting idea.

I just launched a startup, Leonard Cyber. Basically a Pwn2Job platform.

If any hackers on LessWrong are out of work, here are some invite codes:






The mandatory sign-up is a major obstacle to new users. I'm not going to create an account on a website until it has already proven value to me.
I think the multi-hour computer hacking gauntlet probably trumps any considerations of account creation in terms of obstacles to new users. Just in considering things we could pare down. We also need some way to prevent computer hackers from scraping all of the exam boxes, and that means either being enormously creative or at some point requiring the creation of an account that we KYC.

Does EY or RH even read this site anymore?

New to LessWrong?