alyssavance

FWIW I'm not convinced by the article on Haley, having bad conservative policies != being an anti-democratic nut job who wants to rig elections and put all your opponents in jail. She's super unlikely to win, though.

Replying toThe Dictatorship Problem

Most highly educated people lean left, but there really are just very few Stalinists. I quoted a poll above showing that just 8% of Americans would support an AOC hard-left party, and actual Stalinists are a small fraction of that. There's no developed country where tankies get a serious fraction of the vote. See Contrapoints for why communist revolutionaries are super unlikely to take power: https://youtu.be/t3Vah8sUFgI

There are many crazy professors of various stripes, but universities aren't states. They can't shoot you, can't throw you in jail, can't seize your house or business, and are ultimately dependent on government funding to even exist.

Replying toThe Dictatorship Problem

The current constitution isn't that old (although 65 years is still longer than most democracies), but with brief interruptions, France has been a democracy for around 150 years, which is far longer than most countries can claim.

Replying toThe Dictatorship Problem

Thanks for the response! Here are some comments:

- India, Turkey, and Hungary are widely referred to as "hybrid regimes" (https://en.wikipedia.org/wiki/Hybrid_regime), in which opposition still exists and there are still elections, but the state interferes with elections so as to virtually guarantee victory. In Turkey's case, there have been many elections, but Erdogan always wins through a combination of mass arrests, media censorship, and sending his most popular opponent to prison for "insulting public officials" (https://www.bbc.com/news/world-europe-63977555). In India's case, Modi is no doubt very popular, but elections are likewise hardly fair when the main opponent is disqualified and sent to prison for "defamation" (insulting Modi). Rather than being voted out, hybrid regimes usually... (read more)

Replying toWithout specific countermeasures, the easiest path to transformative AI likely leads to AI takeover

(Disclaimer: This is my personal opinion, not that of any movement or organization.)

This post aims to show that, over the next decade, it is quite likely that most democratic Western countries will become fascist dictatorships - this is not a tail risk, but the most likely overall outcome. Politics is not a typical LessWrong topic, and for good reason:

it tends to impair clear thinking;
most well-known political issues are not neglected;
most political "debates" are simply people yelling at each other online; neither saying anything new, nor even really trying to persuade the opposition.

However, like the COVID pandemic, it seems like this particular trend will be so impactful and so disruptive to ordinary Western... (read 3193 more words →)

145

Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover

I'm surprised by how strong the disagreement is here. Even if what we most need right now is theoretical/pre-paradigmatic, that seems likely to change as AI develops and people reach consensus on more things; compare eg. the work done on optics pre-1800 to all the work done post-1800. Or the work done on computer science pre-1970 vs. post-1970. Curious if people who disagree could explain more - is the disagreement about what stage the field is in/what the field needs right now in 2022, or the more general claim that most future work will be empirical?

Wealth as a source of technological stagnation?

(This is an essay I wrote last year, but hadn't published. I'm posting it here because some friends liked it and there's no particular reason to keep it private.)

The World Values Survey is a very comprehensive set of social science data that asks people, broadly, what they care about and how they think about life. It goes back forty years to 1981, and covers dozens of different countries.

The survey includes hundreds of questions, but the data analysts break responses down along two main axes, "traditional vs. secular/rational values" and "survival vs. self-expression values":

"Societies that emphasize survival values have relatively low levels of subjective well-being, report relatively poor health, and are low on... (read 697 more words →)

Replying toIt’s Probably Not Lithium

It’s Probably Not Lithium

I think saying "we" here dramatically over-indexes on personal observation. I'd bet that most overweight Americans have not only eaten untasty food for an extended period (say, longer than a month); and those that have, found that it sucked and stopped doing it. Only eating untasty food really sucks! For comparison, everyone knows that smoking is awful for your health, it's expensive, leaves bad odors, and so on. And I'd bet that most smokers would find "never smoke again" easier and more pleasant (in the long run) than "never eat tasty food again". Yet, the vast majority of smokers continue smoking:

https://news.gallup.com/poll/156833/one-five-adults-smoke-tied-time-low.aspx

Replying toA Quick List of Some Problems in AI Alignment As A Field

A Quick List of Some Problems in AI Alignment As A Field

https://transformer-circuits.pub/ seems impressive to me!

Failing to fix a dangerous intersection

Over the last few years, many people have written about why America can't build things anymore (eg. here, although this is just one of hundreds of relevant essays). Ten years ago, when I was 21 and had just graduated, a friend told me about a dangerous intersection in Berkeley. I tried writing to the city and asking for it to be fixed; I'm posting the email and reply here as a useful data point.

My request:

To the Berkeley Department of Transportation:
I write to urge the City to address a dangerous intersection, at San Pablo and Gilman Streets. When approaching the intersection from the west (on Gilman), there are two lanes of roadway, of

... (read 411 more words →)

110

Replying toA Quick List of Some Problems in AI Alignment As A Field

A Quick List of Some Problems in AI Alignment As A Field

There are now quite a lot of AI alignment research organizations, of widely varying quality. I'd name the two leading ones right now as Redwood and Anthropic, not MIRI (which is in something of a rut technically). Here's a big review of the different orgs by Larks:

https://www.lesswrong.com/posts/C4tR3BEpuWviT7Sje/2021-ai-alignment-literature-review-and-charity-comparison

Replying toNonprofit Boards are Weird

Nonprofit Boards are Weird

Great post. I'm reminded of instructions from the 1944 CIA (OSS) sabotage manual:

"When possible, refer all matters to committees, for “further study and consideration.” Attempt to make the committee as large as possible — never less than five."

AI Training Should Allow Opt-Out

Let's See You Write That Corrigibility Tag

Last year, GitHub announced their Copilot system, an AI assistant for developers based on OpenAI's Codex model, as a free closed beta. Yesterday, they added that Copilot would now be available to everyone, but at a cost of $10 per month per user. Copilot is trained on all public GitHub repos, regardless of copyright, and various other data scraped from the Web (similar to Eleuther's Pile). Hence, GitHub is effectively using the work others made - for personal or non-commercial use, without having GitHub in mind, and without any way to say 'no' - to sell a product back to them, for their own profit. Many people are mad about this. I... (read 1616 more words →)

Replying toLet's See You Write That Corrigibility Tag

alyssavance4y*

Eliezer's writeup on corrigibility has now been published (the posts below by "Iarwain", embedded within his new story Mad Investor Chaos). Although, you might not want to look at it if you're still writing your own version and don't want to be anchored by his ideas.

https://www.projectlawful.com/posts/6075?page=21

Humans are very reliable agents

This post has been recorded as part of the LessWrong Curated Podcast, and an be listened to on Spotify, Apple Podcasts, and Libsyn.

Over the last few years, deep-learning-based AI has progressed extremely rapidly in fields like natural language processing and image generation. However, self-driving cars seem stuck in perpetual beta mode, and aggressive predictions there have repeatedly been disappointing. Google's self-driving project started four years before AlexNet kicked off the deep learning revolution, and it still isn't deployed at large scale, thirteen years later. Why are these fields getting such different results?

Right now, I think the biggest answer is that ML benchmarks judge models by average-case performance, while self-driving cars (and many... (read 723 more words →)

270

Crazy idea: LessWrong and EA have been really successful in forming student groups at elite universities. But in the US, elite university admissions select on some cool traits (eg. IQ, conscientiousness), don't select on others, and anti-select on some (eg. selection against non-conformists). To find capable people who didn't get into an elite school, what if someone offered moderate cash bounties (say, $1,000-$5,000 range) to anyone who could solve some hard problem (eg. an IMO gold medal problem, or something like https://microcorruption.com/), without already being a "recognized expert" (say, under age 25, not at an elite school, not already working in the field, etc.). This would be similar to the old "Quixey Challenge" (https://venturebeat.com/2012/06/09/quixey-gamifies-job-hunting-in-an-entirely-new-way/), where coders were offered $100 to find a bug in one minute, but at a slightly broader scale and for different skills. This would select from a broader range of people, and could be adapted to whatever types of skills are most in demand, eg. hacking challenges for trying to recruit computer security people (https://forum.effectivealtruism.org/posts/ZJiCfwTy5dC4CoxqA/information-security-careers-for-gcr-reduction).

alyssavance's Shortform

A Quick Note on AI Scaling Asymptotes

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Visible Homelessness in SF: A Quick Breakdown of Causes

Two months ago, DeepMind announced their new Chinchilla paper. The headline result was that you could get much better performance with the same computational budget, by training a smaller model for longer:

However, it seems to have been mostly unnoticed that the "scaling law" they found was generally different from the first one in Kaplan et al. Kaplan shows a "pure" power law:

while the law in the Chinchilla paper has a large constant factor and substantially larger overall exponent. Note that N here (parameters) and D (data) are both roughly proportional to the square root of C (compute):

This results in similar behavior within a certain range, but very different asymptotic behavior:

(Graph produced by... (read more)

Is AI Progress Impossible To Predict?

Anyone who's visited downtown San Francisco has seen streets like this one:

The US is the richest country on Earth, excluding small states like Singapore with <10 million people. And the Bay Area is one of the US's economic hubs. Why do we see stuff like this? I'll try to break it down quickly.

First, San Francisco is obviously an expensive place to live. Do housing costs predict high homelessness?

They do! R^2 = 0.69 is really good. (This is a regression on US states, data from here and here. Washington, DC is a high outlier; removing it raises R^2 to 0.73. High outliers include DC, Washington, Nevada, Alaska and New Mexico; low outliers include... (read 448 more words →)

210