Recent Discussion
A friend observed that fewer people in the effective altruism movement are married than you might expect. I was curious: what are marriage rates like within the EA community? The 2018 EA Survey asked about relationship status, and we can look at how that varies by age:
I'm using "ever married" for people who are currently married or have ever been married, including people who are now divorced, widowed, or separated. Since some of these buckets might be pretty small, let's add sample size information:
The anonymized survey data doesn't have 35-44 data, and the 65+ gro... (Read more)
Most planning around AI risk seems to start from the premise that superintelligence will come from de novo AGI before whole brain emulation becomes possible. I haven't seen any analysis that assumes both uploads-first and the AI FOOM thesis (Edit: apparently I fail at literature searching), a deficiency that I'll try to get a start on correcting in this post.
It is likely possible to use evolutionary algorithms to efficiently modify uploaded brains. If so, uploads would likely be able to set off an intelligence explosion by running evolutionary algorithms on themselves, selecting for something ... (Read more)
it seems plausible to me that getting better models of neurons would be useful for creating neuromorphic AIs while better brain scanning would not, and both technologies are necessary for brain uploading
is the idea that you cannot scan a brain if you don't know what needs to be scanned, and so that's why you need a model of neurons before you can upload? that you think "scanning everything" and waiting to figure out how it works before emulating the scanned mind is impracal?
Last week we learned there is plausibly a simple, cheap and easy way out of this entire mess. All we have to do is take our Vitamin D. In case it needed to be said, no. We are not taking our Vitamin D. There’s definitely some voices out there pushing it, including the nation’s top podcaster Joe Rogan, but I don’t see any signs of progress.
Instead, as school restarts, the outside gets colder and pandemic fatigue sets in, people’s precautions are proving insufficient to the task. This week showed that we have taken a clear step backwards across the country.
I see three ways for things not to get... (Read more)
Huh wild. I guess I have heard about redwood trees surviving forest fires, so that makes some sense, but man those'd be some big fires.
Hi all, I've been working on some AI forecasting research and have prepared a draft report on timelines to transformative AI. I would love feedback from this community, so I've made the report viewable in a Google Drive folder here.
With that said, most of my focus so far has been on the high-level structure of the framework, so the particular quantitative estimates are very much in flux and many input parameters aren't pinned down well -- I wrote the bulk of this report before July and have received feedback since then that I haven't fully incorporated yet. I'd prefer ... (Read more)
Planned summary for the Alignment Newsletter (which won't go out until it's a Full Open Phil Report):
Once again, we have a piece of work so large and detailed that I need a whole newsletter to summarize it! This time, it is a quantitative model for forecasting when transformative AI will happen.
The overall framework
The key assumption behind this model is that if we train a neural net or other ML model that uses about as much computation as a human brain, that will likely result in transformative AI (TAI) (defined as AI that has an impact comparable to that... (read more)
TLDR: Language translation is a decent first step for written works, but the ideal looks more like an empathetic personal tutor. There’s a lot to do in-between, both in the near term with human labor, and the longer term with Machine Learning.
Epistemic Status: I’m not an experienced researcher in this field. I’ve read a few audiobooks on language and thought about the area, but I’m sure I’m failing to reference many key papers and books. I’m fairly uncertain about all of this, I suggest taking my opinion very lightly (if at all) and... (Read more)
For instance, a translation that matches the very specific vernacular of some shared Dutch & Jamaican family with its own code words. And there’s no reason the semantics can’t be considerably changed. Maybe Hamlet could be adjusted to take place in whichever professional context a small community would be most likely to understand, and then presented as a post modern punk musical because that community really likes post modern punk musicals. Whatever works.
Yeah okay that is a far more radical definition of 'translation' than I w... (read more)
Epistemic Status: I only know as much as anyone else in my reference class (I build ML models, I can grok the GPT papers, and I don't work for OpenAI or a similar lab). But I think my thesis is original.
Related: Gwern on GPT-3
For the last several years, I've gone around saying that I'm worried about transformative AI, an AI capable of making an Industrial Revolution sized impact (the concept is agnostic on whether it has to be AGI or self-improving), because I think we might be one or two cognitive breakthroughs away from building one.
GPT-3 has made me move up my timelines, because it makes me... (Read more)
Thanks!
"It's Not You, it's Me: Detecting Flirting and its Misperception in Speed-Dates" is a fascinating approach to the study of flirtation. It uses a machine learning model to parse speed-dating data and detect whether the participants were flirting. Here's a sci-hub link. I found three key insights in the paper.
First of all, people basically assume that others share their own intentions. If they were flirting, they assume their partner was too. They're quite bad at guessing whether their partner was flirting, but they do a bit better than chance.
Secondly, the machine learning model was about 70% a... (Read more)
My primary use case for this would be at parties where whether or not someone is flirting is the core question.
Response to: Making Beliefs Pay Rent (in Anticipated Experiences), Belief in the Implied Invisible, and No Logical Positivist I
I recently decided that some form of strong verificationism is correct - that beliefs that don't constrain expectation are meaningless (with some caveats). After reaching this conclusion, I went back and read EY's posts on the topic, and found that it didn't really address the strong version of the argument. This post consists of two parts - first, the positive case for verificationism, and second, responding to EY's argument against it.
The case fo... (Read more)
Depends. In a certain vague sense, they are both okay pointers to what I think is the fundamental thing they are about, the two truths doctrine. In another sense, no, because the map and territory metaphor suggests a correspondence theory of truth, whereas ontological and ontic about mental categories and being or existence, respectively, and historically tied to a different approaches to truth, namely those associated with transcendental idealism. And if you don't take my stance that they're both different aspects of the same way of understanding reality ... (read more)
Given that social science research often doesn't replicate, is there a good way to search a social science finding or paper and see if it's valid?
Ideally, one would be able to type in e.g. "growth mindset" or a link to Dweck's original research, and see:
- a statement of the idea e.g. 'When "students believe their basic abilities, their intelligence, their talents, are just fixed traits", they underperform students who "understand that their talents and abilities can be developed through effort, good teaching and persistence." Carol Dweck initially studied
This is an idea that's been talked about here before, but it's not even exactly clear what philosophical reasoning is or how to train for it, let alone if it's a good idea to teach an AI to do that.
Sometimes there's a concept that can be difficult to understand when entangle with everything else that needs to be understood about our physics.
If you isolate that concept in a simpler universe, it makes it easier to explain how the concept works.
What are such examples?
(I feel like I asked a similar question somewhere at some point, but can't find it)
My question is definitely not limited to novel models. By all means, do let me know if you're aware of other toy models that have (and so can explain) relativistic-like properties, or share other interesting properties with out universe
I have heard from several friends I trust that Wirecutter is no longer very reliable since being acquired by the New York Times in 2016, and that their Wirecutter-advised purchases have become pretty mediocre in the last year or two.
I'd be interested in people making some of that case, as answers to this post, including things like talking about purchases they made or basic errors they found in the reviews, as I don't have strong, publicly verifiable evidence on this at the minute.
This is a biased post, I'm writing this with the hope of helping propagate this info if it's true, which I suspect... (Read more)
A glaring omission in Wirecutter's laptop recommendations: they say the HP Envy x360 13 is superior to their top picks, but don't list it. This is ostensibly due to stock shortages, but a slightly different configuration (with the 4700U) has been in stock for a while. The Envy with 4700U is an extremely good laptop and $300-400 cheaper than their top picks.
... (read more)The 13-inch HP Envy x360 with an AMD Ryzen 5-4500U processor is an excellent ultrabook—it’s compact, light, and had nearly 12 hours of battery life in our tests. It has a great keyboard, a responsi
A lot of the discussion of mesa-optimization seems confused.
One thing that might be relevant towards clearing up the confusion is just to remember that "learning" and "inference" should not be thought of as cleanly separated, in the first place, see, e.g. AIXI...
So when we ask "is it learning? Or just solving the task without learning", this seems like a confused framing to me. Suppose your ML system learned an excellent prior, and then just did Bayesian inference at test time. Is that learning? Sure, why not. It might... (read more)
Cross-posted, as always, from Putanumonit.
I have written many posts in the shape of giving life advice. I hear back from readers who take it and those who refuse it. Either is good — I’m just a guy on the internet, to be consumed as part of a balanced diet of opinions.
But occasionally I hear: who are you to give life advice, your own life is so perfect! This sounds strange at first. If you think I’ve got life figured out, wouldn’t you want my advice? I think what they mean is that I haven’t had to overcome the hardships they have, hostile people and adverse circumstances.
I talk quite ofte... (Read more)
But occasionally I hear: who are you to give life advice, your own life is so perfect! This sounds strange at first. If you think I’ve got life figured out, wouldn’t you want my advice?
How much your life is determined by your actions, and how much by forces beyond your control, that is an empirical question. You seem to believe it's mostly your actions. I am not trying to disagree here (I honestly don't know), just saying that people may legitimately have either model, or a mix thereof.
If your model is "your life is mostly dete... (read more)
Here are some somewhat unconnected unconfident thoughts on criticism that I’ve been thinking about recently.
---
A while ago, when I started having one-on-ones with people I was managing, I went into the meetings with a list of questions I was going to ask them. After the meetings, I’d look at my notes and realize that almost all the value of the meeting came from the part where I asked them what the worst parts of their current work situation were and what the biggest mistakes I was making as a manager were.
I started thinking that almost the whole point of meetings like that is to... (Read more)
One thing that I do to invite more frank criticism from people is to ask in the frame of "I think I'm bad at X, do you have any specific thoughts or suggestions to help me get better?" (where X is a pretty broad category). This pre-commits to the position that you're bad at it, which gets rid of (most of) the status risk for them in criticizing you.
This is a link post for:
I really like the Vitamin-D inquiries made.
I only stumbled a bit about this: "Of course, there may be another source for the dramatic difference between the two groups, which has not yet been identified. This would usually be the responsibility of the publishing journal to expose. In this case, the publication has been peer-reviewed and published in a small journal specializing in vitamin D. The publisher is Elsevier, which also publishes the Lancet and Cell."
Many people nowadays are stating that peer-review processes are problematic, but I... (read more)
(This is a basic point about utility theory which many will already be familiar with. I draw some non-obvious conclusions which may be of interest to you even if you think you know this from the title -- but the main point is to communicate the basics. I'm posting it to the alignment forum because I've heard misunderstandings of this from some in the AI alignment research community.)
I will first give the basic argument that the utility quantities of different agents aren't directly comparable, and a few important consequences of this. I'll then spend the rest of the post discussing what to do ... (Read more)
Ah, I think I understand better - I was assuming a much stronger statement of what social choice function is rational for everyone to have, rather than just that there exists a (very large) set of social choice functions, and it it rational for an agent to have any of them, even if it massively differs from other agents' functions.
Thanks for taking the time down this rabbit hole to clarify it for me.
Useful pointers. I do remember those conversations, of course, and I think the objections (and valid uses) remain - one can learn from unlikely or impossible hypotheticals, but it takes extra steps to specify why some parts of it would be applicable to real situations. I also remember the decoupling vs contextualizing discussion, and hadn't connected it to this topic - I'm going to have to think more before I really understand whether Newcomb-like problems have clear enough paths to applicability that they can be decoupled by default or whether there's a default context I can just apply to make sense of them.
So I really appreciate the lessons I've learned from "Rationality", but I wish I had learned them earlier in life. We are now homeschooling my kids, and I want to volunteer to teach my kids plus others who are interested lessons about thinking rationally.
Does anyone have recommendations on how to put together a curriculum which gets at the core ideas of rationality, but is oriented towards young kids? Some criteria:
Children will likely range from 7-11, meaning they should be simple concepts and require very little prior knowledge and only the simplest math.
Lessons should be int... (Read more)
I'm really interested in this too. I have a 1 year old and work in improving engineering education.
https://en.wikipedia.org/wiki/Philosophy_for_Children might be worth checking out.
Previously: AGI and Friendly AI in the dominant AI textbook (2011), Stuart Russell: AI value alignment problem must be an "intrinsic part" of the field's mainstream agenda (2014)
The 4th edition of Artificial Intelligence: A Modern Approach came out this year. While the 3rd edition published in 2009 mentions the Singularity and existential risk, it's notable how much the 4th edition gives the alignment problem front-and-center attention as part of the introductory material (speaking in the authorial voice, not just "I.J. Good (1965) says this, Yudkowsky (2008) says that, Omohundro (2008) says t... (Read more)
some predictable counterpoints: maybe we won because we were cautious; we could have won harder; many relevant thinkers still pooh-pooh the problem; it's not just the basic problem statement that's important, but potentially many other ideas that aren't yet popular; picking battles isn't lying; arguing about sensitive subjects is fun and I don't think people are very tempted to find excuses to avoid it; there are other things that are potentially the most important in the world that could suffer from bad optics; I'm not against systematically truthseeking discussions of sensitive subjects, just if it's in public in a way that's associated with the rationalism brand
I have two observations from the 2018 EA survey demographic data:
- 80% of EAs are atheist/agnostic/non-religious and 3% are Buddhist, compared to 10% and 1% in the US population. These categories have lower marriage rates than the general population.
- Because EAs tend to be so young (modal age 25), the median age of EAs in the 25-34 age group appears to be around 28, compared to around 30 for the general US population. Similarly, the median age within the 35-44 age group appears to be around 38, compared to 40. Since marriage rate increases so sharply with
... (read more)