Wiki Contributions


Answer by ThomasJOct 23, 20223-1

Here are some attributes I've noticed among people who self-identify as rationalists. They are:

  • Overwhelmingly white and male. In the in-person or videoconference meetups I've attended, I don't think I've met more than a couple non-white people, and perhaps 10% were non-male.
  • Skew extremely young. I would estimate the median age is somewhere in the early to mid 20s. I don't think I've ever met a rat over the age of 50. I'm not saying that they don't exist, but they seem extremely underrepresented relative to the general population. 
  • Overweight the impact / power of rationalism, despite having life outcomes that are basically average for people with similar socioeconomic backgrounds and demographics
  • Tend to be more willing than average to admit that they're wrong if pressed on a factual issue, but have extreme confidence in subjective beliefs (e.g., values, philosophy, etc). This might just be a side effect of the age issue, since I think this describes most people in this age group. Or perhaps the overconfidence in subjective beliefs is just normal, but seems high relative to the willingness to switch beliefs on more factual matters.
  • Have a very high "writing and talking / doing" ratio. I think this is a selection bias kind of issue: people who are actually out doing stuff in the world probably don't have a lot of time to engage in a community that strongly values multi-page essays with a half-dozen subheadings. Although perhaps this is also just another side effect of the age skew.
  • Undervalue domain knowledge relative to first-principles thinking. As just one example, many rats will gladly outline what they believe are likely Ukraine / Russia outcomes despite not having any particular expertise in politics, international relations, or military strategy. Again, perhaps this is normal relative to the general population and it just seems unusual given rat values. 

Is this like "have the hackathon participants do manual neural architecture search and train with L1 loss"?

Ah, I misinterpreted your question. I thought you were looking for ideas for your team that was participating in the hackation, not as the organizer of the hackation. 

In my experience, most hackathons are judged qualitatively, so I wouldn't worry about ideas (mine or others') without a strong metric 

Do a literature survey for the latest techniques on detecting if a image/prose text/piece of code is computer-generated or human-generated. Apply it to a new medium (i.e. if it's an article about text, borrow techniques to apply it to images, or vice-versa). 


Alternatively, take the opposite approach and show AI safety risks. Can you train a system that looks very accurate, but gives incorrect output on specific examples that you choose during training? Just as one idea, some companies use face recognition as a key part of their security system. Imagine a face recognition system that labels 50 "employees" that are images of faces you pull from the internet, including images of Jeff Bezos. Train that system to correctly classify all the images, but also label anyone wearing a Guy Fawkes mask as Jeff Bezos. Think about how you would audit something like this if a malicious employee handed you a new set of weights and you were put in charge of determining if they should be deployed or not.

>75% confidence: No consistent strong play in simple game of imperfect information (e.g. battleship) for which it has not been specifically trained.

>50% confidence: No consistent "correct" play in a simple game of imperfect information (e.g. battleship) for which it has not been specifically train. Correct here means making only valid moves, and no useless moves. For example, in battleship a useless move would be attacking the same grid coordinate twice.

>60% confidence: Bad long-term sequence memory, particularly when combined with non-memorization tasks. For example, suppose A=1, B=2, etc. What is the sum of the characters in a given page of text (~500 words)?

Above 99% certainty: 

Run inference in reasonable latency (e.g. < 1 second for text completion) on a typical home gaming computer (i.e. one with a single high-powered GPU). 

Didn't this basically happen with LTCM? They had losses of $4B on $5B in assets and a borrow of $120B. The US government had to force coordination of the major banks to avoid blowing up the financial markets, but meltdown was avoided.

Edit: Don't pyramid schemes do this all the time, unintentionally? Like, Madoff basically did this and then suddenly (unintentionally) defaulted. 

But if I had to use the billion dollars on evil AI specifically, I'd use the billion dollars to start an AI-powered hedge fund and then deliberately engineer a global liquidity crisis.

How exactly would you do this? Lots of places market "AI powered" hedge funds, but (as someone in the finance industry) I haven't heard much about AI beyond things like regularized regression actually giving significant benefit.

Even if you eventually grew your assets to $10B, how would you engineer a global liquidity crisis?

+1, CLion is vastly superior to VsCode or emacs/vi for capabilities and ease of setup, particularly for C++ and Rust

It seems like this is a single building version of a gated community / suburb? In "idealized" America (where by idealized I mean somewhat affluent, morally homogeneous within the neighborhood, reasonably safe, etc), all the stuff you're describing already happens. Transportation for kids is provided by carpools or by the school, kids wander from house to house for meals and play, etc. Families get referrals for help (housekeeping, etc) from other families, or because there are a limited number of service providers in the area. In general, these aren't the hard things about having kids. 

In my experience, here are the hard things:

  • The early months / years are miserable. The kid wakes you up in the middle of the night and won't go back to sleep and you don't know why. You're in a constant state of sleep deprivation. This happened to me even though I had a night nanny for the first few months (which was hugely helpful, but did not completely eliminate the problem). I got off easier than my friends who had such a problem that they finally hired a "sleep coach" (yes this is a thing).
  • Your kid is sick, and you need to take care of them. You could outsource this if you had live-in help, but in practice there is a biological imperative to make you want to do the caretaking yourself.
  • Your kid has physical or mental issues. This doesn't necessarily mean anything like they're in a wheelchair or have severe learning disabilities, it could mean something like attention issues or delayed fine motor skills.
  • The kid needs almost constant supervision, particularly in the early years. Again you can outsource this to a limited extent (e.g., with daycare) but as a parent you want to spend some time with them (because if not, why have the kid at all?)
  • Even when things are going smoothly, there are significant coordination costs. Do you and your partner both need to stay late at work? Figure out who's going to pick up the child from school (and make sure the school has all the appropriate forms allowing that person to pick up), arrange childcare for the night (will you be home early enough to put your kid to bed?), etc.
  • You finally got home and you're dead tired. Unfortunately at 3am your kid wakes you up because they had a bad dream. This happens more than once per week, for various reasons. 
  • There's a trade-off between living in the best place for your work and living in the best place for your kid. Would it be better for you to live in the heart of Manhattan (or wherever) for your job and career socializing? Probably yes. Is it the best place to raise kids? Probably no. 
  • You can never again give 110%. You know those couple of weeks you had crunch period and had to work 80 hours? You can't do that anymore. No one else can actually replace you as a parent for your own kid. Or rather you can, but you have to be aware that you're now actively putting on the trade of "sell relationship with child." 
Load More