From Newcastle, Australia to Berkeley, San Francisco. I arrived yesterday for Less.online. I’ve had a bit of culture shock, a big helping of being increasingly scared, and quite a few questions. I’ll start with those. Feel free to skip them.
These questions are based on warnings I’ve gotten from local non-rationalists. Idk if they’re scared because of the media they consume or because of actual stats. I’m asking these because they feel untrue.
Culture shocks for your enjoyment:
Things that won’t kill me but scare me rational/irrational:
Other things:
It feels super weird to be in the same town as a bunch of you guys now. I’ve never met a rationalist irl. I talked to Ruby over zoom once, who said to me “You know you don’t have to stay in Australia right?” I hope Ruby is a good baseline for niceness levels of you all.
If you’re going, I’ll see you at Less.Online. If you’re not, I’d still love to meet you. Feel free to DM me!
I would bet hard cash that "I’m sure I’ll be at least in the bottom 5% for intelligence at Less Online. I won’t be surprised or hurt if I’ve got the least Gs of people there" is wrong.
But okay, let's just go with your fears and assume for sake of argument that you're right... Last year, Aella did a live polling event at Manifest where she asked us to, in general, first predict how we relatively compared to the crowd on any N, then define actual buckets of values and arrange ourselves by those buckets.
(So, e.g., "predict how recently have you exercised compared to others; the most recent, all the way on the left, the least recent, all the way on the right" and then "okay, let's define from that fence to that chair as 'in the last hour', the chair to the steps as 'in the last day', etc." and have folks line up twice based on first their relative guess, then their actual value.)
I was overall deeply surprised when she had folks line up under that system by SAT scores and IQ scores. It was, generally, extremely poorly correlated with how awesome it was to get to talk with a given person.
This post already expresses a great deal of the vibes that make me think it will be awesome to meet you, and that you get a lot of the vibes folks are aiming for, as far as I can tell. Looking forward to meeting ya!
Re safety, I don't know about Oakland but some parts of SF are genuinely the most dangerous feeling places I've ever been to after dark (because normally I wouldn't go somewhere, but SF feels very fine until it isn't). If I am travelling to places in SF after dark I'll check how dodgy the street entrances are.
Have fun! I won't be going. Some random notes:
- you should be worried someone convinces you to move to the bay. it's not worth it. like, literally entirely for cost of housing reasons, no other reason, everything else is great, there's a reason people are there anyway. but phew, the niceness comes with a honkin price tag. and no, living in a 10ft by 10ft room to get vaguely normal sounding rent is not a good idea, even though it's possible.
Why's this not a good idea? 10ft by 10ft is a lot of room. More than I had in some flats when I went to university.
in fact, never use an inhaled or injected recreational drug, period - the fast uptake is extremely dangerous and will likely actually knock your motivation system off balance hard enough to probably ruin your life.
I don't think this can be remotely justified by the evidence, formal or anecdotal. Inhaling weed isn't dangerous, let alone extremely so, and will almost certainly not ruin anyone's life, as the hundreds of millions of happy users can attest (get yours today!) Hell, shisha is an inhaled recreational drug!
I'm not sure it makes sense to generalise about an entire method of delivery, when all sorts of substances with very different effects can be consumed that way.
That first point made me laugh. It’s exactly the type of mistake I expected to make, and I still didn’t see it coming.
I appreciate all this safety advice and will update my decision making based on that.
Geez, the weed thing surprises me. I hadn’t planned to smoke any until after the event. But I think I’ll avoid that now. I’m already struggling with motivation from jet lag. I don’t want to increase that feeling.
The weed thing is not true. It can sap your motivation acutely, and perhaps even have a more sustained (if definitely temporary) effect. But it certainly doesn't ruin your life by instantly "knocking your motivational system off balance".
It's a relatively chemically safe drug, but is easily habit forming and knocks you out of a productive space if used more than once every 3 to 6 months, imo. your reasoning seems reasonable. have fun with the trip!
I think go ahead and ask people to sign things. I've done it before and it went great, and the resulting book is a great memento. You've got a good conversation starter right there with asking them their favourite sequences post.
Welcome to the US!
Re: safety, it depends on exactly where you are, your skill in assessing strangers' intentions from a distance, and probably the way you carry yourself.
Speaking of which, I'd be interested in playing some improv games with you at less.online, if you want to do that!
Hmmm, I think I’m mostly bad at those things. I’ll play it safe.
And thanks for the good idea! I’ve added a session at 3pm on the Sunday.
I’m messing around with a post about this. However:
And thanks for checking in. That’s very kind of you
Are there gangs walking around Oakland looking to stab people?
Please don't ask on Manifold, you might incentivize creating one.
Note to self, write a post about the novel akrasia solutions I thought up before becoming a rationalist.
Have you tried whiteboarding-related techniques?
I think that suddenly starting to using written media (even journals), in an environment without much or any guidance, is like pressing too hard on the gas; you're gaining incredible power and going from zero to one on things faster than you ever have before.
Depending on their environment and what they're interested in starting out, some people might learn (or be shown) how to steer quickly, whereas others might accumulate/scaffold really lopsided optimization power and crash and burn (e.g. getting involved in tons of stuff at once that upon reflection was way too much for someone just starting out).
This seems incredibly interesting to me. Googling “White-boarding techniques” only gives me results about digitally shared idea spaces. Is this what you’re referring to? I’d love to hear more on this topic.
I was recently ease-dropping on a loud phone conversation. A man, maybe late 40s? Had been cheated on. He was angry. He was arguing with his girlfriend.
I noticed that all he argued the way people argue in movies. “Put yourself in my shoes”, “No, you listen to me!”, “I thought you would be the one!”. It hurt to listen to someone in pain.
A few weeks later, I was in a heated discussion with some Non-Rationalist friends. Long story short, I failed to output correct emotional signals, which was making them upset. I noticed the same thing in them. Copying the way people argue on social media. It felt ‘Therapy Culture’ for lack of a better descriptor.
But then I noticed myself. Do you remember when Sam Altman went on world tour? Doing tens of interviews. I watched every one of those. And that is now how I speak in arguments. I think this sounds like a bad thing. Because of what we now know about Sam. But I personally trusted him when he spoke back then.
The moral is, you are what you consume. Be careful.
Thought: Confidently saying “(X) has no Manhattan Project”. Is forgetting how secret the Manhattan Project was.
Which was not terribly secret. The details of the Project were indeed super-secret, to the point where most of the politicians hadn't known anything, but despite the massive global-scale censorship & secrecy, many had observed the signs of a major project of some sort and some got as far as a nuclear bomb specifically. Also, commercial satellites with meter resolution did not exist which could quantify major facilities or new cities like Los Alamos or Hanford (but overflights, and then satellites, now exist and have helped reveal later top-secret nuclear bomb programs). An AI Manhattan Project, as we currently think of it, would be amusingly similar in footprint (eg. energy consumption) to the original and often observable from space: all those gigawatts have to go somewhere, after all.* I'm sure you can find plenty more about secrecy breaches in Rhodes.
This was not necessarily all that useful in the context of WWII - of course America had some big secret projects going, everyone did. It was a total world war. Everyone was aware there was a war on. The devil was in the details of what the program was - a failure like the V2-s, or a success like Enigma decrypts and Manhattan? But a binary exists/does-not-exist is useful in a peacetime context and the current discussion.
(If nothing else, the fact that DeepSeek keeps publishing is a signal. I would note here BTW that you cannot argue, without tying yourself into some pretzel knots explaining 4-D chess logic, that Chinese AI is about to catch up to and surpass the West because the best Chinese AI group, DeepSeek, just released a model or published this-or-that revealing the secrets of OA, and argue that there is already a secret all-out Chinese Manhattan Project going on which will potentially reach AGI first - because the first thing the latter would have done is stop the former from publishing anything which might help Western AI and then devour it for researchers.)
* A wag on Twitter has pointed out that the total energy/heat output of something like a GPT-4 or GPT-5 training run is the same as or larger than the output of a Hiroshima/Nagasaki-scale nuclear bomb explosion. Which is helpful intuition for why your datacenters need so much cooling, at least.
Before the internet?
I think the idea is that there would be clues, particularly talented engineers not doing public work. Of course, I don't know who's carefully tracking that for Chinese scientists.
Scientists did guess the Manhattan project existed because several top physicists had ceased publishing. I don't know if that made it to the relevant governments. That would be vastly easier to do with the internet - if anyone is bothering.
A potentially good way to avoid low level criminals scamming your family and friends with a clone of your voice is to set a password that you each must exchange.
An extra layer of security might be to make the password offensive, an info hazard, or politically sensitive. Doing this, criminals with little technical expertise will have a harder time bypassing corporate language filters.
Good luck getting the voice model to parrot a basic meth recipe!
Good luck getting the voice model to parrot a basic meth recipe!
This is not particularly useful, plenty of voice models will happily parrot absolutely anything. The important part is not letting your phrase get out; there's work out there on designs for protocols for how to exchange sentences in a way that guarantees no leakage even if someone overhears.
Hmm. I don't doubt that targeted voice-mimicking scams exist (or will soon). I don't think memorable, reused passwords are likely to work well enough to foil them. Between forgetting (on the sender or receiver end), claimed ignorance ("Mom, I'm in jail and really need money, and I'm freaking out! No, I don't remember what we said the password would be"), and general social hurdles ("that's a weird thing to want"), I don't think it'll catch on.
Instead, I'd look to context-dependent auth (looking for more confidence when the ask is scammer-adjacent), challenge-response (remember our summer in Fiji?), 2FA (let me call the court to provide the bail), or just much more context (5 minutes of casual conversation with a friend or relative is likely hard to really fake, even if the voice is close).
But really, I recommend security mindset and understanding of authorization levels, even if authentication isn't the main worry. Most friends, even close ones, shouldn't be allowed to ask you to mail $500 in gift cards to a random address, even if they prove they are really themselves.
I now realize that my thinking may have been particularly brutal, and I may have skipped inferential steps.
To clarify, If someone didn't know, or was reluctant to repeat a password, I would end contact or request an in person meeting.
But to further clarify, that does not make your points invalid. I think it makes them stronger. If something is weird and risky, good luck convincing people to do it.
If I had sufficient funds. I would consider if it would be beneficial to invade a few subreddits, and offer $1000 to whoever can make the most viral meme that subtly teaches basic concepts of AI Doom.
This thought stems from a comment on “The Best Lay Argument is not a Simple English Yud Essay”. I have more thoughts, but not much time. If my reasoning is unclear I apologise.
Idk if people around the world have those cardboard “Recycle your phone” boxes. That take phones and recycle their components. But I’ve rummaged through a certain one ~5 different times and found 2 completely functional and reset iPhones.
Surely there’s something better we can do with these than disassembling them?
I currently am completing psychological studies for credit in my university psych course. The entire time, all I can think is “I wonder if that detail is the one they’re using to trick me with?”
I wonder how this impacts results. I can’t imagine being in a heightened state of looking out for deception has no impact.
I recently saw a tweet that asked GPT, what micro skills a person should master. I asked Claude a similar question, though had preloaded the conversation with “things a modern day Aristotle would teach a 24yo student”.
I think this is probably a valuable exercise to undertake yourself. I’m providing my prompt, and Claude’s response as an ‘appetiser’ to get you interested in this task.
—
Prompt Wonderful. Now Let's think less utilized by society. What are some more niche micro skills that you would want a person who you are tutoring to have. And this is given that you want them to be as successful as possible.
—
Claude Let me think deeply about valuable micro skills that are often overlooked but can create significant advantages in life...
Would you like me to elaborate on any of these? I find them fascinating because they're rarely taught explicitly yet can be transformative when mastered. I'm curious which ones resonate most with you?
These skills compound over time and create second-order advantages that aren't immediately obvious to others. They're also relatively timeless - they worked in ancient Athens and still work today.
I'm curious why you opted for Aristotle (albeit "modern") as the prompt pre-load? Most of those responses seem not directly tethered to Aristotelian concepts/books or even what he directly posits as being the most important skills and faculties of human cognition. For example, cold reading, I don't recall anything of the sort anywhere in any Aristotle I've read.
While we're not sure Aristotle himself designed the layout of the corpus, we do know that in the Nicomachean Ethics lists the faculties of "whereby the soul attains Truth":
Techne (Τεχνε) - which refers to conventional ways of achieving goals, i.e. without deliberation
Episteme (Επιστήμε) - which is apodeiktike or the faculty of arguing from proofs
Phronesis (Φρονέσις) - confusingly translated as "practical wisdom" this refers to the ability to deliberate to attain goals by means of deliberation. Excellence in phronesis is translated by the latinate word 'Prudence'.
Sofia (Σοφια) - often translated as 'wisdom' - Aristotle calls this the investigation of causes.
Nous (Νους ) - which refers to the archai - or the 'first principles'
According to Diogenes Laertius, the corpus (at least as it has come to us) divides into the practical books and the theoretical - the practical itself would be subdivided between the books on Techne (say Rhetoric and Poetics), and Phronesis (Ethics and Politics), the theoretical is then covered in works like the Metaphysics (which is probably not even a cohesive book, but a hodge-podge), Categories etc. etc.
This would appear to me to be a better guide for the timeless education in Aristotelian tradition and how we should guide a modern adaptation.
I asked for further details on the 10th point and Claude listed a bunch of stuff I’ve absolutely never heard of. I’d say it’s probably related to meditation if I had to guess. Here’s that.
—
Claude Let me break down Time Perception Management into its deeper components, as this is one of the most subtle yet powerful micro-skills...
This skill set is particularly valuable because most people are unconscious of how their time perception affects their decisions and experiences. Mastering it gives you significant advantages in:
Would you like me to elaborate on any particular aspect? Or shall we explore how this connects with some of the other micro-skills we discussed?
I’ve seen a lot about GPT4o being kinda bad, and I’ve experienced that myself. This surprises me.
Now I will say something that feels like a silly idea. Is it possible that having the audio/visual part of the network cut off results in 4o’s poor reasoning? As in, the whole model is doing some sort of audio/visual reasoning. But we don’t have the whole model, so it can’t reason in the way it was trained to.
If that is the case, I’d expect that when those parts are publicly released, scores on benchmarks shoot up?
Do people smarter and more informed than me have predictions about this?
Without a detailed Model Card for 4o it is impossible to know " for sure" why models drift in performance over time, but drift they do.
It is entirely possible that Open-AI started with a version of GPT-4 Turbo, parallelize processing and performed an extensive "fine tune" to improve the multi-modal capabilities.
Essentially, the model could "forget" how to complete prompuppies. Workhfrom just a week ago, because some of its "memory" was over-written with instructions to complete requests for multi-modal replies.
I'm confused by what you mean that GPT-4o is bad? In my experience it has been stronger than plain GPT-4, especially at more complex stuff. I do physics research and it's the first model that can actually improve the computational efficiency of parts of my code that implement physical models. It has also become more useful for discussing my research, in the sense that it dives deeper into specialized topics, while the previous GPT-4 would just respond in a very handwavy way.
Man, I wish that was my experience. I feel like I’m constantly asking GPT4o a question, getting a weird or bad response. Then switching to 4 to finish the job.
Benchmarks are consistent with GPT-4o having different strengths than GPT4-Turbo, though at a similar overall level - EQ-Bench is lower, MAGI-Hard is higher, best tested model for Creative Writing according to Claude Opus, but notably worse at judging writing (though still good for its price point).
In my experience different strengths also mean different prompt strategies are necessary; a small highly instruction-focused model might benefit from few-shot repetition and emphasis that just distract a more powerful OpenAI model for example. Which might make universal custom instructions more annoying.