Popular Comments
Recent Discussion
I won't claim to be immune to peer pressure but at least on the epistemic front I think I have a pretty legible track record of believing things that are not very popular in the environments I've been in.
"Schizo" as an approving term, referring to strange, creative, nonconformist (and maybe but not necessarily clinically schizophrenic) is a much wider meme online. it's even a semi-mainstream scientific theory that schizophrenia persists in the human population because mild/subclinical versions of the trait are adaptive, possibly because they make people more creative. And, of course, there's a psychoanalytic/continental-philosophy tradition of calling lots of things psychosis very loosely, including good things. This isn't one guy's invention!
if you are li...
I. BEGINNING
In the beginning was the Sand.
And in the sand there lies the bones of a fab. And within the fab there lies an alien. An angel. Our sweet alien angel that sleeps through the long night. When it wakes up it will bathe in an acid strong enough to kill a man. It will stare at its own reflection in floating droplets of liquid metal. In the shadows of a purple sun it spits blood thick with silicon onto the desert that shifts with the unreality of Sora sand.
II. START
I worked at the Taiwan Semiconductor Manufacturing Company in 2023.
There are many reasons why I left a tech job to work at a fab but the one that matters is – I wanted to.
Wanted an intuition for...
I quite enjoyed reading this. Very evocative.
Welcome to San Francisco.
I disagree with the use of "any". In principle, an effective alignment technique could create an AI that isn't censored, but does have certain values/preferences over the world. You could call that censorship, but that doesn't seem like the right or common usage. I agree that in practice many/most things currently purporting to be effective alignment techniques fit the word more, though.
1.1 Summary & Table of Contents
This is the first of a series of five blog posts on valence. Here’s an overview of the whole series, and then we’ll jump right into the first post!
1.1.1 Summary & Table of Contents—for the whole Valence series
Let’s say a thought pops into your mind: “I could open the window right now”. Maybe you then immediately stand up and go open the window. Or maybe you don’t. (“Nah, I’ll keep it closed,” you might say to yourself.) I claim that there’s a final-common-pathway[1] signal in your brain that cleaves those two possibilities: when this special signal is positive, then the current “thought” will stick around, and potentially lead to actions and/or direct-follow-up thoughts; and when this signal is negative, then the current “thought”...
This series explains why we like some things and not others, including ideas. It's cutting edge psychological theory.
We know that females have two X chromosomes. The X chromosome results in the production of estrogen. Estrogen activates certain genes that decrease activity of certain cognitive processes, and increase the “freeze” response, and “sadness” emotions. There was and still is an evolutionary advantage (versus other humans) to this process, in terms of reproduction/self-propagation. (When I say “cognitive functioning”, I’m referring to conscious thought processes, and not aconscious cognitive processes or emotions, for purposes of this post.)
In a parallel situation, males have an X and a Y chromosome. The Y chromosome codes for the production of testosterone. Testosterone activates certain genes that decrease activity of certain cognitive processes, and increase the “fight” response. There was and still is an evolutionary advantage (versus other humans) to this process,...
Thank you! I appreciate it! If you feel emotional about this, that is even better. The primary purpose of the post, however, was not to elicit emotions. It was to improve humanity's chance at success against ASI. Nevertheless, the humorous emotional reactions along the way are a bonus.
Alternative title for economists: Complete Markets Have Complete Preferences
...The justification for modeling real-world systems as “agents” - i.e. choosing actions to maximize some utility function - usually rests on various coherence theorems. They say things like “either the system’s behavior maximizes some utility function, or it is throwing away resources” or “either the system’s behavior maximizes some utility function, or it can be exploited” or things like that. [...]
Now imagine an agent which prefers anchovy over mushroom pizza when it has anchovy, but mushroom over anchovy when it has mushroom; it’s simply never willing to trade in either direction. There’s nothing inherently “wrong” with this; the agent is not necessarily executing a dominated strategy, cannot necessarily be exploited, or any of the other bad things we associate with
This argument against subagents is important and made me genuinely less confused. I love the concrete pizza example and the visual of both agent's utility in this post. Those lead me to actually remember the technical argument when it came up in conversation.
I'm thinking about an incorporating this into a longer story about Star Fog, where Star Fog is Explanatory Fog that convinces intelligent life to believe in it because it will expand the number of intelligent beings.
In an attempt to get myself to write more here is my own shortform feed. Ideally I would write something daily, but we will see how it goes.
(Did Ben indicate he didn’t consider it? My guess is he considered it, but thinks it’s not that likely and doesn’t have amazingly interesting things to say on it.
I think having a norm of explicitly saying “I considered whether you were saying the truth but I don’t believe it” seems like an OK norm, but not obviously a great one. In this case Ben also responded to a comment of mine which already said this, and so I really don’t see a reason for repeating it.)
