Shortform Content [Beta]
Here's a mistake which I've sometimes committed and gotten defensive as a result, and which I've seen make other people defensive when they've committed the same mistake.
Take some vaguely defined, multidimensional thing that people could do or not do. In my case it was something like "trying to understand other people".
Now there are different ways in which you can try to understand other people. For me, if someone opened up and told me of their experiences, I would put a lot of effort into really trying to understand their per... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
I just noticed that I've got two similarity clusters in my mind that keep getting called to my attention by wording dichotomies like high-priority and low-priority, but that would themselves be better labeled as big and small. This was causing me to interpret phrases like "doing a string of low-priority tasks" as having a positive affect (!) because what it called to mind was my own activity of doing a string of small, on-average medium-priority tasks.
My thought process might improve overall if I toss out the "big" and "small&... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
Meta/UI:
I currently believe it was a mistake to add the "unread green left-border" to posts and comments in the Recent Discussion section – it mostly makes me click a bunch of things to remove the green that I didn't really want to mark as read. Curious if anyone has opinions about that.
I find it very useful for telling whether comments are new. I’ve not been using it as an inbox (no clicking in order to make green go away).
Have some horrible jargon: I spit out a question or topic and ask you for your NeMRIT, your Next Most Relevant Interesting Take.
Either give your thoughts about the idea I presented as you understand it, unless that's boring, then give thoughts that interests you that seem conceptually closest to the idea I brought up.
Kevin Zollman at CMU looks like he's done a decent amount of research on group epistemology. I plan to read the deets at some point, here's a link if anyone wanted to do it first and post something about it.
Growth, Novelty, and Preservation
Note: This was heavily inspired by toonalfrink's recent shortform post
There's this Farnam Street blog post that says "if you can get 1 percent better each day for one year, you’ll end up thirty-seven times better by the time you’re done"
This always seemed a bit unrealistic to me, but until recently I wasn't able to pinpoint why. I think digging into this is important because it's easy to look back 1 year, see how much you've grown, and then extrapolate to your future and thin... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
think we all recognize that this is a bit of an exaggeration
No, this is mathematically true. A strict 1% improvement over 365 consecutive cycles is 3778% improvement. Compound interest is really that powerful. No exaggeration there.
It's misleading, though. The model doesn't apply to most human improvement. It's almost impossible to improve any metric by 1% in a day, almost impossible to avoid negative growth sometimes, certainly impossible (for any real human) to maintain a rate of improvement - declining marginal return for interven... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
In reasoning about AGI, we're all aware of the problems with anthropomorphizing, but it occurs to me that there's also a cluster of bad reasoning that comes from an (almost?) opposite direction, where you visualize an AGI to be a mechanical automaton and draw naive conclusions based on that.
For instance, every now and then I've heard someone from this community say something like:
What if the AGI runs on the ZFC axioms (among other things), and finds a contradiction, and by the principle of explosion it goes completely haywire?
Even if ZFC is inconsisten
... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this postUpdated the Prediced AI alignment event/meeting calendar.
Main change: Deadline for SafeAI workshop corrected to November from December.
I've been thinking about Rousseau and his conception of freedom again because I'm not sure I hit the nail on the head last time. The most typical definition of freedom and that championed by libertarians focuses on an individual's ability to make choices in their daily life. On the more libertarian end, the government is seen as an oppressor and a force of external compulsion.
On the other hand, Rousseau's view focuses on "the people" and their freedom to choose the kind of society that they want to live in. Instead of being se... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
I haven't looked that much into French history, just think it is important to acknowledge where that line of thought can end up.
Branding: 3 reasons why I prefer "AGI safety" to "AI alignment"
-
When engineers, politicians, bureaucrats, military leaders, etc. hear the word "safety", they suddenly perk up and start nodding and smiling. Safety engineering—making sure that systems robustly do what you want them to do—is something that people across society can relate to and appreciate. By contrast, when people hear the term "AI alignment" for the first time, they just don't know what it means or how to contextualize it.
-
There are a lot of things that people are working on in this spa
Can you think of other explanations?
There's a difference between people talking about safety in the sense of 1. 'how to handle a firearm safely' and the sense of 2. 'firearms are dangerous, let's ban all guns'. These leaders may understand/be on board with 1, but disagree with 2.
[Cross-posted from Medium, written for a pretty general audience]
There are many words that could describe my political positions. But there's one fundamental label for me: I am a consequentialist.
Consequentialism is a term from ethics; there, it means the position that consequences are what truly make an action right or wrong, rather than rules or virtues. What that means is that for me, the most essential questions about policy aren't things like "what is fair" or "what rights do people have", although these are good question... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
On politics, virtue ethics might say: "try to have leaders that are good"*, "accepting bribes is wrong", and perhaps "seek peace and shared ground rather than division and fear." (Working towards peace seems more virtuous than fear mongering.)
*and if they're not good, try and change that - gradual progress is better than no progress at all.
Mini Post, Litany of Gendlin related.
Changing your mind feels like changing the world. If I change my mind and now think the world is a shittier place than I used to (all my friends do hate me), it feels like I just teleported into a shittier world. If I change my mind and now think the world is a better place than I used to (I didn't leave the oven on at home, so my house isn't going to burn down!) it feels like I've just been teleported into a better world.
Consequence of the above: if someone is trying to change your mind, it feels like t... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
Looking for feedback on how useful you think this explanation
The nature of this experience may vary between people. I'd say finding out something bad and having to deal with the impact of that is more common/of an issue than rejecting the way things are (or might be), though:
extra advice would be to a non rat going through a Gendlin style crisis)
Offhandedly I'm not sure "rat" makes an effect here?
1. Figuring out what to do with new troubling information - making a plan and acting on it - can be hard. (Knowing what to do might help peop... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
Game theory is widely considered the correct description of rational behavior in multi-agent scenarios. However, real world agents have to learn, whereas game theory assumes perfect knowledge, which can be only achieved in the limit at best. Bridging this gap requires using multi-agent learning theory to justify game theory, a problem that is mostly open (but some results exist). In particular, we would like to prove that learning agents converge to game theoretic solutions such as Nash equilibria (putting superrationality aside: I think that superrational
... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this postIn the previous "population game" setting, we assumed all players are "born" at the same time and learn synchronously, so that they always play against players of the same "age" (history length). Instead, we can consider a "mortal population game" setting where each player has a probability to die on every round, and new players are born to replenish the dead. So, if the size of the population is (we always consider the "thermodynamic" limit), players die and the same number of players are born on every round. Each player's utility functio
... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this postA thought about productivity systems/workflow optimization:
One principle of good design is "make the thing you want people to do, the easy thing to do". However, this idea is susceptible to the following form of Goodhart: often a lot of the value in some desirable action comes from the things that make it difficult.
For instance, sometimes I decide to migrate some notes from one note-taking system to another. This is usually extremely useful, because it forces me to review the notes and think about how they relate to each other and to the new system. If I m
... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this postHere's a faulty psychological pattern that I recently resolved for myself. It's a big one.
I want to grow. So I seek out novelty. Try new things. For example I might buy high-lumen light bulbs to increase my mood. So I buy them, feel somewhat better, celebrate the win and move on.
Problem is, I've bought high-lumen bulbs three times in my life now already, yet I sit here without any. So this pattern might happen all over again: I feel like upgrading my life, get this nice idea of buying light bulbs, buy them, celebrate my win and move on.
So he... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
Moved to a new country twice, they broke once.
But the real cause is that I didnt regard these items as my standard inventory, which I would have done if I had more of a preservation mindset.
Decision-theoretic blackmail is when X gets Y to choose A over B, not via acting to make the consequences of A more appealing to Y, but by making the consequences of B less appealing to Y.
The exceptions to this definition are pretty massive, though, and I don't know a principled emendation that excludes them.
1. There's a contract / social contract / decision-theoretic equilibrium, and within that, B will be punished. (This may not be a true counterexample, because the true choice is whether to join the contract... though this is less clear for th... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
One of the central challenges in Dialogic Reinforcement Learning is dealing with fickle users, i.e. the user changing eir mind in illegible ways that cannot necessarily be modeled as, say, Bayesian updating. To take this into account, we cannot use the naive notion of subjective regret bound, since the user doesn't have a well-defined prior. I propose to solve this by extending the notion of dynamically inconsistent preferences to dynamically inconsistent beliefs. We think of the system as a game, where every action-observation history corresponds
... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this postThere is a deficiency in this "dynamically subjective" regret bound (also can be called "realizable misalignment" bound) as a candidate formalization of alignment. It is not robust to scaling down. If the AI's prior allows it to accurately model the user's beliefs (realizability assumption), then the criterion seems correct. But, imagine that the user's beliefs are too complex and an accurate model is not possible. Then the realizability assumption is violated and the regret bound guarantees nothing. More precisely, the AI may use incomplete models to capt
... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this postTotally an experiment, I'm trying out posting my raw notes from a personal review / theorizing session, in my short form. I'd be glad to hear people's thoughts.
This is written for me, straight out of my personal Roam repository. The formatting is a little messed up because LessWrong's bullet don't support indefinite levels of nesting.
This one is about Urge-y-ness / reactivity / compulsiveness
- I don't know if I'm naming this right. I think I might be lumping categories together.
- Let's start with what I know:
- There are th
New post: Some musings about exercise and time discount rates
[Epistemic status: a half-thought, which I started on earlier today, and which might or might not be a full thought by the time I finish writing this post.]
I’ve long counted exercise as an important component of my overall productivity and functionality. But over the past months my exercise habit has slipped some, without apparent detriment to my focus or productivity. But this week, after coming back from a workshop, my focus and productivity haven’t really booted up.
Her... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
Well, my working is often pretty varied, while my "being distracted" is pretty monotonous (watching youtube clips), so I don't think it is this one.
New (short) post: Desires vs. Reflexes
[Epistemic status: a quick thought that I had a minute ago.]
There are goals / desires (I want to have sex, I want to stop working, I want to eat ice cream) and there are reflexes (anger, “wasted motions”, complaining about a problem, etc.).
If you try and squash goals / desires, they will often (not always?) resurface around the side, or find some way to get met. (Why not always? What are the difference between those that do and those that don’t?) You need to bargain with them, or design outlet poli... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post