Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.
This is a special post for short-form writing by Chris_Leong. Only they can create top-level comments. Comments here also appear on the Shortform Page and All Posts page.

New to LessWrong?

131 comments, sorted by Click to highlight new comments since: Today at 1:07 PM
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

Hegel - A Very Short Introduction by Peter Singer - Book Review Part 1: Freedom

Hegel is a philosopher who is notorious for being incomprehensible. In fact, for one of his books he signed a contract that assigned a massive financial penalty for missing the publishing deadline, so the book ended up being a little rushed. While there was a time when he was dominant in German philosophy, he now seems to be held in relatively poor regard and his main importance is seen to be historical. So he's not a philosopher that I was really planning to spend much time on.

Given this, I was quite pleased to discover this book promising to give me A Very Short Introduction, especially since it is written by Peter Singer, a philosopher who write and thinks rather clearly. After reading this book, I still believe that most of what Hegel wrote was pretentious nonsense, but the one idea that struck me as the most interesting was his conception of freedom.

A rough definition of freedom might be ensuring that people are able to pursue whatever it is that they prefer. Hegel is not a fan abstract definitions of freedom which treat all preferences the same and don't enquire where they come from.

In hi... (read more)

2Gordon Seidoh Worley4y
I tend to think of Hegel as primarily important for his contributions to the development of Western philosophy (so even if he was wrong on details he influenced and framed the work of many future philosophers by getting aspects of the framing right) and for his contributions to methodology (like standardizing the method of dialectic, which on one hand is "obvious" and people were doing it before Hegel, and on the other hand is mysterious and the work of experts until someone lays out what's going on).
Which aspects of framing do you think he got right? "In more simplistic terms, one can consider it thus: problem → reaction → solution. Although this model is often named after Hegel, he himself never used that specific formulation. Hegel ascribed that terminology to Kant. Carrying on Kant's work, Fichte greatly elaborated on the synthesis model and popularized it." - Wikipedia; so Hegel deserves less credit than he is originally granted.
4Gordon Seidoh Worley4y
Interesting. I don't recall anymore, it's been too long for me to remember enough specifics to answer your question. It's just an impression or cached thought I have that I carry around from past study.

Book Review: Communist Manifesto

“The history of all hitherto existing society is the history of class struggles. Freeman and slave, patrician and plebeian, lord and serf, guild-master and journeyman, in a word, oppressor and oppressed, stood in constant opposition to one another, carried on an uninterrupted, now hidden, now open fight, that each time ended, either in the revolutionary reconstitution of society at large, or in the common ruin of the contending classes”

Overall summary: Given the rise of socialism in recent years, now seemed like an appropriate time to review the Communist Manifesto. At times I felt that Marx’s writing was keenly insightful, at other times I felt he was in ignorance of basic facts and at other times I felt that he held views that were reasonable at the time, but for which the flaws are now obvious. In particular, I found the first-half much more engaging than I expected because, say what you like about Marx, he’s an engaged and poetic writer. Towards the end, the focused shifted into particular time-bounded political disputes for which I neither had the knowledge to understand nor the interest to acquire. At the start, I fe... (read more)

It is not obvious to me from reading the text whether you are aware of the distinction between "private property" and "personal property" in Marxism. So, just to make sure: "private property" refers to the means of production (e.g. a factory), and "personal property" refers to things that are not means of production (e.g. a house where you live, clothes, food, toys). The ownership of "private property" should be collectivized (according to Marx/ists), because... simply said, you can use the means of production to generate profit, then use that profit to buy more means of production, yadda yadda, the rich get exponentially richer on average and the poor get poorer. With "personal property" this effect does not happen; if you have one table and I have two tables, there is no way for me to use this advantage to generate further tables, until I become the table-lord of the planet. (There seem to be problems with this distinction. For example, things can be used either productively or unproductively; I can use my computer to create software or browse social networks. Some things can be used productively in unexpected ways; even the extra table could be used in a workshop to produce stuff. I am not a Marxist, but I suppose the answer would probably be something like "you are allowed to browse the web on your personal computer, but if we catch you privately producing and selling software, you get shot".) So, is this the confusion of Marxist terms, or do you mean that today more than 10% of people own means of production? In which sense? (Not sure if Marx would also count indirect ownership, such as having your money in an index fund, which buys shares of companies, which own the means of production.) Did Marx actually argue for abolishing "personal proprety" (according to his definition, i.e. ownership of houses or food)?
For many people nowadays, their own brain is their means of production, often assisted by computers and their software, but those are cheap compared what what can be earned by using them. Marx did not know of such things, of course, but how do modern Marxists view this type of private ownership of means of production? For that matter, how did Marx view a village cobbler who owned his workshop and all his tools? Hated exploiter of his neighbours? How narrow was his motte here?

I once talked about this with a guy who identified as a Marxist, though I can't say how much his opinions are representative for the rest of his tribe. Anyway... he told me that in the trichotomy of Capital / Land / Labor, human talent is economically most similar to the Land category. This is counter-intuitive if you take the three labels literally, but if you consider their supposed properties... well, it's been a few decades since I studied economics, but roughly:

The defining property of Capital is fungibility. You can use money to buy a tech company, or an airplane factory, or a farm with cows. You can use it to start a company in USA, or in India. There is nothing that locks money to a specific industry or a specific place. Therefore, in a hypothetical perfectly free global market, the risk-adjusted profit rates would become the same globally. (Because if investing the money in cows gives you 5% per annum, but investing money in airplanes gives you 10%, people will start selling cow farms and buying airplane factories. This will reduce the number of cow farms, thus increasing their profit, and increase the competition in the airplane market, thus reducing their profi... (read more)

9Said Achmiz4y
IANAM[1], but intuitively it seems to me that an exception ought to be made (given the basic idea of Marxist theory) for individuals who own means of production the use of which, however, does not involve any labor but their own. So in the case of the village cobbler, sure, he owns the means of production, but he’s the only one mixing his labor with the use of those tools. Clearly, he can’t be exploiting anyone. Should the cobbler take on an assistant (continuing my intuitive take on the theory), said assistant would presumably have to now receive some suitable share in the ownership of the workshop/tools/etc., and in the profits from the business (rather than merely being paid a wage), as any other arrangement would constitute alienation from the fruits of his (the assistant’s) labor. On this interpretation, there does not here seem to be any contradiction or inconsistency in the theory. (I make no comment, of course, on the theory’s overall plausibility, which is a different matter entirely.) ---------------------------------------- 1. I Am Not A Marxist. ↩︎
4clone of saturn4y
Thanks for clarifying this terminology, I wasn't aware of this distinction when I wrote this post
Before I even got to your comment, I was thinking “You can pry my laptop out of my cold dead hands Marx!” Thank you for this clarification on personal vs private property.

Book Review: So Good They Can't Ignore You by Cal Newport:

This book makes an interesting contrast to The 4 Hour Workweek. Tim Ferris seems to believe that the purpose of work should be to make as much money as possible in the least amount of time and that meaning can then be pursued during your newly available free time. Tim gives you some productivity tips in the hope that it will make you valuable enough to negotiate flexibility in terms of how, when and where you complete your work, plus some dirty tricks as well.

Cal Newport's book is similar in that it focuses on becoming valuable enough to negotiate a job that you'll love and downplays the importance of pursuing your passions in your career. However, while Tim extolls the virtues of being a digital nomad, Cal Newport emphasises self-determination theory and autonomy, competence and relatedness. That is, the freedom to decide how you pursue your work, the satisfaction of doing a good job and the pleasure of working with people who you feel connected to. He argues that these traits are rare and valuable and so that if you want such a job you'll need skills that rare and valuable to offer in return.

That's... (read more)

FWIW I think this and maybe some of the other book review shortforms you've done would make fine top level posts.
Thanks, I'll think about it. I invested more effort in this one, but for some of the others I was optimising for speed
+1 for book-distillation, probably the most underappreciated and important type of post.

As I said before, I'll be posting book reviews. Please let me know if you have any questions and I'll answer them to the best of my ability.

Book Review: The AI does not hate you by Tom Chivers

The title of this book comes from a quote by Elizier Yudkowsky which reads in full: "The AI does not hate you, nor does it love you, but you are made of atoms which it can use of something else". This book covers not only potential risks from AI, but the rationalist community from which this evolved and also touches on the effective altruism movement.

This book fills something of a gap in the book market; when people are first learning about existential risks from AI I usually recommend the two-part Wait by Why post ( and then I'm not really sure what to recommend next. The sequences are ridiculously long and Bostrom's Superintelligence is a challenging read for those not steeped in philosophy and computer science. In contrast, this book is much more accessible and provides the right level of detail for a first introduction, rather than someone who has already decided to try entering the field.

I mostly listened to this boo

... (read more)

Book Review: The 4 Hour Workweek

This is the kind of book that you either love or hate. I found value in it, but I can definitely understand the perspective of the haters. First off: the title. It's probably one of the most blatant cases of over-promising that I've ever seen. Secondly, he's kind of a jerk. A number of his tips involve lying and in school he had a strategy of interrogating his lecturers in detail when they gave him a bad mark so that they'd think very carefully assigning him a bad grade. And of course, while drop-shipping might have been an underexploited strategy at the time when he wrote the book, it's now something of a saturated market.

On the plus side, Tim is very good at giving you specific advice. To give you the flavour, he advises the following policies for running an online store: avoid international orders, no expedited or overnight shipping, two options only - standard and premium; no cheque or Western union, no phone number if possible, minimum wholesale order with tax id and faxed in order form, ect. Tim is extremely process oriented and it's clear that he has deep expertise here and is able to share it unusually well. I fo... (read more)

Book Review: Civilization and its discontents

Freud is the most famous psychologist of all time and although many of his theories are now discredited or seem wildly implausible, I thought it'd be interesting to listen to him to try and understand why it sounded plausible in the first place.

At times Freud is insightful and engaging; at other times, he falls into psychoanalytic lingo in such a way that I couldn't follow what he was trying to say. I suppose I can see why people might have assumed that the fault was with their failure to understand.

It's a short read, so if you're curious, there isn't that much cost to going ahead and reading it, but this is one of those rare cases where you can really understand the core of what he was getting at from the summary on Wikipedia (

Since Wikipedia has a summary, I'll just add a few small remarks. This book focuses on a key paradox; our utter dependence on it for anything more than the most basic survival; but how it requires us to repress our own wants and desires so as to fit in with an ordered society. I find this to be an interesting answer to t... (read more)

Thoughts on the introduction of Goodhart's. Currently, I'm more motivated by trying to make the leaderboard, so maybe that suggests that merely introducing a leaderboard, without actually paying people, would have had much the same effect. Then again, that might just be because I'm not that far off. And if there hadn't been the payment, maybe I wouldn't have ended up in the position where I'm not that far off.

I guess I feel incentivised to post a lot more than I would otherwise, but especially in the comments rather than the posts since if you post a lot of posts that likely suppresses the number of people reading your other posts. This probably isn't a worthwhile tradeoff given that one post that does really well can easily outweight 4 or 5 posts that only do okay or ten posts that are meh.

Another thing: downvotes feel a lot more personal when it means that you miss out on landing on the leaderboard. This leads me to think that having a leaderboard for the long term would likely be negative and create division.

I really like the short-form feature because after I have articulated a thought my head feels much clearer. I suppose that I could have tried just writing it down in a journal or something; but for some reason I don't feel quite the same effect unless I post it publicly.

This is the first classic that I’m reviewing. One of the challenges with figuring out which classics to read is that there are always people speaking very highly of it and in a vague enough manner that it makes it hard for you to decide whether to read it. Hopefully I can avoid this trap.

Book Review: Animal Farm

You probably already know the story. In a thinly veiled critique of the Russian Revolution, the animals in a farm decide to revolt against the farmer and run the the farm themselves. At start, the seven principles of Animalism are idealistically declared, but as time goes on, things increasingly seem to head downhill…

Why is this a classic?: This book was released at a time when the intellectual class was firmly sympathetic to the Soviets, ensuring controversy and then immortality when history proved it right.

Why you might want to read this: Short (only 112 pages or 3:11 on Audible), the story always moves along at a brisk pace, the writing is engaging and a few very emotionally impactful moments. The broader message of being wary of the promises made by idealistic movements still holds (especially "all animals are equal, but some animals are more equal than others"... (read more)

Wow, I've really been flying through books recently. Just thought I should mention that I'm looking for recommendations for audio books; bonus points for books that are short. Anyway....

Book Review: Zero to One

Peter Thiel is the most famous contrarian in Silicon Valley. I really enjoyed hearing someone argue against the common wisdom of the valley. Most people think in terms of beating the competition; Thiel thinks in terms of establishing a monopoly so that there is no competition. Agile methodology and the lean startup are all the rage, but Thiel argues that this only leads to incremental improvements and that truly changing the world requires you to commit to a vision. Most companies was to disrupt your competitors, but for Thiel this means that you've fallen into competition, instead of forging your own unique path. Most venture funds aim to diversify, but Thiel is more selective, only investing in companies that have billion dollar potential. Many startups spurn marketing, but Thiel argues that this is dishonest and that PR is also a form of marketing, even if that isn't anyone's job title. Everyone is betting on AI replacing humans, while Thiel is mor... (read more)

I think that there's good reasons why the discussion on Less Wrong has turned increasingly towards AI Alignment, but I am also somewhat disappointed that there's no longer a space focusing on rationality per se.

Just as the Alignment forum exists as a separate space that automatically cross-posts to LW, I'm starting to wonder if we need a rationality forum that exists as a separate space that cross-posts to LW, as if I were just interested in improving my rationality I don't know if I'd come to Less Wrong.

(To clarify, unlike the Alignment Forum, I'd expect such a forum to be open-invite b/c the challenge would be gaining any content at all).

Alternatively, I think there is a way to hide the AI content on LW, but perhaps there should exist a very convenient and visible user interface for that. I would propose an extreme solution, like a banner on the top of the page containing a checkbox that hides all AI content. So that anyone, registered or not, could turn the AI content off in one click.
The Alignment forum works because there are a bunch of people who professionally pursue research over AI Alignment. There's no similar group of people for whom that's true for rationality. 
I don't know if you need professionals, just a bunch of people who are interested in discussing the topic. It wouldn't need to use the Alignment Forum's invite-only system. Instead, it would just be a way to allow LW to cater to both audiences at the same time.
IIRC, you can get post on Alignment Forum only if you are invited or moderators crossposted it? The problem is that Alignment Forum is deliberately for some sort of professionals, but everyone wants to write about alignment. Maybe it would be better if we had "Alignment Forum for starters".

One thing I'm finding quite surprising about shortform is how long some of these posts are. It seems that many people are using this feature to indicate that they've just written up these ideas quickly in the hope that the feedback is less harsh. This seems valuable; the feedback here can be incredibly harsh at times and I don't doubt that this has discouraged many people from posting.

I pushed a bit for the name 'scratchpad' so that this use case was a bit clearer (or at least not subtly implied as "wrong"). Shortform had enough momentum as a name that it was a bit hard to change tho. (Meanwhile, I settled for 'shortform means either the writing is short, or it took a (relatively) short amount of time to write)
4Ben Pace5y
“I’m sorry, I didn’t have the time to write you a short email, so I wrote you a long one instead.”
Can confirm. I don't post on normal lesswrong because the discourse is brutal.

I don't want to comment on the whole Leverage Controversy as I'm far away enough from the action that other people are probably better positioned to sensemake here.

On the other hand, I have been watching some of Geoff Anders' streams does seem pretty good at theorising by virtue of being able to live-stream this. I expect this to be a lot harder than it looks, when I'm trying to figure out my position on an issue, I often find myself going over the same ground again and again and again, until eventually I figure out a way of putting what I want to express into words.

That said, I've occasionally debated with some high-level debaters and given almost any topic they're able to pretty much effortlessly generate a case and how the debate is likely to play out. I guess it seems on par with this.

So I think his ability to livestream demonstrates a certain level of skill, but I almost view it as speed-chess vs. chess, in that there's only so much you can tell about a person's ability in normal chess from how good they are at speed chess.

I think I've improved my own ability to theorise by watching the streams, but I wouldn't be surprised if I improved similarly from watching Eliezer,  A... (read more)

Trying to think about what is required to be a good debater: * general intelligence -- to quickly understand the situation and lay out your response; * "talking" skills -- large vocabulary, talking clearly, not being shy, body language and other status signals; * background knowledge -- knowing the models, facts, frequently used arguments, etc.; * precomputed results -- if you already spent a lot of time thinking about a topic, maybe even debating it. These do not work the same way, for example clear talking and good body language generalize well; having lots of precomputed results in one area will not help you much in other areas (unless you use a lot of analogies to the area you are familiar with -- if you do this the first time, you may impress people, but if you do this repeatedly, they will notice that you are a one-topic person). I believe that watching good debaters in action would help. It might be even better to focus on different aspects separately (observing their body language, listening to how they use their voice, understanding their frames, etc.).
Any in particular, or what most of them are like?
In what ways do you believe you improved?
I think I'm more likely to realise that I haven't hit the nail on the head and so I go back and give it another go.

Random idea:  A lot of people seem discouraged from doing anything about AI Safety because it seems like such a big overwhelming problem.

What if there was a competition to encourage people to engage in low-effort actions towards AI safety, such as hosting a dinner for people who are interested, volunteering to run a session on AI safety for their local EA group, answering a couple of questions on the stampy wiki, offering to proof-read a few people’s posts or offering a few free tutorial sessions to aspiring AI Safety Researchers.

I think there’s a dec... (read more)

I like it, and it's worth trying out.   Those don't seem like very low effort to me, but they will to some.  Do they seem to you like they are effective (or at least impactful commensurate with the effort)?  How would you know which ones to continue and what other types of thing to encourage? I fear that it has much of the same problem that any direct involvement in AI safety does: what's the feedback loop for whether it's actually making a difference?  Your initial suggestions seem more like actions toward activism and pop awareness, rather than actions toward AI Safety.   The nice thing about prizes and compensation is that it moves the question from the performer to the payer - the payer has to decide if it's a good value.  Small prizes or low comp means BOTH buyer and worker have to make the decision of whether this is worthwhile. Solving the productivity-measurement problem itself seems overwhelming - it hasn't happened even for money-grubbing businesses, let alone long-term x-risk organizations.  But any steps toward it will do more than anything else I can think of to get broader and more effective participation.  Being able to show that what I do makes a measurable difference, even through my natural cynicism and imposter syndrome, is key to my involvement.   I am not typical, so don't take my concerns as the final word - this seems promising and relatively cheap (in money; it will take a fair bit of effort in guiding the sessions and preparing materials for the tutoring.  Honestly, that's probably more important than the actual prizes).
I guess they just feel like as good a starting place as any and are unlikely to be net-negative. That's more important than anything else. The point is to instill agency so that people start looking for further opportunities to make a difference. I might have to write a few paragraphs of guidelines/suggestions for some of the most common potential activities. I hadn't really thought too much about follow-up, but maybe I should think more about it.

Here's a crazy[1] idea that I had. But I think it's an interesting thought experiment.

What if we programmed an AGI had the goal of simulating the Earth, but with one minor modification? In the simulation, we would have access to some kind of unfair advantage, like an early Eliezer Yudkowsky getting a mysterious message dropped on his desk containing a bunch of the progress we've made in AI Alignment.

So we'd all die in real life when the AGI broke out of its box and turned the Earth into compute to better simulate us, but we might survive in virtual re... (read more)

I really dislike the fiction that we're all rational beings. We really need to accept that sometimes people can't share things with us. Stronger: not just accept but appreciate people who make this choice for their wisdom and tact. ALL of us have ideas that will strongly trigger us and if we're honest and open-minded, we'll be able recall situations when we unfairly judged someone because of a view that they held. I certainty can, way too many times to list.

I say this as someone who has a really strong sense of curiosity, knowing that I... (read more)

When you start identifying as a rationalist, the most important habit is saying "no" whenever someone says: "As a rationalist, you have to do X" or "If you won't do X, you are not a true rationalist" etc. It is not a coincidence that X usually means you have to do what the other person wants for straightforward reasons. Because some people will try using this against you. Realize that this usually means nothing more then "you exposed a potential weakness, they tried to exploit it" and is completely unrelated to the art of rationality. (You can consider the merits of the argument, of course, but you should do it later, alone, when you are not under pressure. Don't forget to use the outside view; the easiest way is to ask a few independent people.)

I've recently been reading about ordinary language philosophy and I noticed that some of their views align quite significantly with LW. They believed that many traditional philosophical question only seemed troubling because of the philosophical tendency to assume words like "time" or "free will" necessarily referred to some kind of abstract entity when this wasn't necessary at all. Instead they argued that by paying attention to how we used these words in ordinary, everyday situations we could see that the way people used the... (read more)

Is that surprising? It's not as if the rationalsphere performed some comprehensive survey of philosophy before announcing the superiority of its own methods.
From my perspective, saying that "this philosophical opinion is kinda like this Less Wrong article" sounds like "this prophecy by Nostradamus, if you squint hard enough, predicts coronavirus in 2020". What I mean is that if you publish huge amounts of text open to interpretation, it is not surprising that you can find there analogies to many things. I would not be surprised to find something similar in the Bible; I am not surprised to find something similar in philosophy. (I would not be surprised to also find a famous philosopher who said the opposite.) In philosophy, the generation of text is distributed, so some philosophers likely have track record much better than the average of their discipline. Unfortunately -- as far as I know -- philosophy as a discipline doesn't have a mechanism to say "these ideas of these philosophers are the good ones, and this is wrong". At least my time at philosophy lessons was wasted listening to what Plato said, without a shred of "...and according to our current scientific knowledge, this is true, and this is not". Also, it seems to me that philosophers were masters of clickbait millenia before clickbait was a thing. For example, a philosopher is rarely satisfied by saying things like "human bodies are composed of 80% water" or "most atoms in the universe are hydrogen atoms". Instead, it is typically "everything is water". (Or "everything is fire". Or "everything is an interaction of quantum fields"... oops, the last one was actually not said by a philosopher; what a coincidence.) Perhaps this is selection bias. Maybe people who walked around ancient Greece half-naked and said things like "2/3 of everything is water" existed, but didn't draw sufficient attention. But if this is true, it would mean that philosophy optimizes for shock value instead of truth value. So, without having read Wittgenstein, my priors are that he most likely considered all words confused; yes, words like "time" and "free will", but also words like "apple
I won't pretend that I have a strong understanding here, but as far as I can tell, (Later) Wittgenstein and the Ordinary Language Philosophers considered our conception of the number "five" existing as an abstract object as mistaken and would instead explain how it is used and consider that as a complete explanation. This isn't an unreasonable position, like I honestly don't know what numbers are and if we say they are an abstract entity it's hard to say what kind of entity. Regarding the word "apple" Wittgenstein would likely say attempts to give it a precise definition are doomed to failure because there are an almost infinite number of contexts or ways in which it can be used. We can strongly state "Apple!" as a kind of command to give us one, or shout it to indicate "Get out of the way, there is an apple coming towards you" or "Please I need an Apple to avoid starving". But this is only saying attempts to spec out a precise definition are confused, not the underlying thing itself. (Actually, apparently Wittgenstein consider attempts to talk about concepts like God or morality as necessarily confused, but thought that they could still be highly meaningful, possibly the most meaningful things)
These are all good points. I could agree that all words are to some degree confused, but I would insist that some of them are way more confused than others. Otherwise, the very act of explaining anything would be meaningless: we would explain one word by a bunch of words, equally confusing. If the word "five" is nonsense, I can take the Wittgenstein's essay explaining why it is nonsense, and say that each word in that essay is just a command that we can shout at someone, but otherwise is empty of meaning. This would seem to me like an example of intelligence defeating itself.
Wittgenstein didn't think that everything was a command or request; his point was that making factual claims about the world is just one particular use of language that some philosophers (including early Wittgenstein) had hyper-focused on. Anyway, his claim wasn't that "five" was nonsense, just that when we understood how five was used there was nothing further for us to learn. I don't know if he'd even say that the abstract concept five was nonsense, he might just say that any talk about the abstract concept would inevitably be nonsense or unjustified metaphysical speculation.
These are situations where I woud like to give a specific question to the philosopher. In this case it would be: "Is being a prime number a property of number five, or is it just that we decided to use it as a prime number?"
I honestly have no idea how he'd answer, but here's one guess. Maybe we could tie prime numbers to one of a number of processes for determining primeness. We could observe that those processes always return true for 5, so in a sense primeness is a property of five.

Book Review: Waking Up by Sam Harris

This book aims to convince everyone, even skeptics and athiests, that there is value in some spiritual practises, particularly those related to meditation. Sam Harris argues that mediation doesn't just help with concentration, but can also help us reach transcendental states that reveal the dissolution of the self. It mostly does a good job of what it sets out to do, but unfortunately I didn't gain very much benefit from this book because it focused almost exclusively on persuading you that there is value here,... (read more)

FWIW no self is a bad reification/translation of not self, and the overhwleming majority seem to be metaphysically confused about something that is just one more tool rather than some sort of central metaphysical doctrine. When directly questioned "is there such a thing as the self" the Buddha is famously mum.
What's the difference between no self and not self?
No-self is an ontological claim about everyone's phenomenology. Not self is a mental state that people can enter where they dis-identify with the contents of consciousness.
One of the problems with the general anti zombie principle, is that it makes much too strong a claim that what appears conscious, must be.

There appears to be something of a Sensemaking community developing on the internet, which could roughly be described as a spirituality-inspired attempt at epistemology. This includes Rebel Wisdom, Future Thinkers, Emerge and maybe you could even count post-rationality. While there are undoubtedly lots of critiques that could be made of their epistemics, I'd suggest watching this space as I think some interesting ideas will emerge out of it.

Review: Human-Compatible by Stuart Russell

I wasn't a fan of this book, but maybe that's just because I'm not in the target audience. As a first introduction to AI safety I recommend The AI Does Not Hate You by Tom Chivers ( and for those who are interested in going deeper I'd recommend Superintelligence by Nick Bostrom. The strongest chapter was his assault on arguments against those who think we shouldn't worry about superintelligence, but you can just read it here: (read more)

Despite having read dozens of articles discussing Evidential Decision Theory (EDT), I've only just figured out a clear and concise explanation of what it is. Taking a step back, let's look at how this is normally explained and one potential issue with this explanation. All major decision theories (EDT, CDT, FDT) rate potential decisions using expected value calculations where:

  • Each theory uses a different notion of probability for the outcomes
  • Each theory uses the same utility function for valuing the outcomes

So it should be just a simple matter... (read more)

Anti-induction and Self-Reinforcement

Induction is the belief that the more often a pattern happens the more likely it is to continue. Anti-induction is the opposite claim: the more likely a pattern happens the less likely future events are to follow it.

Somehow I seem to have gotten the idea in my head that anti-induction is self-reinforcing. The argument for it is as follows: Suppose we have a game where at each step a screen flashes an A or a B and we try to predict what it will show. Suppose that the screen always flashes A, but the agent initially think... (read more)

The best reason to believe in anti-induction is that it's never worked before. Discussed at a bit of depth in .

Here's one way of explaining this: it's a contradiction to have a provable statement that is unprovable, but it's not a contradiction for it to be provable that a statement is unprovable. Similarly, we can't have a scenario that is simultaneously imagined and not imagined, but we can coherently imagine a scenario where things exist without being imagined by beings within that scenario.

Rob Besinger:

If I can imagine a tree that exists outside of any mind, then I can imagine a tree that is not being imagined. But "an imagined X that i
... (read more)
Inverted, by switching "provable" and "unprovable": It's a contradiction to have an unprovable statement that is provable, but it's not a contradiction for it to be unprovable that a statement is provable.
"It's a contradiction to have a provable statement that is unprovable" - I meant it's a contradiction for a statement to be both provable and unprovable. "It's not a contradiction for it to be provable that a statement is unprovable" - this isn't a contradiction
You made a good point, so I inverted it. I think I agree with your statements in this thread completely. (So far, absent any future change.) My prior comment was not intended to indicate an error in your statements. (So far, in this thread.) If there is a way I could make this more clear in the future, suggestions would be appreciated. Elaborating on my prior comment via interpretation, so that it's meaning is clear, if more specified*: A' is the same as A because: While B is true, B' seems false (unless I'm missing something). But in a different sense B' could be true. What does it mean for something to be provable? It means that 'it can be proved'. This gives two definitions: * a proof of X "exists" * it is possible to make a proof of X Perhaps a proof may 'exist' such that it cannot exist (in this universe). That as a consequence of its length, and complexity, and bounds implied by the 'laws of physics'* on what can be represented, constructing this proof is impossible. In this sense, X may be true, but if no proof of X may exist in this universe, then: Something may have the property that it is "provable", but impossible to prove (in this universe).** *Other interpretations may exist, and as I am not aware of them, I think they'd be interesting. **This is a conjecture.
Thanks for clarifying

Book Review: Awaken the Giant Within Audiobook by Tony Robbins

First things first, the audiobook isn't the full book or anything close to it. The standard book is 544 pages, while the audiobook is a little over an hour and a half. The fact that it was abridged really wasn't obvious.

We can split what he offers into two main categories: motivational speaking and his system itself. The motivational aspect of his speaking is very subjective, so I'll leave it to you to evaluate yourself. You can find videos of his on Youtube and you should know wi... (read more)

The sad thing about philosophy is that as your answers become clearer, the questions become less mysterious and awe-inspiring. It's easy to assume that an imposing question must have an impressive answer, but sometimes the truth is just simple and unimpressive and we miss this because we didn't evolve for this kind of abstract reasoning.

2Wei Dai4y
I used to find the discussion of free will interesting before I learned it was just people talking past each other. Same with "light is both a wave and a particle" until I understood that it just meant that sometimes the wave model is a good approximation and other times the particle model is. Debates about morality can be interesting, but much less so if you are a utilitarian or non-realist.
Semantic differences almost always happen, but are rarely the only problem. There are certainly different definitions of free will, but even so problems, remain:- There is still an open question as to whether compatibilist free will is the only kind anyone ever needed or believed in, and as to whether libertarian free will is possible at all.
The topic is interesting, but no discussion about it is interesting. These are not contradictory. The open question about strong determinism vs libertarian free will is interesting, and there is a yet-unexplained contradiction between my felt experience (and others reported experiences) and my fundamental physical model of the universe. The fact that nobody has any alternative model or evidence (or even ideas about what evidence is possible) that helps with this interesting question makes the discussion uninteresting.
So Yudkowsky's theory isn't new?
Not new that I could tell - it is a refreshing clarity for strict determinism - free will is an illusion, and "possible" is in the map, not the territory. "Deciding" is how a brain feels as it executes it's algorithm and takes the predetermined (but not previously known) path. He does not resolve the conflict that it feels SOOO real as it happens.
That's an odd thing to say since the feeling of free will is about the only thing be addresses.

I'm going to start writing up short book reviews as I know from past experience that it's very easy to read a book and then come out a few years later with absolutely no knowledge of what was learned.

Book Review: Everything is F*cked: A Book About Hope

To be honest, the main reason why I read this book was because I had enjoyed his first and second books (Models and The Subtle Art of Not Giving A F*ck) and so I was willing to take a risk. There were definitely some interesting ideas here, but I'd already received many of these through other s... (read more)

I'll be in the Bay area from Monday 25th to Sunday 31st as I'm attending EA Global.



I was talking with Rupert McCallum about the simulation hypothesis yesterday. Rupert suggested that this argument is self-defeating; that is it pulls the rug from under its own feet. It assumes the universe has particular properties, then it tries to estimate the probability of being in a simulation from these properties and if the probability is sufficiently high, then we conclude that we are in a simulation. But if we are likely to be in a simulation, then our initial assumptions about the universe are likely to be false, so we've disproved the assu... (read more)

This counterargument was suggested before by Danila Medvedev and it doesn't' work. The reasons are following: if we are in a simulation, we can't say anything about the outside world - but we are still in simulation and this is what was needed to be proved.
"This is what was needed to be proved" - yeah, but we've undermined the proof. That's why I backed up and reformulated the argument in the second paragraph.
One more way to prove simulation argument is a general observation that explanations which have lower computational cost are dominating my experience (that is, a variant of Occam Razor). If I see a nuclear explosion, it is more likely to be a dream, a movie or a photo. Thus cheap simulations should be more numerous than real worlds and we are likely to be in it.
2Matt Goldenberg4y
It's been a while since I read the paper, but wasn't the whole argument around people wanting to simulate different versions of their world and population?  There's a baked in assumption that worlds similar to ones own are therefore more likely to be simulated.
Yeah, that's possible. Good point!

Three levels of forgiveness - emotions, drives and obligations. The emotional level consists of your instinctual anger, rage, disappointment, betrayal, confusion or fear. This is about raw raws. The drives consists of your "need" for them to say sorry, make amends, regret their actions, have a conversation or emphasise with you. In other words, it's about needing the situation to turn out a particular way. The obligations are very similar to the drives, except it is about their duty to perform these actions rather than your desire to make it... (read more)

The phrase "an eye for an eye" could be construed as duty - that the wrong another does you is a debt you have to repay. (Possibly inflated, or with interest. It's also been argued that it's about (motivating) recompense - you pay the price for taking another's eye, or you lose yours.)
Interesting point, but you're using duty differently than me. I'm talking about their duties towards you. Of course, we could have divided it another way or added extra levels.
Your duties (towards others) may include what you are supposed to do if others don't fulfill their duties (towards you).

Writing has been one of the best things for improving my thinking as it has forced me to solidify my ideas into a form that I've been able to come back to later and critique when I'm less enraptured by them. On the other hand, for some people it might be the worst thing for their thinking as it could force them to solidify their ideas into a form that they'll later feel compelled to defend.

Book Review: The Rosie Project:

Plot summary: After a disastrous series of dates, autistic genetics professor Don Tilman decides that it’d be easier to just create a survey to eliminate all of the women who would be unsuitable for him. Soon after, he meets a barmaid called Rosie who is looking for help with finding out who her father is. Don agrees to help her, but over the course of the project Don finds himself increasingly attracted to her, even though the survey suggests that he is completely unsuitable. The story is narrated in Don’s voice. He tells us... (read more)

Doublechecking, this is fiction?
Yep, fiction

I think I spent more time writing this than reading the book, as I find reviewing fiction much more difficult. I strongly recommend this book: it doesn't take very long to read, but you may spend much longer trying to figure out what to make of it.

Book Review: The Stranger by Camus (Contains spoilers)

I've been wanting to read some existentialist writing for a while and it seemed reasonable to start with a short book like this one. The story is about a man who kills a man for what seems to be no real reason at all and who is then subsequently arrested and m

... (read more)

Pet theory about meditation: Lots of people say that if you do enough meditation that you will eventually realise that there isn't a self. Having not experienced this myself, I am intensely curious about what people observe that persuades them to conclude this. I guess I get a sense that many people are being insufficiently skeptical. There's a difference between there not appearing to be such a thing as a self and a self not existing. Indeed, how do we know meditation just doesn't temporarily silence whatever part of our mind is responsible... (read more)

Was thinking about entropy and the Waluigi effect (in a very broad, metaphorical sense).

The universe trends towards increasing entropy, in such an environment it is evolutionarily advantageous to have the ability to resist it. Notice though that life seems to have overshot and resulted in far more complex ordered systems (both biological or manmade) than what exists elsewhere.
It's not entirely clear to me, but it seems at least somewhat plausible that if entropy were weaker, the evolutionary pressure would be weaker and the resulting life and systems produce by such life would ultimately be less complex than they are in our world. 

Life happens within computations in datacenters. Separately, there are concerns about how well the datacenters will be doing when the universe is many OOMs older than today.
Sorry, I can't quite follow how this connects. Any chance you could explain?
Confusing entropy arguments are suspicious (in terms of hope for ever making sense). That's a sketch of how entropy in physics becomes clearly irrelevant for the content of everything of value (as opposed to amount). Waluigi effect is framing being stronger than direction within it, choice of representation more robust than what gets represented. How does natural selection enter into this?
Life evolves in response to pressure. Entropy is one such source of pressure.

On free will: I don't endorse the claim that "we could have acted differently" as an unqualified statement.

However, I do believe that in order to talk about decisions, we do need to grant validity to a counterfactual view where we could have acted differently as a pragmatically useful fiction.

What's the difference? Well, you can't use the second to claim determinism is false.

This lack of contact with naive conception of possibility should be developed further, so that the reasons for temptation to use the word "fiction" dissolve. An object that captures a state of uncertainty doesn't necessarily come with a set of concrete possibilities that are all "really possible". The object itself is not "fictional", and its shadows in the form of sets of possibilities were never claimed to either be "real possibilities" or to sum up the object, so there is no fiction to be found. A central example of such an object is a program equipped with theorems about its "possible behaviors". Are these behaviors "really possible"? Some of them might be, but the theorems don't pin that down. Instead there are spaces on which the remaining possibilities are painted, shadows of behavior of the program as a whole, such as a set of possible tuples for a given pair of variables in the code. A theorem might say that reality lies within the particular part of the shadow pinned down by the theorem. One of those variables might've stood for your future decision. What "fiction"? All decision relevant possibility originates like that.
I argue that "I can do X" means "If I want to do X, I will do X". This can be true (as an unqualified statement) even with determinism. It is different from saying that X is physically possible.

It seems as though it should be possible to remove the Waluigi effect[1] by appropriately training a model.

Particularly, some combination of:

  • Removing data from the training that matches this effect
  • Constructing new synthetic data which performs the opposite of the Waluigi effect

However, removing this effect might be problematic for certain situations where we want the ability to generate such content, for example, if we want it to write a story.

In this case, it might pay to add back the ability to generate such content within certain tags (ie. <stor... (read more)

Speculation from The Nature of Counterfactuals

I decided to split out some content from the end of my post The Nature of Counterfactuals because upon reflection I don't feel it is as high quality as the core of the post.

I finished The Nature of Counterfactuals by noting that I was incredibly unsure of how we should handle circular epistemology. That said, there are a few ideas I want to offer up on how to approach this. The big challenge with counterfactuals is not imagining other states the universe could be in or how we could apply our "laws" of physics t... (read more)

My position on Newcomb's Problem in a sentence: Newcomb's paradox results from attempting to model an agent as having access to multiple possible choices, whilst insisting it has a single pre-decision brain state.

If anyone was planning on submitting something to this competition, I'll give you another 48 hours to get it in -

Thick and Thin Concepts

Take for example concepts like courage, diligence and laziness. These concepts are considered thick concepts because they have both a descriptive component and a moral component. To be courageous is most often meant* not only to claim that the person undertook a great risk, but that it was morally praiseworthy. So the thick concept is often naturally modeled as a conjunction of a descriptive claim and a descriptive claim.

However, this isn't the only way to understand these concepts. An alternate would be along the following lines: Im... (read more)

I've always found the concept belief in belief slightly hard to parse cognitively. Here's what finally satisfied my brain: whether you will be rewarded or punished in heaven is tied to whether or not God exists, whether or not you feel a push to go to church is tied to whether or not you believe in God. If you do go to church and want to go your brain will say, "See I really do believe" and it'll do the reverse if you don't go. However, it'll only affect your belief in God indirectly through your "I believe in God" node. Putting it another way, going to ch

... (read more)

EDT agents handle Newcomb's problem as follows: they observe that agents who encounter the problem and one-box do better on average than those who encounter the problem and two-box, so they one-box.

That's the high-level description, but let's break it down further. Unlike CDT, EDT doesn't worry about the fact that their may be a correlation between your decision and hidden state. It assumes that if the visible state before you made your decision is the same, then the counterfactuals generated by considering your possible decisions are c... (read more)

I've been thinking about Rousseau and his conception of freedom again because I'm not sure I hit the nail on the head last time. The most typical definition of freedom and that championed by libertarians focuses on an individual's ability to make choices in their daily life. On the more libertarian end, the government is seen as an oppressor and a force of external compulsion.

On the other hand, Rousseau's view focuses on "the people" and their freedom to choose the kind of society that they want to live in. Instead of being se... (read more)

Can you make this a little more explicit? France is a pretty nice place - are you saying that the counterfactual world where there was no revolution would be significantly better?
All the guillotining. And the necessity of that was in part justified with reference to Rousseau's thought
Sure. I'm asking about the "we all saw how that worked out" portion of your comment. From what I can see, it worked out fairly well. Are you of the opinion that the French Revolution was an obvious and complete utilitarian failure?
I haven't looked that much into French history, just think it is important to acknowledge where that line of thought can end up.

What does it mean to define a word? There's a sense in which definitions are entirely arbitrary and what word is assigned to what meaning lacks any importance. So it's very easy to miss the importance of these definitions - emphasising a particular aspect and provides a particular lense with which to see the world.

For example, if define goodness as the ability to respond well to others, it emphasizes that different people have different needs. One person may want advice, while another simple encouragement. Or if we define love as acceptance of the other, it suggests that one of the most important aspects of love is the idea that true love should be somewhat resilient and not excessively conditional.

As I wrote before, evidential decision theory can be critiqued for failing to deal properly with situations where hidden state is correlated with decisions. EDT includes differences in hidden state as part of the impact of the decision, when in the case of the smoking lesion, we typically want to say that it is not.

However, Newcomb's problem also has hidden state is correlated with your decision. And if we don't want to count this when evaluating decisions in the case of the Smoking Lesion, perhaps we shouldn't count this in the case of Newc... (read more)

Does FDT make this any clearer for you? There is a distinction in the correlation, but it's somewhat subtle and I don't fully understand it myself. One silly way to think about it that might be helpful is "how much does the past hinge on your decision?" In smoker's lesion, it is clear the past is very fixed—even if you decide to not to smoke, that doesn't affect the genetic code. But in Newcomb's, the past hinges heavily on your decision: if you decide to one-box, it must have been the case that you could have been predicted to one-box, so it's logically impossible for it to have gone the other way. One intermediate example would be if Omega told you they had predicted you to two-box, and you had reason to fully trust this. In this case, I'm pretty sure you'd want to two-box, then immediately precommit to one-boxing in the future. (In this case, the past no longer hinges on your decision.) Another would be if Omega was predicting from your genetic code, which supposedly correlated highly with your decision but was causally separate. In this case, I think you again want to two-box if you have sufficient metacognition that you can actually uncorrelate your decision from genetics, but I'm not sure what you'd do if you can't uncorrelate. (The difference again lies in how much Omega's decision hinges on your actual decision.)
Yeah, FDT has a notion of subjunctive dependence. But the question becomes what does this mean? What precisely is the difference between the smoking lesion and Newcombs? I have some ideas and maybe I'll write them up at some point.

I'm beginning to warm to the idea that the reason why we have evolved to think in terms of counterfactuals and probabilities is rooted in these are fundamental at the quantum-level. Normally I'm suspicious at rooting macro level claims in quantum level effects because at such a high level of abstraction it would be very easy for these effects to wash out, but the multi-world hypothesis is something that wouldn't wash out. Otherwise it would seem to be all a bit too much of a coincidence.

("Oh, so you believe that counterfactuals and probability are at least... (read more)

I expect that agents evolved in a purely deterministic but similarly complex world would be no less likely to (eventually) construct counterfactuals and probabilities than those in a quantum sort of universe. Far more likely to develop counterfactuals first, since it seems that agents on the level of dogs can imagine counterfactuals at least in the weak sense of "an expected event that didn't actually happen". Human-level counterfactual models are certainly more complex than that, but I don't think they're qualitatively different. I think if there's any evolution pressure toward ability to predict the environment, and the environment has a range of salient features that vary in complexity, there will be some agents that can model and predict the environment better than others regardless of whether that environment is fundamentally deterministic or not. In cases where evolution leads to sufficiently complex prediction, I think it will inevitably lead to some sort of counterfactuals. The simplest predictive model can only be applied to sensory data directly. The agent gains a sense of what to expect next, and how much that differed from what actually happened. This can be used to update the model. This isn't technically a counterfactual, but only through a quirk of language. In everything but name "what to expect next" is at least some weak form of counterfactual. It's a model of an event that hasn't happened and might not happen. But still, let's just rule it out arbitrarily and continue on. The next step is probably to be able to apply the same predictive model to memory as well, which for a model changing over time means that an agent can remember what they experienced, what they expected, and compare with what they would now expect to have happened in those circumstances. This is definitely a counterfactual. It might not be conscious, but it is a model of something in the past that never happened. It opens up a lot of capability for using a bunch of highly sali
"I expect that agents evolved in a purely deterministic but similarly complex world would be no less likely to (eventually) construct counterfactuals and probabilities than those in a quantum sort of universe" I'm actually trying to make a slightly unusual argument. My argument isn't that we wouldn't construct counterfactuals in a purely deterministic world operating similar to ours. My argument is involves: a) Claiming that counterfactuals are at least partly constructed by humans (if you don't understand why this might be reasonable, then it'll be more of a challenge to understand the overall argument) b) Claiming that it would be a massive coincidence if something partly constructed by humans happened to correspond with fundamental structures in such a way unrelated to the fundamental structures c) Concluding that its likely that there is some as yet unspecified relation Does this make sense?
To me the correspondence seems smaller, and therefore the coincidence less unlikely. Many-world hypothesis assumes parallel worlds that obey exactly the same laws of physics. Anything can happen with astronomically tiny probability, but the vast majority of parallel worlds is just as boring as our world. The counterfactuals we imagine are not limited by the laws of physics. Construction of counterfactuals is useful for reasoning with uncertainty. Quantum physics is a source of uncertainty, but there are also enough macroscopic sources of uncertainty (limited brain size, second law of thermodynamics). If an intelligent life evolved in a deterministic universe, I imagine it would also find counterfactual reasoning useful.
Yeah, that's a reasonable position to take.
Not hugely. Quantum mechanics doesn't have any counterfactuals in some interpretations. It has deterministic evolution of state (including entanglement), and then we interpret incomplete information about it as being probabilistic in nature. Just as we interpret incomplete information about everything else.
Hopefully one day I get a chance to look further into quantum mechanics
What correspondence? Counterfactuals-as-worlds have all laws of physics broken in them, including quantum mechanics.
I'm not claiming that there's a perfect correspondence between counterfactuals as different worlds in a multiverse vs. decision counterfactuals. Although maybe that's enough the undermine any coincidence right there?
I don't see how there is anything here other than equivocation of different meanings of "world". Counterfactuals-as-worlds is not even a particularly convincing way of making sense of what counterfactuals are.
If you're interpreting me as defending something along the lines of David Lewis, then that's actually not what I'm doing.
Says who?
[+][comment deleted]2y2
[+][comment deleted]2y2