This is a special post for quick takes by Chris_Leong. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
0
[Image]
0
With quotes from: Yoshua Bengio, Oppenheimer, Toby Ord, Edward Wilson, Nick Bostrom, Carl Sagan, Pope Francis, Antonio Guterres, Stephen Hawking
There's been a lot of discussion about how Less Wrong is mostly just AI these days.
If that's something that folk want to address, I suspect that the best way to do this would be to run something like the Roots of Progress Blog-Building Intensive. My admittedly vague impression is that it seems to have been fairly successful.
Between Less Wrong for distribution, Lighthaven for a writing retreat and Less Online for networking, a lot of the key infrastructure is already there to run a really strong program if the Lightcone team ever decided to pursue this.
Ther...
4
We've actually been thinking about something quite related! More info soon.
(Typo: Lightcone for a writing retreat -> Lighthaven for a writing retreat)
I guess orgs need to be more careful about who they hire as forecasting/evals researchers.
Sometimes things happen, but three people at the same org...
This is also a massive burning of the commons. It is valuable for forecasting/evals orgs to be able to hire people with a diversity of viewpoints in order to counter bias. It is valuable for folks to be able to share information freely with folks at such orgs without having to worry about them going off and doing something like this.
But this only works if those less worried about AI risks who join such a collaboration don't use the knowledge they gain to cash in on the AI boom in an acceleratory way. Doing so undermines the very point of such a project, namely, to try to make AI go well. Doing so is incredibly damaging to trust within the community.
Now let's suppose you're an x-risk funder considering whether to fund their previous org. This org does really high-quality work, but the argument for them being net-positive is now significantly weaker. This is quite likely to make finding future funding harder for them.
This is less about attacking those three folks and more just noting that we need to strive to avoid situations where things like this happen in the first place. This requires us to be more careful in terms of who gets hired.
31
https://www.dwarkesh.com/p/ege-tamay
2
There's lots of things that "might" happen. When we're talking about the future of humanity, we can't afford to just glaze over mights.
I think the conclusion is not Epoch shouldn't have hired Matthew, Tamay, and Ege but rather [Epoch / its director] should have better avoided negative-EV projects (e.g. computer use evals) (and shouldn't have given Tamay leadership-y power such that he could cause Epoch to do negative-EV projects — idk if that's what happened but seems likely).
9
Seems relevant to note here that Tamay had a leadership role from the very beginning: he was the associate director already when Epoch was first announced as an org.
3
This seems like a better solution on the surface, but once you dig in, I'm not so sure.
Once you hire someone, assuming they're competent, it's very hard for you to decide to permanently bar them from gaining a leadership role. How are you going to explain promoting someone who seems less competent than them to a leadership role ahead of them? Or is the plan to never promote them and refuse to ever discuss it, which would create weird dynamics within an organisation.
I would love to hear if you think otherwise, but it seems unworkable to me.
2
I think its not all that uncommon for people who are highly competent in their current role to be passed over for promotion to leadership. LeBron James isn't guaranteed to job as the MBA commissioner just because he balls hard. Things like "avoid[ing] negative-EV projects" would be prime candidates for something like this. If you're amazing at executing technical work on your assigned projects but aren't as good at prioritizing projects or coming up with good ideas for projects, then I could definitely see that blocking a move to leadership even if you're considered insanely competent technically.
5
Can you state more specifically what the alleged bad actions are here? Based on some of the discussions under your post about professional norms surrounding information disclosure, I think it is worth distinguishing two cases.
First, consider a norm that limits the disclosure of some relatively specific and circumscribed pieces of information, such as a doctor not being allowed to reveal personal health information of patients outside of what is needed to provide care.
Second, a general norm that if you cooperate with someone and they provide you some info, you won't use that info contrary to their interests. Its not 100% clear to me, but your post sounds a lot like this second one.
I think the second scenario raises a lot of issues. Its seems challenging to enforce, hard to understand and navigate, costly for people to attempt to conform to, and potentially counterproductive for what seems to be your goal. You are considering a specific case at a specific point in time, but I don't think that gives the full picture of the impact of such a norm. For example, consider ex-OpenAI employees who left due to concerns about AI safety. Should the expectation be that they only use information and experience they gained at OpenAI in a way that OpenAI would approve of?
Now, if Epoch and/or specific individuals made commitments that they violated, that might be more like the first case, but its not clear that is what happened here. If it is, more explanation of how this is the case would be helpful, I think.
3
I agree that this issue is complex and I don't pretend to have all of the solutions.
I just think it's really bad if people feel that they can't speak relatively freely with the forecasting organisations because they'll misuse the information. I think this is somewhat similar to how it is important for folks to be able to speak freely to their doctor/lawyer/psychologist though I admit that the analogy isn't perfect and that straightforwardly copying these norms over would probably be a mistake.
Nonetheless, I think it is worthwhile discussing whether there should be some kind of norms and what they should be. As you've rightly pointed out, are a lot of issues that would need to be considered. I'm not saying I know exactly what these norms should be. I see myself as more just starting a discussion.
(This is distinct from my separate point about it being a mistake to hire folk who do things like this. It is a mistake to have hired folks who act strongly against your interests even if they don't break any ethical injuctions)
3
To "misuse" to me implies taking a bad action. Can you explain what misuse occurred here? If we assume that people at OpenAI now feel less able to speak freely after things that ex-OpenAI employees have said/done would you likewise characterize those people as having "misused" information or experience they gained at OpenAI? I understand you don't have fully formed solutions and that's completely understandable, but I think my questions go to a much more fundamentally issue about what the underlying problem actually is. I agree it is worth discussing, but I think it would clarify the discussion to understand what the intent of such a norm would (and if achieving that intent would in fact be desirable).
If Coca-Cola hires someone who later leaves and goes to work for Pepsi because Pepsi offered them higher compensation, I'm not sure it would make sense for Coca-Cola to conclude that they should make big changes to their hiring process, other than perhaps increasing their own compensation if they determine that is a systematic issue. Coca-Cola probably needs to accept that "its not personal" is sometimes going to be the natural of the situation. Obviously details matter, so maybe this case is different, but I think working in an environment where you need to cooperate with other people/institutions means you also have to sometimes accept that people you work with will make decisions based on their own judgements and interests, and therefore may do things you don't necessarily agree with.
2
They're recklessly accelerating AI. Or, at least, that's how I see it. I'll leave it to others to debate whether or not this characterisation is accurate.
Details matter. It depends on how bad it is and how rare these actions are.
8
I know I've responded to a lot of your comments, and I get the sense you don't want to keep engaging with me, so I'll try to keep it brief.
We both agree that details matter, and I think the details of what the actual problem is matter. If, at bottom, the thing that Epoch/these individuals have done wrong is recklessly accelerate AI, I think you should have just said that up top. Why all the "burn the commons", "sharing information freely", "damaging to trust" stuff? It seems like you're saying at the end of the day, those things aren't really the thing you have a problem with. On the other hand, I think invoking that stuff is leading you to consider approaches that won't necessarily help with avoiding reckless acceleration, as I hope my OpenAI example demonstrates.
3
I believe those are useful frames for understanding the impacts.
It is valuable for forecasting/evals orgs to be able to hire people with a diversity of viewpoints in order to counter bias.
This requires us to be more careful in terms of who gets hired in the first place.
I mean, good luck hiring people with a diversity of viewpoints who you're also 100% sure will never do anything that you believe to be net negative. Like what does "diversity of viewpoints" even mean apart from that?
3
Everything has trade-offs.
I agree that attempting to be 100% sure that they're responsible would be a mistake. Specifically, the unwanted impacts would likely be too high.
But this only works if those less worried about AI risks who join such a collaboration don't use the knowledge they gain to cash in on the AI boom in an acceleratory way. Doing so undermines the very point of such a project, namely, to try to make AI go well. It is incredibly damaging to trust within the community.
...This is less about attacking those three folks and more just noting that we need to strive to avoid situations where things like this happen in the first place.
(note: I work at Epoch) This attitude feels like a recipe for creating an intellectual bubble. Of course people will use the knowledge they gain in collaboration with you for the purposes that they think are best. I think it would be pretty bad for the AI safety community if it just relied on forecasting work from card-carrying AI safety advocates.
126
It is entirely normal for there to be widely accepted, clearly formalized, and meaningfully enforced restrictions on how people use knowledge they've gotten in this or that setting... regardless of what they think is best. It's a commonplace of professional ethics.
2
I don't think this is true. People can't really restrict their use of knowledge, and subtle uses are pretty unenforceable. So it's expected that knowledge will be used in whatever they do next. Patents and noncompete clauses are attempts to work around this. They work a little, for a little.
2
Agreed. This is how these codes form. Someone does something like this and then people discuss and decide that there should be a rule against it or that it should at least be frowned upon.
7
Sure, there are in some very specific settings with long held professional norms that people agree to (e.g. doctors and lawyers). I don't think this applies in this case, though you could try to create such a norm that people agree to.
1
I largely agree with the underlying point here, but I don't think its quite correct that something like this only applies in specific professions. For example, I think every major company is going to expect employees to be careful about revealing internal info, and there are norms that apply more broadly (trade secrets, insider trading etc.).
As far as I can tell though, those are all highly dissimilar to this scenario because they involve an existing widespread expectation of not using information in a certain way. Its not even clear to me in this case what information was used in what way that is allegedly bad.
3
I would like to see serious thought given to instituting such a norm. There's a lot of complexities here, figuring out what is or isn't kosher would be challenging, but it should be explored.
Thanks for weighing in.
This attitude feels like a recipe for creating an intellectual bubble
Oh, additional screening could very easily have unwanted side-effects. That's why I wrote: "It is valuable for forecasting/evals orgs to be able to hire people with a diversity of viewpoints in order to counter bias" and why it would be better for this issue to never have arisen in the first place. Actions like this can create situations with no good trade-offs.
I think it would be pretty bad for the AI safety community if it just relied on forecasting work from card-carrying AI safety advocates.
I was definitely not suggesting that the AI safety community should decide which forecasts to listen to based on the views of the forecasters. That's irrelevant, we should pay attention to the best forecasters.
I was talking about funding decisions. This is a separate matter.
If someone else decides to fund a forecaster even though we're worried they're net-negative or they do work voluntarily, then we should pay attention to their forecasts if they're good at their job.
Of course people will use the knowledge they gain in collaboration with you for the purposes that they think are best
Seems like several...
Random thought: We should expect LLM's trained on user responses to have much more situational knowledge than early LLM's trained on the pre-Chatbot internet because users will occasionally make reference to the meta-context.
It may be possible to get some of this information from pre-training on chatlogs/excerpts that make their way onto the internet, but the information won't be quite as accessible because of differences in the context.
If this were a story, there'd be some kind of academy taking in humanity's top talent and skilling them up in alignment.
Most of the summer fellowships seem focused on finding talent that is immediately useful. And I can see how this is tempting given the vast numbers of experienced and talented folks seeking to enter the space. I'd even go so far as to suggest that the majority of our efforts should probably be focused on finding people who will be useful fairly quickly.
Nonetheless, it does seem as though there should be at least one program that aims to find the best talent (even if they aren't immediately useful) and which provides them with the freedom to explore and the intellectual environment in which to do so.
I wish I could articulate my intuition behind this clearer, but the best I can say for now is that my intuition is that continuing to scale existing fellowships would likely provide decreasing marginal returns and such an academy wouldn't be subject to this because it would be providing a different kind of talent.
2
I think SPARC and its decedents are something like this.
2
How long does SPARC go for?
8
Upskilling bright young people “to do alignment” is tricky to do in a systematic way, since bright young people want / need to do whatever they’re curious about.
2
Maybe you're thinking younger than I was thinking.
I expect you'd mostly want folks who'd already completed an undergraduate degree, with sufficiently talented folks being pulled in earlier.
Collapsable boxes are amazing. You should consider using them in your posts.
They are a particularly nice way of providing a skippable aside. For example, filling in background information, answering an FAQ or including evidence to support an assertion.
Compared to footnotes, collapsable boxes are more prominent and are better suited to contain paragraphs or formatted text.
Less Wrong might want to consider looking for VC funding for their forum software in order to deal with the funding crunch. It's great software. It wouldn't surprise me if there were businesses who would pay for it and it could allow an increase in the rate of development. There's several ways this could go wrong, but it at least seems worth considering.
6
I'm sure they thought about it.
I think this is dramatically tougher than a lot of people think. I wrote more about it here.
https://www.facebook.com/ozzie.gooen/posts/pfbid0377Ga4W8eK89aPXDkEndGtKTgfR34QXxxNCtwvdPsMifSZBY8abLmhfybtMUkLd8Tl
For the record, I see the new field of "economics of transformative AI" as overrated.
Economics has some useful frames, but it also tilts people towards being too "normy" on the impacts of AI and it doesn't have a very good track record on advanced AI so far.
I'd much rather see multidisciplinary programs/conferences/research projects, including economics as just one of the perspectives represented, then economics of transformative AI qua economics of transformative AI.
(I'd be more enthusiastic about building economics of transformative AI as a field if we w...
I just created a new Discord server for generated AI safety reports (ie. using Deep Research or other tools). Would be excited to see you join (ps. Open AI now provides uses on the plus plan 10 queries per month using Deep Research).
https://discord.gg/bSR2hRhA
Why the focus on wise AI advisors? (Metapost[1] with FAQ 📚🙋🏻♂️, Informal[2] Edition 🕺, Working Draft 🛠️)
About this Post - 🔗: 🕺wiseaiadvisors.com , 🕴️Formal Edition (coming soon)
🕺wiseaiadvisors.com , 🕴️Formal Edition (coming soon) This post is still in draft, so any feedback would be greatly appreciated 🙏. It'll be posted as a full, proper Less Wrong/EA Forum/Alignment forum post, as opposed to just a short-form, when it's ready 🌱🌿🌳.
📲 QR Code
This post is a collaboration between Chris Leong (primary author) and Christopher C
14
Despite my contention on the associated paper post that focusing on wisdom in this sense is ducking the hard part of the alignment problem, I'll stress here that it Iseems thoroughly useful if it's a supplement not a substitute for work on the hard parts of the problem - technical, theoretical and societal.
I also think it's going to be easier to create wise advisors than you think, at least in the weak sense that they make their human users effectively wiser.
In short, think simple prompting schemes and eventually agentic scaffolds can do a lot of the extra work it takes to turn knowledge into wisdom, and that there's an incentive for orgs to train for "wisdom" in the sense you mean as well. So we'll get wiser advisors as we go, at little or no extra effort. More effort would of course help more.
I believe Deep Research has already made me wiser. I can get a broader context for any given decision.
And that was primarily achieved by prompting; the o3 model that powers OpenAI's version does seem to help but Perplexity introducing a nearly-as-good system just a week or two later indicates that just the right set of prompts were extremely valuable.
Current systems aren't up to helping very much with the hypercomplex problems surrounding alignment. But they can now help a little. And any improvements will be a push in the right direction.
Training specifically for "wisdom" as you define it is a push toward a different type of useful capability, so it may be that frontier labs pursue similar training by default.
(As an aside, I think your "comparisons" are all wildly impractical and highly unlikely to be executed before we hit AGI, even on longer realistic estimates. It's weird that they're considered valid points of comparison, as all plans that will never be executed have exactly the same value. But that's where we're at in the project right now.)
To return from the tangent, I don't think wise advisors is actually asking anyone to go far out of their d
2
Thanks for sharing your thoughts.
I agree that humans with wise AI advisors is a more promising approach, at least at first, then attempting to directly program wisdom into an autonomously acting agent.
Beyond that, I personally haven't made up my mind yet about the best way to use wisdom tech.
2
Why do you think wise AI advisors avoid the general problems with other AI?
2
Well, we're going to be training AI anyway. If we're just training capabilities, but not wisdom, I think things are unlikely to go well. More thoughts on this here.
2
Hm, I thought this use of "wise" is almost identical to capabilities. It's sort of like capabilities with less slop or confabulation, and probably more ability to take the context of the problem/question into account. Both of those are pretty valuable, although people might not want to bother even swerving capabilities in that direction.
6
My current working take is that it is at the level of a median-but-dedicated undergraduate of a top university who is interested and enthusiastic about AI safety. But Deep Research can do in 10 minutes what would take that undergraduate about 20 hours.
Happy to try a prompt for you and see what you think.
How about "Please summarise Eliezer Yudkowsky's views on decision theory and its relevance to the alignment problem".
7
Here you go: https://chatgpt.com/share/67a34222-e7d8-800d-9a86-357defc15a1d
5
Thanks, seems pretty good on a quick skim, I'm a bit less certain on the corrigibility section, also more issues might become apparent if I read through it more slowly.
8
People have said that to get a good prompt it's better to have a discussion with a model like o3-mini, o1, or Claude first, and clarify various details about what you are imagining, then give the whole conversation as a prompt to OA Deep Research.
Sharing this resource doc on AI Safety & Entrepreneurship that I created in case anyone finds this helpful:
https://docs.google.com/document/d/1m_5UUGf7do-H1yyl1uhcQ-O3EkWTwsHIxIQ1ooaxvEE/edit?usp=sharing
There is a world that needs to be saved. Saving the world is a team sport. All we can do is to contribute our part of the puzzle, whatever that may be and no matter how small, and trust in our companions to handle the rest. There is honor in that, no matter how things turn out in the end.
7
I have no interest in honor if it's celebrated on a field of the dead. Virtue ethics is fine, as long as it's not an excuse to not figure out what needs doing and how it's going to get done.
Doing ones own part and trusting that the other parts are done by anonymous unknown others is a very silly coordination strategy. We need plans that amount to success, not just everyone doing whatever sounds nice to them.
Edit: I very much agree that saving the world is a team sport. Perhaps it's relevant that successful teams always do some planning and coordinating.
-8
I'll post some extracts from the Seoul Summit. I can't promise that this will be a particularly good summary, I was originally just writing this for myself, but maybe it's helpful until someone publishes something that's more polished:
Frontier AI Safety Commitments, AI Seoul Summit 2024
The major AI companies have agreed to Frontier AI Safety Commitments. In particular, they will publish a safety framework focused on severe risks: "internal and external red-teaming of frontier AI models and systems for severe and novel threats; to work toward information sharing; to invest in cybersecurity and insider threat safeguards to protect proprietary and unreleased model weights; to incentivize third-party discovery and reporting of issues and vulnerabilities; to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated; to publicly report model or system capabilities, limitations, and domains of appropriate and inappropriate use; to prioritize research on societal risks posed by frontier AI models and systems; and to develop and deploy frontier AI models and systems to help address the world’s greatest challenges"
"Risk assessments should consid...
Was thinking about entropy and the Waluigi effect (in a very broad, metaphorical sense).
The universe trends towards increasing entropy, in such an environment it is evolutionarily advantageous to have the ability to resist it. Notice though that life seems to have overshot and resulted in far more complex ordered systems (both biological or manmade) than what exists elsewhere.
It's not entirely clear to me, but it seems at least somewhat plausible that if entropy were weaker, the evolutionary pressure would be weaker and the resulting life and systems produce by such life would ultimately be less complex than they are in our world.
2
Life happens within computations in datacenters. Separately, there are concerns about how well the datacenters will be doing when the universe is many OOMs older than today.
2
Sorry, I can't quite follow how this connects. Any chance you could explain?
2
Confusing entropy arguments are suspicious (in terms of hope for ever making sense). That's a sketch of how entropy in physics becomes clearly irrelevant for the content of everything of value (as opposed to amount). Waluigi effect is framing being stronger than direction within it, choice of representation more robust than what gets represented. How does natural selection enter into this?
2
Life evolves in response to pressure. Entropy is one such source of pressure.
I think that there's good reasons why the discussion on Less Wrong has turned increasingly towards AI Alignment, but I am also somewhat disappointed that there's no longer a space focusing on rationality per se.
Just as the Alignment forum exists as a separate space that automatically cross-posts to LW, I'm starting to wonder if we need a rationality forum that exists as a separate space that cross-posts to LW, as if I were just interested in improving my rationality I don't know if I'd come to Less Wrong.
(To clarify, unlike the Alignment Forum, I'd expect such a forum to be open-invite b/c the challenge would be gaining any content at all).
2
Alternatively, I think there is a way to hide the AI content on LW, but perhaps there should exist a very convenient and visible user interface for that. I would propose an extreme solution, like a banner on the top of the page containing a checkbox that hides all AI content. So that anyone, registered or not, could turn the AI content off in one click.
2
The Alignment forum works because there are a bunch of people who professionally pursue research over AI Alignment. There's no similar group of people for whom that's true for rationality.
2
I don't know if you need professionals, just a bunch of people who are interested in discussing the topic. It wouldn't need to use the Alignment Forum's invite-only system.
Instead, it would just be a way to allow LW to cater to both audiences at the same time.
1
IIRC, you can get post on Alignment Forum only if you are invited or moderators crossposted it? The problem is that Alignment Forum is deliberately for some sort of professionals, but everyone wants to write about alignment. Maybe it would be better if we had "Alignment Forum for starters".
On free will: I don't endorse the claim that "we could have acted differently" as an unqualified statement.
However, I do believe that in order to talk about decisions, we do need to grant validity to a counterfactual view where we could have acted differently as a pragmatically useful fiction.
What's the difference? Well, you can't use the second to claim determinism is false.
1
I argue that "I can do X" means "If I want to do X, I will do X". This can be true (as an unqualified statement) even with determinism. It is different from saying that X is physically possible.
2
This lack of contact with naive conception of possibility should be developed further, so that the reasons for temptation to use the word "fiction" dissolve. An object that captures a state of uncertainty doesn't necessarily come with a set of concrete possibilities that are all "really possible". The object itself is not "fictional", and its shadows in the form of sets of possibilities were never claimed to either be "real possibilities" or to sum up the object, so there is no fiction to be found.
A central example of such an object is a program equipped with theorems about its "possible behaviors". Are these behaviors "really possible"? Some of them might be, but the theorems don't pin that down. Instead there are spaces on which the remaining possibilities are painted, shadows of behavior of the program as a whole, such as a set of possible tuples for a given pair of variables in the code. A theorem might say that reality lies within the particular part of the shadow pinned down by the theorem. One of those variables might've stood for your future decision. What "fiction"? All decision relevant possibility originates like that.
It seems as though it should be possible to remove the Waluigi effect[1] by appropriately training a model.
Particularly, some combination of:
- Removing data from the training that matches this effect
- Constructing new synthetic data which performs the opposite of the Waluigi effect
However, removing this effect might be problematic for certain situations where we want the ability to generate such content, for example, if we want it to write a story.
In this case, it might pay to add back the ability to generate such content within certain tags (ie. <stor...
Speculation from The Nature of Counterfactuals
I decided to split out some content from the end of my post The Nature of Counterfactuals because upon reflection I don't feel it is as high quality as the core of the post.
I finished The Nature of Counterfactuals by noting that I was incredibly unsure of how we should handle circular epistemology. That said, there are a few ideas I want to offer up on how to approach this. The big challenge with counterfactuals is not imagining other states the universe could be in or how we could apply our "laws" of physics t...
My position on Newcomb's Problem in a sentence: Newcomb's paradox results from attempting to model an agent as having access to multiple possible choices, whilst insisting it has a single pre-decision brain state.
Here's a crazy[1] idea that I had. But I think it's an interesting thought experiment.
What if we programmed an AGI had the goal of simulating the Earth, but with one minor modification? In the simulation, we would have access to some kind of unfair advantage, like an early Eliezer Yudkowsky getting a mysterious message dropped on his desk containing a bunch of the progress we've made in AI Alignment.
So we'd all die in real life when the AGI broke out of its box and turned the Earth into compute to better simulate us, but we might survive in virtual re...
Random idea: A lot of people seem discouraged from doing anything about AI Safety because it seems like such a big overwhelming problem.
What if there was a competition to encourage people to engage in low-effort actions towards AI safety, such as hosting a dinner for people who are interested, volunteering to run a session on AI safety for their local EA group, answering a couple of questions on the stampy wiki, offering to proof-read a few people’s posts or offering a few free tutorial sessions to aspiring AI Safety Researchers.
I think there’s a dec...
2
I like it, and it's worth trying out.
Those don't seem like very low effort to me, but they will to some. Do they seem to you like they are effective (or at least impactful commensurate with the effort)? How would you know which ones to continue and what other types of thing to encourage?
I fear that it has much of the same problem that any direct involvement in AI safety does: what's the feedback loop for whether it's actually making a difference? Your initial suggestions seem more like actions toward activism and pop awareness, rather than actions toward AI Safety.
The nice thing about prizes and compensation is that it moves the question from the performer to the payer - the payer has to decide if it's a good value. Small prizes or low comp means BOTH buyer and worker have to make the decision of whether this is worthwhile.
Solving the productivity-measurement problem itself seems overwhelming - it hasn't happened even for money-grubbing businesses, let alone long-term x-risk organizations. But any steps toward it will do more than anything else I can think of to get broader and more effective participation. Being able to show that what I do makes a measurable difference, even through my natural cynicism and imposter syndrome, is key to my involvement.
I am not typical, so don't take my concerns as the final word - this seems promising and relatively cheap (in money; it will take a fair bit of effort in guiding the sessions and preparing materials for the tutoring. Honestly, that's probably more important than the actual prizes).
4
I guess they just feel like as good a starting place as any and are unlikely to be net-negative. That's more important than anything else. The point is to instill agency so that people start looking for further opportunities to make a difference. I might have to write a few paragraphs of guidelines/suggestions for some of the most common potential activities.
I hadn't really thought too much about follow-up, but maybe I should think more about it.
Thoughts on the introduction of Goodhart's. Currently, I'm more motivated by trying to make the leaderboard, so maybe that suggests that merely introducing a leaderboard, without actually paying people, would have had much the same effect. Then again, that might just be because I'm not that far off. And if there hadn't been the payment, maybe I wouldn't have ended up in the position where I'm not that far off.
I guess I feel incentivised to post a lot more than I would otherwise, but especially in the comments rather than the posts since if you post a lot of posts that likely suppresses the number of people reading your other posts. This probably isn't a worthwhile tradeoff given that one post that does really well can easily outweight 4 or 5 posts that only do okay or ten posts that are meh.
Another thing: downvotes feel a lot more personal when it means that you miss out on landing on the leaderboard. This leads me to think that having a leaderboard for the long term would likely be negative and create division.
If anyone was planning on submitting something to this competition, I'll give you another 48 hours to get it in - https://www.lesswrong.com/posts/Gzw6FwPD9FeL4GTWC/usd1000-usd-prize-circular-dependency-of-counterfactuals.
Thick and Thin Concepts
Take for example concepts like courage, diligence and laziness. These concepts are considered thick concepts because they have both a descriptive component and a moral component. To be courageous is most often meant* not only to claim that the person undertook a great risk, but that it was morally praiseworthy. So the thick concept is often naturally modeled as a conjunction of a descriptive claim and a descriptive claim.
However, this isn't the only way to understand these concepts. An alternate would be along the following lines: Im...
I don't want to comment on the whole Leverage Controversy as I'm far away enough from the action that other people are probably better positioned to sensemake here.
On the other hand, I have been watching some of Geoff Anders' streams does seem pretty good at theorising by virtue of being able to live-stream this. I expect this to be a lot harder than it looks, when I'm trying to figure out my position on an issue, I often find myself going over the same ground again and again and again, until eventually I figure out a way of putting what I want to express into words.
That said, I've occasionally debated with some high-level debaters and given almost any topic they're able to pretty much effortlessly generate a case and how the debate is likely to play out. I guess it seems on par with this.
So I think his ability to livestream demonstrates a certain level of skill, but I almost view it as speed-chess vs. chess, in that there's only so much you can tell about a person's ability in normal chess from how good they are at speed chess.
I think I've improved my own ability to theorise by watching the streams, but I wouldn't be surprised if I improved similarly from watching Eliezer, A...
2
Any in particular, or what most of them are like?
2
In what ways do you believe you improved?
2
I think I'm more likely to realise that I haven't hit the nail on the head and so I go back and give it another go.
3
Trying to think about what is required to be a good debater:
* general intelligence -- to quickly understand the situation and lay out your response;
* "talking" skills -- large vocabulary, talking clearly, not being shy, body language and other status signals;
* background knowledge -- knowing the models, facts, frequently used arguments, etc.;
* precomputed results -- if you already spent a lot of time thinking about a topic, maybe even debating it.
These do not work the same way, for example clear talking and good body language generalize well; having lots of precomputed results in one area will not help you much in other areas (unless you use a lot of analogies to the area you are familiar with -- if you do this the first time, you may impress people, but if you do this repeatedly, they will notice that you are a one-topic person).
I believe that watching good debaters in action would help. It might be even better to focus on different aspects separately (observing their body language, listening to how they use their voice, understanding their frames, etc.).
I'm beginning to warm to the idea that the reason why we have evolved to think in terms of counterfactuals and probabilities is rooted in these are fundamental at the quantum-level. Normally I'm suspicious at rooting macro level claims in quantum level effects because at such a high level of abstraction it would be very easy for these effects to wash out, but the multi-world hypothesis is something that wouldn't wash out. Otherwise it would seem to be all a bit too much of a coincidence.
("Oh, so you believe that counterfactuals and probability are at least...
1
What correspondence? Counterfactuals-as-worlds have all laws of physics broken in them, including quantum mechanics.
-1
Says who?
2
I'm not claiming that there's a perfect correspondence between counterfactuals as different worlds in a multiverse vs. decision counterfactuals. Although maybe that's enough the undermine any coincidence right there?
2
I don't see how there is anything here other than equivocation of different meanings of "world". Counterfactuals-as-worlds is not even a particularly convincing way of making sense of what counterfactuals are.
2
If you're interpreting me as defending something along the lines of David Lewis, then that's actually not what I'm doing.
6
I expect that agents evolved in a purely deterministic but similarly complex world would be no less likely to (eventually) construct counterfactuals and probabilities than those in a quantum sort of universe. Far more likely to develop counterfactuals first, since it seems that agents on the level of dogs can imagine counterfactuals at least in the weak sense of "an expected event that didn't actually happen". Human-level counterfactual models are certainly more complex than that, but I don't think they're qualitatively different.
I think if there's any evolution pressure toward ability to predict the environment, and the environment has a range of salient features that vary in complexity, there will be some agents that can model and predict the environment better than others regardless of whether that environment is fundamentally deterministic or not. In cases where evolution leads to sufficiently complex prediction, I think it will inevitably lead to some sort of counterfactuals.
The simplest predictive model can only be applied to sensory data directly. The agent gains a sense of what to expect next, and how much that differed from what actually happened. This can be used to update the model. This isn't technically a counterfactual, but only through a quirk of language. In everything but name "what to expect next" is at least some weak form of counterfactual. It's a model of an event that hasn't happened and might not happen. But still, let's just rule it out arbitrarily and continue on.
The next step is probably to be able to apply the same predictive model to memory as well, which for a model changing over time means that an agent can remember what they experienced, what they expected, and compare with what they would now expect to have happened in those circumstances. This is definitely a counterfactual. It might not be conscious, but it is a model of something in the past that never happened. It opens up a lot of capability for using a bunch of highly sali
2
"I expect that agents evolved in a purely deterministic but similarly complex world would be no less likely to (eventually) construct counterfactuals and probabilities than those in a quantum sort of universe"
I'm actually trying to make a slightly unusual argument. My argument isn't that we wouldn't construct counterfactuals in a purely deterministic world operating similar to ours. My argument is involves:
a) Claiming that counterfactuals are at least partly constructed by humans (if you don't understand why this might be reasonable, then it'll be more of a challenge to understand the overall argument)
b) Claiming that it would be a massive coincidence if something partly constructed by humans happened to correspond with fundamental structures in such a way unrelated to the fundamental structures
c) Concluding that its likely that there is some as yet unspecified relation
Does this make sense?
1
Not hugely. Quantum mechanics doesn't have any counterfactuals in some interpretations. It has deterministic evolution of state (including entanglement), and then we interpret incomplete information about it as being probabilistic in nature. Just as we interpret incomplete information about everything else.
2
Hopefully one day I get a chance to look further into quantum mechanics
5
To me the correspondence seems smaller, and therefore the coincidence less unlikely.
Many-world hypothesis assumes parallel worlds that obey exactly the same laws of physics. Anything can happen with astronomically tiny probability, but the vast majority of parallel worlds is just as boring as our world. The counterfactuals we imagine are not limited by the laws of physics.
Construction of counterfactuals is useful for reasoning with uncertainty. Quantum physics is a source of uncertainty, but there are also enough macroscopic sources of uncertainty (limited brain size, second law of thermodynamics). If an intelligent life evolved in a deterministic universe, I imagine it would also find counterfactual reasoning useful.
4
Yeah, that's a reasonable position to take.
I was talking with Rupert McCallum about the simulation hypothesis yesterday. Rupert suggested that this argument is self-defeating; that is it pulls the rug from under its own feet. It assumes the universe has particular properties, then it tries to estimate the probability of being in a simulation from these properties and if the probability is sufficiently high, then we conclude that we are in a simulation. But if we are likely to be in a simulation, then our initial assumptions about the universe are likely to be false, so we've disproved the assu...
2
It's been a while since I read the paper, but wasn't the whole argument around people wanting to simulate different versions of their world and population? There's a baked in assumption that worlds similar to ones own are therefore more likely to be simulated.
2
Yeah, that's possible. Good point!
4
This counterargument was suggested before by Danila Medvedev and it doesn't' work. The reasons are following: if we are in a simulation, we can't say anything about the outside world - but we are still in simulation and this is what was needed to be proved.
4
"This is what was needed to be proved" - yeah, but we've undermined the proof. That's why I backed up and reformulated the argument in the second paragraph.
3
One more way to prove simulation argument is a general observation that explanations which have lower computational cost are dominating my experience (that is, a variant of Occam Razor). If I see a nuclear explosion, it is more likely to be a dream, a movie or a photo. Thus cheap simulations should be more numerous than real worlds and we are likely to be in it.
I really dislike the fiction that we're all rational beings. We really need to accept that sometimes people can't share things with us. Stronger: not just accept but appreciate people who make this choice for their wisdom and tact. ALL of us have ideas that will strongly trigger us and if we're honest and open-minded, we'll be able recall situations when we unfairly judged someone because of a view that they held. I certainty can, way too many times to list.
I say this as someone who has a really strong sense of curiosity, knowing that I...
6
When you start identifying as a rationalist, the most important habit is saying "no" whenever someone says: "As a rationalist, you have to do X" or "If you won't do X, you are not a true rationalist" etc. It is not a coincidence that X usually means you have to do what the other person wants for straightforward reasons.
Because some people will try using this against you. Realize that this usually means nothing more then "you exposed a potential weakness, they tried to exploit it" and is completely unrelated to the art of rationality.
(You can consider the merits of the argument, of course, but you should do it later, alone, when you are not under pressure. Don't forget to use the outside view; the easiest way is to ask a few independent people.)
I've always found the concept belief in belief slightly hard to parse cognitively. Here's what finally satisfied my brain: whether you will be rewarded or punished in heaven is tied to whether or not God exists, whether or not you feel a push to go to church is tied to whether or not you believe in God. If you do go to church and want to go your brain will say, "See I really do believe" and it'll do the reverse if you don't go. However, it'll only affect your belief in God indirectly through your "I believe in God" node. Putting it another way, going to ch
Pet theory about meditation: Lots of people say that if you do enough meditation that you will eventually realise that there isn't a self. Having not experienced this myself, I am intensely curious about what people observe that persuades them to conclude this. I guess I get a sense that many people are being insufficiently skeptical. There's a difference between there not appearing to be such a thing as a self and a self not existing. Indeed, how do we know meditation just doesn't temporarily silence whatever part of our mind is responsible...
I've recently been reading about ordinary language philosophy and I noticed that some of their views align quite significantly with LW. They believed that many traditional philosophical question only seemed troubling because of the philosophical tendency to assume words like "time" or "free will" necessarily referred to some kind of abstract entity when this wasn't necessary at all. Instead they argued that by paying attention to how we used these words in ordinary, everyday situations we could see that the way people used the...
2
From my perspective, saying that "this philosophical opinion is kinda like this Less Wrong article" sounds like "this prophecy by Nostradamus, if you squint hard enough, predicts coronavirus in 2020". What I mean is that if you publish huge amounts of text open to interpretation, it is not surprising that you can find there analogies to many things. I would not be surprised to find something similar in the Bible; I am not surprised to find something similar in philosophy. (I would not be surprised to also find a famous philosopher who said the opposite.) In philosophy, the generation of text is distributed, so some philosophers likely have track record much better than the average of their discipline. Unfortunately -- as far as I know -- philosophy as a discipline doesn't have a mechanism to say "these ideas of these philosophers are the good ones, and this is wrong". At least my time at philosophy lessons was wasted listening to what Plato said, without a shred of "...and according to our current scientific knowledge, this is true, and this is not".
Also, it seems to me that philosophers were masters of clickbait millenia before clickbait was a thing. For example, a philosopher is rarely satisfied by saying things like "human bodies are composed of 80% water" or "most atoms in the universe are hydrogen atoms". Instead, it is typically "everything is water". (Or "everything is fire". Or "everything is an interaction of quantum fields"... oops, the last one was actually not said by a philosopher; what a coincidence.) Perhaps this is selection bias. Maybe people who walked around ancient Greece half-naked and said things like "2/3 of everything is water" existed, but didn't draw sufficient attention. But if this is true, it would mean that philosophy optimizes for shock value instead of truth value.
So, without having read Wittgenstein, my priors are that he most likely considered all words confused; yes, words like "time" and "free will", but also words like "apple
4
I won't pretend that I have a strong understanding here, but as far as I can tell, (Later) Wittgenstein and the Ordinary Language Philosophers considered our conception of the number "five" existing as an abstract object as mistaken and would instead explain how it is used and consider that as a complete explanation. This isn't an unreasonable position, like I honestly don't know what numbers are and if we say they are an abstract entity it's hard to say what kind of entity.
Regarding the word "apple" Wittgenstein would likely say attempts to give it a precise definition are doomed to failure because there are an almost infinite number of contexts or ways in which it can be used. We can strongly state "Apple!" as a kind of command to give us one, or shout it to indicate "Get out of the way, there is an apple coming towards you" or "Please I need an Apple to avoid starving". But this is only saying attempts to spec out a precise definition are confused, not the underlying thing itself.
(Actually, apparently Wittgenstein consider attempts to talk about concepts like God or morality as necessarily confused, but thought that they could still be highly meaningful, possibly the most meaningful things)
2
These are all good points. I could agree that all words are to some degree confused, but I would insist that some of them are way more confused than others. Otherwise, the very act of explaining anything would be meaningless: we would explain one word by a bunch of words, equally confusing.
If the word "five" is nonsense, I can take the Wittgenstein's essay explaining why it is nonsense, and say that each word in that essay is just a command that we can shout at someone, but otherwise is empty of meaning. This would seem to me like an example of intelligence defeating itself.
2
Wittgenstein didn't think that everything was a command or request; his point was that making factual claims about the world is just one particular use of language that some philosophers (including early Wittgenstein) had hyper-focused on.
Anyway, his claim wasn't that "five" was nonsense, just that when we understood how five was used there was nothing further for us to learn. I don't know if he'd even say that the abstract concept five was nonsense, he might just say that any talk about the abstract concept would inevitably be nonsense or unjustified metaphysical speculation.
2
These are situations where I woud like to give a specific question to the philosopher. In this case it would be: "Is being a prime number a property of number five, or is it just that we decided to use it as a prime number?"
2
I honestly have no idea how he'd answer, but here's one guess. Maybe we could tie prime numbers to one of a number of processes for determining primeness. We could observe that those processes always return true for 5, so in a sense primeness is a property of five.
3
Is that surprising? It's not as if the rationalsphere performed some comprehensive survey of philosophy before announcing the superiority of its own methods.
Three levels of forgiveness - emotions, drives and obligations. The emotional level consists of your instinctual anger, rage, disappointment, betrayal, confusion or fear. This is about raw raws. The drives consists of your "need" for them to say sorry, make amends, regret their actions, have a conversation or emphasise with you. In other words, it's about needing the situation to turn out a particular way. The obligations are very similar to the drives, except it is about their duty to perform these actions rather than your desire to make it...
3
The phrase "an eye for an eye" could be construed as duty - that the wrong another does you is a debt you have to repay. (Possibly inflated, or with interest. It's also been argued that it's about (motivating) recompense - you pay the price for taking another's eye, or you lose yours.)
2
Interesting point, but you're using duty differently than me. I'm talking about their duties towards you. Of course, we could have divided it another way or added extra levels.
1
Your duties (towards others) may include what you are supposed to do if others don't fulfill their duties (towards you).
There appears to be something of a Sensemaking community developing on the internet, which could roughly be described as a spirituality-inspired attempt at epistemology. This includes Rebel Wisdom, Future Thinkers, Emerge and maybe you could even count post-rationality. While there are undoubtedly lots of critiques that could be made of their epistemics, I'd suggest watching this space as I think some interesting ideas will emerge out of it.
EDT agents handle Newcomb's problem as follows: they observe that agents who encounter the problem and one-box do better on average than those who encounter the problem and two-box, so they one-box.
That's the high-level description, but let's break it down further. Unlike CDT, EDT doesn't worry about the fact that their may be a correlation between your decision and hidden state. It assumes that if the visible state before you made your decision is the same, then the counterfactuals generated by considering your possible decisions are c...
Hegel - A Very Short Introduction by Peter Singer - Book Review Part 1: Freedom
Hegel is a philosopher who is notorious for being incomprehensible. In fact, for one of his books he signed a contract that assigned a massive financial penalty for missing the publishing deadline, so the book ended up being a little rushed. While there was a time when he was dominant in German philosophy, he now seems to be held in relatively poor regard and his main importance is seen to be historical. So he's not a philosopher that I was really planning to spend much time on.
Given this, I was quite pleased to discover this book promising to give me A Very Short Introduction, especially since it is written by Peter Singer, a philosopher who write and thinks rather clearly. After reading this book, I still believe that most of what Hegel wrote was pretentious nonsense, but the one idea that struck me as the most interesting was his conception of freedom.
A rough definition of freedom might be ensuring that people are able to pursue whatever it is that they prefer. Hegel is not a fan abstract definitions of freedom which treat all preferences the same and don't enquire where they come from.
In hi...
2
I tend to think of Hegel as primarily important for his contributions to the development of Western philosophy (so even if he was wrong on details he influenced and framed the work of many future philosophers by getting aspects of the framing right) and for his contributions to methodology (like standardizing the method of dialectic, which on one hand is "obvious" and people were doing it before Hegel, and on the other hand is mysterious and the work of experts until someone lays out what's going on).
4
Which aspects of framing do you think he got right?
"In more simplistic terms, one can consider it thus: problem → reaction → solution. Although this model is often named after Hegel, he himself never used that specific formulation. Hegel ascribed that terminology to Kant. Carrying on Kant's work, Fichte greatly elaborated on the synthesis model and popularized it." - Wikipedia; so Hegel deserves less credit than he is originally granted.
4
Interesting.
I don't recall anymore, it's been too long for me to remember enough specifics to answer your question. It's just an impression or cached thought I have that I carry around from past study.
Book Review: Waking Up by Sam Harris
This book aims to convince everyone, even skeptics and athiests, that there is value in some spiritual practises, particularly those related to meditation. Sam Harris argues that mediation doesn't just help with concentration, but can also help us reach transcendental states that reveal the dissolution of the self. It mostly does a good job of what it sets out to do, but unfortunately I didn't gain very much benefit from this book because it focused almost exclusively on persuading you that there is value here,...
1
One of the problems with the general anti zombie principle, is that it makes much too strong a claim that what appears conscious, must be.
6
FWIW no self is a bad reification/translation of not self, and the overhwleming majority seem to be metaphysically confused about something that is just one more tool rather than some sort of central metaphysical doctrine. When directly questioned "is there such a thing as the self" the Buddha is famously mum.
2
What's the difference between no self and not self?
4
No-self is an ontological claim about everyone's phenomenology. Not self is a mental state that people can enter where they dis-identify with the contents of consciousness.
Anti-induction and Self-Reinforcement
Induction is the belief that the more often a pattern happens the more likely it is to continue. Anti-induction is the opposite claim: the more likely a pattern happens the less likely future events are to follow it.
Somehow I seem to have gotten the idea in my head that anti-induction is self-reinforcing. The argument for it is as follows: Suppose we have a game where at each step a screen flashes an A or a B and we try to predict what it will show. Suppose that the screen always flashes A, but the agent initially think...
2
The best reason to believe in anti-induction is that it's never worked before. Discussed at a bit of depth in https://www.lesswrong.com/posts/zmSuDDFE4dicqd4Hg/you-only-need-faith-in-two-things .
I've been thinking about Rousseau and his conception of freedom again because I'm not sure I hit the nail on the head last time. The most typical definition of freedom and that championed by libertarians focuses on an individual's ability to make choices in their daily life. On the more libertarian end, the government is seen as an oppressor and a force of external compulsion.
On the other hand, Rousseau's view focuses on "the people" and their freedom to choose the kind of society that they want to live in. Instead of being se...
2
Can you make this a little more explicit? France is a pretty nice place - are you saying that the counterfactual world where there was no revolution would be significantly better?
2
All the guillotining. And the necessity of that was in part justified with reference to Rousseau's thought
2
Sure. I'm asking about the "we all saw how that worked out" portion of your comment. From what I can see, it worked out fairly well. Are you of the opinion that the French Revolution was an obvious and complete utilitarian failure?
2
I haven't looked that much into French history, just think it is important to acknowledge where that line of thought can end up.
Review: Human-Compatible by Stuart Russell
I wasn't a fan of this book, but maybe that's just because I'm not in the target audience. As a first introduction to AI safety I recommend The AI Does Not Hate You by Tom Chivers (facebook.com/casebash/posts/10100403295741091) and for those who are interested in going deeper I'd recommend Superintelligence by Nick Bostrom. The strongest chapter was his assault on arguments against those who think we shouldn't worry about superintelligence, but you can just read it here: https://spectrum.ie...
Book Review: Communist Manifesto
“The history of all hitherto existing society is the history of class struggles. Freeman and slave, patrician and plebeian, lord and serf, guild-master and journeyman, in a word, oppressor and oppressed, stood in constant opposition to one another, carried on an uninterrupted, now hidden, now open fight, that each time ended, either in the revolutionary reconstitution of society at large, or in the common ruin of the contending classes”
Overall summary: Given the rise of socialism in recent years, now seemed like an appropriate time to review the Communist Manifesto. At times I felt that Marx’s writing was keenly insightful, at other times I felt he was in ignorance of basic facts and at other times I felt that he held views that were reasonable at the time, but for which the flaws are now obvious. In particular, I found the first-half much more engaging than I expected because, say what you like about Marx, he’s an engaged and poetic writer. Towards the end, the focused shifted into particular time-bounded political disputes for which I neither had the knowledge to understand nor the interest to acquire. At the start, I fe...
9
It is not obvious to me from reading the text whether you are aware of the distinction between "private property" and "personal property" in Marxism. So, just to make sure: "private property" refers to the means of production (e.g. a factory), and "personal property" refers to things that are not means of production (e.g. a house where you live, clothes, food, toys).
The ownership of "private property" should be collectivized (according to Marx/ists), because... simply said, you can use the means of production to generate profit, then use that profit to buy more means of production, yadda yadda, the rich get exponentially richer on average and the poor get poorer.
With "personal property" this effect does not happen; if you have one table and I have two tables, there is no way for me to use this advantage to generate further tables, until I become the table-lord of the planet.
(There seem to be problems with this distinction. For example, things can be used either productively or unproductively; I can use my computer to create software or browse social networks. Some things can be used productively in unexpected ways; even the extra table could be used in a workshop to produce stuff. I am not a Marxist, but I suppose the answer would probably be something like "you are allowed to browse the web on your personal computer, but if we catch you privately producing and selling software, you get shot".)
So, is this the confusion of Marxist terms, or do you mean that today more than 10% of people own means of production? In which sense? (Not sure if Marx would also count indirect ownership, such as having your money in an index fund, which buys shares of companies, which own the means of production.)
Did Marx actually argue for abolishing "personal proprety" (according to his definition, i.e. ownership of houses or food)?
1
Before I even got to your comment, I was thinking “You can pry my laptop out of my cold dead hands Marx!”
Thank you for this clarification on personal vs private property.
3
Thanks for clarifying this terminology, I wasn't aware of this distinction when I wrote this post
4
For many people nowadays, their own brain is their means of production, often assisted by computers and their software, but those are cheap compared what what can be earned by using them. Marx did not know of such things, of course, but how do modern Marxists view this type of private ownership of means of production? For that matter, how did Marx view a village cobbler who owned his workshop and all his tools? Hated exploiter of his neighbours? How narrow was his motte here?
4
https://www.marxists.org/archive/marx/works/1847/communist-league/1850-ad1.htm
I once talked about this with a guy who identified as a Marxist, though I can't say how much his opinions are representative for the rest of his tribe. Anyway... he told me that in the trichotomy of Capital / Land / Labor, human talent is economically most similar to the Land category. This is counter-intuitive if you take the three labels literally, but if you consider their supposed properties... well, it's been a few decades since I studied economics, but roughly:
The defining property of Capital is fungibility. You can use money to buy a tech company, or an airplane factory, or a farm with cows. You can use it to start a company in USA, or in India. There is nothing that locks money to a specific industry or a specific place. Therefore, in a hypothetical perfectly free global market, the risk-adjusted profit rates would become the same globally. (Because if investing the money in cows gives you 5% per annum, but investing money in airplanes gives you 10%, people will start selling cow farms and buying airplane factories. This will reduce the number of cow farms, thus increasing their profit, and increase the competition in the airplane market, thus reducing their profi...
9
IANAM[1], but intuitively it seems to me that an exception ought to be made (given the basic idea of Marxist theory) for individuals who own means of production the use of which, however, does not involve any labor but their own.
So in the case of the village cobbler, sure, he owns the means of production, but he’s the only one mixing his labor with the use of those tools. Clearly, he can’t be exploiting anyone. Should the cobbler take on an assistant (continuing my intuitive take on the theory), said assistant would presumably have to now receive some suitable share in the ownership of the workshop/tools/etc., and in the profits from the business (rather than merely being paid a wage), as any other arrangement would constitute alienation from the fruits of his (the assistant’s) labor.
On this interpretation, there does not here seem to be any contradiction or inconsistency in the theory. (I make no comment, of course, on the theory’s overall plausibility, which is a different matter entirely.)
----------------------------------------
1. I Am Not A Marxist. ↩︎
What does it mean to define a word? There's a sense in which definitions are entirely arbitrary and what word is assigned to what meaning lacks any importance. So it's very easy to miss the importance of these definitions - emphasising a particular aspect and provides a particular lense with which to see the world.
For example, if define goodness as the ability to respond well to others, it emphasizes that different people have different needs. One person may want advice, while another simple encouragement. Or if we define love as acceptance of the other, it suggests that one of the most important aspects of love is the idea that true love should be somewhat resilient and not excessively conditional.
Here's one way of explaining this: it's a contradiction to have a provable statement that is unprovable, but it's not a contradiction for it to be provable that a statement is unprovable. Similarly, we can't have a scenario that is simultaneously imagined and not imagined, but we can coherently imagine a scenario where things exist without being imagined by beings within that scenario.
If I can imagine a tree that exists outside of any mind, then I can imagine a tree that is not being imagined. But "an imagined X that i...
2
Inverted, by switching "provable" and "unprovable":
It's a contradiction to have an unprovable statement that is provable, but it's not a contradiction for it to be unprovable that a statement is provable.
3
"It's a contradiction to have a provable statement that is unprovable" - I meant it's a contradiction for a statement to be both provable and unprovable.
"It's not a contradiction for it to be provable that a statement is unprovable" - this isn't a contradiction
4
You made a good point, so I inverted it. I think I agree with your statements in this thread completely. (So far, absent any future change.) My prior comment was not intended to indicate an error in your statements. (So far, in this thread.)
If there is a way I could make this more clear in the future, suggestions would be appreciated.
Elaborating on my prior comment via interpretation, so that it's meaning is clear, if more specified*:
A' is the same as A because:
While B is true, B' seems false (unless I'm missing something). But in a different sense B' could be true. What does it mean for something to be provable? It means that 'it can be proved'. This gives two definitions:
* a proof of X "exists"
* it is possible to make a proof of X
Perhaps a proof may 'exist' such that it cannot exist (in this universe). That as a consequence of its length, and complexity, and bounds implied by the 'laws of physics'* on what can be represented, constructing this proof is impossible. In this sense, X may be true, but if no proof of X may exist in this universe, then:
Something may have the property that it is "provable", but impossible to prove (in this universe).**
*Other interpretations may exist, and as I am not aware of them, I think they'd be interesting.
**This is a conjecture.
4
Thanks for clarifying
As I wrote before, evidential decision theory can be critiqued for failing to deal properly with situations where hidden state is correlated with decisions. EDT includes differences in hidden state as part of the impact of the decision, when in the case of the smoking lesion, we typically want to say that it is not.
However, Newcomb's problem also has hidden state is correlated with your decision. And if we don't want to count this when evaluating decisions in the case of the Smoking Lesion, perhaps we shouldn't count this in the case of Newc...
5
Does FDT make this any clearer for you?
There is a distinction in the correlation, but it's somewhat subtle and I don't fully understand it myself. One silly way to think about it that might be helpful is "how much does the past hinge on your decision?" In smoker's lesion, it is clear the past is very fixed—even if you decide to not to smoke, that doesn't affect the genetic code. But in Newcomb's, the past hinges heavily on your decision: if you decide to one-box, it must have been the case that you could have been predicted to one-box, so it's logically impossible for it to have gone the other way.
One intermediate example would be if Omega told you they had predicted you to two-box, and you had reason to fully trust this. In this case, I'm pretty sure you'd want to two-box, then immediately precommit to one-boxing in the future. (In this case, the past no longer hinges on your decision.) Another would be if Omega was predicting from your genetic code, which supposedly correlated highly with your decision but was causally separate. In this case, I think you again want to two-box if you have sufficient metacognition that you can actually uncorrelate your decision from genetics, but I'm not sure what you'd do if you can't uncorrelate. (The difference again lies in how much Omega's decision hinges on your actual decision.)
2
Yeah, FDT has a notion of subjunctive dependence. But the question becomes what does this mean? What precisely is the difference between the smoking lesion and Newcombs? I have some ideas and maybe I'll write them up at some point.
Writing has been one of the best things for improving my thinking as it has forced me to solidify my ideas into a form that I've been able to come back to later and critique when I'm less enraptured by them. On the other hand, for some people it might be the worst thing for their thinking as it could force them to solidify their ideas into a form that they'll later feel compelled to defend.
Despite having read dozens of articles discussing Evidential Decision Theory (EDT), I've only just figured out a clear and concise explanation of what it is. Taking a step back, let's look at how this is normally explained and one potential issue with this explanation. All major decision theories (EDT, CDT, FDT) rate potential decisions using expected value calculations where:
- Each theory uses a different notion of probability for the outcomes
- Each theory uses the same utility function for valuing the outcomes
So it should be just a simple matter...
Book Review: Awaken the Giant Within Audiobook by Tony Robbins
First things first, the audiobook isn't the full book or anything close to it. The standard book is 544 pages, while the audiobook is a little over an hour and a half. The fact that it was abridged really wasn't obvious.
We can split what he offers into two main categories: motivational speaking and his system itself. The motivational aspect of his speaking is very subjective, so I'll leave it to you to evaluate yourself. You can find videos of his on Youtube and you should know wi...
Book Review: The Rosie Project:
Plot summary: After a disastrous series of dates, autistic genetics professor Don Tilman decides that it’d be easier to just create a survey to eliminate all of the women who would be unsuitable for him. Soon after, he meets a barmaid called Rosie who is looking for help with finding out who her father is. Don agrees to help her, but over the course of the project Don finds himself increasingly attracted to her, even though the survey suggests that he is completely unsuitable. The story is narrated in Don’s voice. He tells us...
2
Doublechecking, this is fiction?
3
Yep, fiction
Book Review: So Good They Can't Ignore You by Cal Newport:
This book makes an interesting contrast to The 4 Hour Workweek. Tim Ferris seems to believe that the purpose of work should be to make as much money as possible in the least amount of time and that meaning can then be pursued during your newly available free time. Tim gives you some productivity tips in the hope that it will make you valuable enough to negotiate flexibility in terms of how, when and where you complete your work, plus some dirty tricks as well.
Cal Newport's book is similar in that it focuses on becoming valuable enough to negotiate a job that you'll love and downplays the importance of pursuing your passions in your career. However, while Tim extolls the virtues of being a digital nomad, Cal Newport emphasises self-determination theory and autonomy, competence and relatedness. That is, the freedom to decide how you pursue your work, the satisfaction of doing a good job and the pleasure of working with people who you feel connected to. He argues that these traits are rare and valuable and so that if you want such a job you'll need skills that rare and valuable to offer in return.
That's...
8
FWIW I think this and maybe some of the other book review shortforms you've done would make fine top level posts.
2
Thanks, I'll think about it. I invested more effort in this one, but for some of the others I was optimising for speed
8
+1 for book-distillation, probably the most underappreciated and important type of post.
Book Review: The 4 Hour Workweek
This is the kind of book that you either love or hate. I found value in it, but I can definitely understand the perspective of the haters. First off: the title. It's probably one of the most blatant cases of over-promising that I've ever seen. Secondly, he's kind of a jerk. A number of his tips involve lying and in school he had a strategy of interrogating his lecturers in detail when they gave him a bad mark so that they'd think very carefully assigning him a bad grade. And of course, while drop-shipping might have been an underexploited strategy at the time when he wrote the book, it's now something of a saturated market.
On the plus side, Tim is very good at giving you specific advice. To give you the flavour, he advises the following policies for running an online store: avoid international orders, no expedited or overnight shipping, two options only - standard and premium; no cheque or Western union, no phone number if possible, minimum wholesale order with tax id and faxed in order form, ect. Tim is extremely process oriented and it's clear that he has deep expertise here and is able to share it unusually well. I fo...
Book Review: Civilization and its discontents
Freud is the most famous psychologist of all time and although many of his theories are now discredited or seem wildly implausible, I thought it'd be interesting to listen to him to try and understand why it sounded plausible in the first place.
At times Freud is insightful and engaging; at other times, he falls into psychoanalytic lingo in such a way that I couldn't follow what he was trying to say. I suppose I can see why people might have assumed that the fault was with their failure to understand.
It's a short read, so if you're curious, there isn't that much cost to going ahead and reading it, but this is one of those rare cases where you can really understand the core of what he was getting at from the summary on Wikipedia (https://en.m.wikipedia.org/wiki/Civilization_and_Its_Discontents)
Since Wikipedia has a summary, I'll just add a few small remarks. This book focuses on a key paradox; our utter dependence on it for anything more than the most basic survival; but how it requires us to repress our own wants and desires so as to fit in with an ordered society. I find this to be an interesting answer to t...
I think I spent more time writing this than reading the book, as I find reviewing fiction much more difficult. I strongly recommend this book: it doesn't take very long to read, but you may spend much longer trying to figure out what to make of it.
Book Review: The Stranger by Camus (Contains spoilers)
I've been wanting to read some existentialist writing for a while and it seemed reasonable to start with a short book like this one. The story is about a man who kills a man for what seems to be no real reason at all and who is then subsequently arrested and m
I really like the short-form feature because after I have articulated a thought my head feels much clearer. I suppose that I could have tried just writing it down in a journal or something; but for some reason I don't feel quite the same effect unless I post it publicly.
The sad thing about philosophy is that as your answers become clearer, the questions become less mysterious and awe-inspiring. It's easy to assume that an imposing question must have an impressive answer, but sometimes the truth is just simple and unimpressive and we miss this because we didn't evolve for this kind of abstract reasoning.
2
Examples?
5
I used to find the discussion of free will interesting before I learned it was just people talking past each other. Same with "light is both a wave and a particle" until I understood that it just meant that sometimes the wave model is a good approximation and other times the particle model is. Debates about morality can be interesting, but much less so if you are a utilitarian or non-realist.
2
Semantic differences almost always happen, but are rarely the only problem.
There are certainly different definitions of free will, but even so problems, remain:-
There is still an open question as to whether compatibilist free will is the only kind anyone ever needed or believed in, and as to whether libertarian free will is possible at all.
5
The topic is interesting, but no discussion about it is interesting. These are not contradictory.
The open question about strong determinism vs libertarian free will is interesting, and there is a yet-unexplained contradiction between my felt experience (and others reported experiences) and my fundamental physical model of the universe. The fact that nobody has any alternative model or evidence (or even ideas about what evidence is possible) that helps with this interesting question makes the discussion uninteresting.
1
So Yudkowsky's theory isn't new?
2
Not new that I could tell - it is a refreshing clarity for strict determinism - free will is an illusion, and "possible" is in the map, not the territory. "Deciding" is how a brain feels as it executes it's algorithm and takes the predetermined (but not previously known) path.
He does not resolve the conflict that it feels SOOO real as it happens.
1
That's an odd thing to say since the feeling of free will is about the only thing be addresses.
This is the first classic that I’m reviewing. One of the challenges with figuring out which classics to read is that there are always people speaking very highly of it and in a vague enough manner that it makes it hard for you to decide whether to read it. Hopefully I can avoid this trap.
Book Review: Animal Farm
You probably already know the story. In a thinly veiled critique of the Russian Revolution, the animals in a farm decide to revolt against the farmer and run the the farm themselves. At start, the seven principles of Animalism are idealistically declared, but as time goes on, things increasingly seem to head downhill…
Why is this a classic?: This book was released at a time when the intellectual class was firmly sympathetic to the Soviets, ensuring controversy and then immortality when history proved it right.
Why you might want to read this: Short (only 112 pages or 3:11 on Audible), the story always moves along at a brisk pace, the writing is engaging and a few very emotionally impactful moments. The broader message of being wary of the promises made by idealistic movements still holds (especially "all animals are equal, but some animals are more equal than others"...
Wow, I've really been flying through books recently. Just thought I should mention that I'm looking for recommendations for audio books; bonus points for books that are short. Anyway....
Book Review: Zero to One
Peter Thiel is the most famous contrarian in Silicon Valley. I really enjoyed hearing someone argue against the common wisdom of the valley. Most people think in terms of beating the competition; Thiel thinks in terms of establishing a monopoly so that there is no competition. Agile methodology and the lean startup are all the rage, but Thiel argues that this only leads to incremental improvements and that truly changing the world requires you to commit to a vision. Most companies was to disrupt your competitors, but for Thiel this means that you've fallen into competition, instead of forging your own unique path. Most venture funds aim to diversify, but Thiel is more selective, only investing in companies that have billion dollar potential. Many startups spurn marketing, but Thiel argues that this is dishonest and that PR is also a form of marketing, even if that isn't anyone's job title. Everyone is betting on AI replacing humans, while Thiel is mor...
As I said before, I'll be posting book reviews. Please let me know if you have any questions and I'll answer them to the best of my ability.
Book Review: The AI does not hate you by Tom Chivers
The title of this book comes from a quote by Elizier Yudkowsky which reads in full: "The AI does not hate you, nor does it love you, but you are made of atoms which it can use of something else". This book covers not only potential risks from AI, but the rationalist community from which this evolved and also touches on the effective altruism movement.
This book fills something of a gap in the book market; when people are first learning about existential risks from AI I usually recommend the two-part Wait by Why post (https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html) and then I'm not really sure what to recommend next. The sequences are ridiculously long and Bostrom's Superintelligence is a challenging read for those not steeped in philosophy and computer science. In contrast, this book is much more accessible and provides the right level of detail for a first introduction, rather than someone who has already decided to try entering the field.
I mostly listened to this boo
I'm going to start writing up short book reviews as I know from past experience that it's very easy to read a book and then come out a few years later with absolutely no knowledge of what was learned.
Book Review: Everything is F*cked: A Book About Hope
To be honest, the main reason why I read this book was because I had enjoyed his first and second books (Models and The Subtle Art of Not Giving A F*ck) and so I was willing to take a risk. There were definitely some interesting ideas here, but I'd already received many of these through other s...
One thing I'm finding quite surprising about shortform is how long some of these posts are. It seems that many people are using this feature to indicate that they've just written up these ideas quickly in the hope that the feedback is less harsh. This seems valuable; the feedback here can be incredibly harsh at times and I don't doubt that this has discouraged many people from posting.
3
Can confirm. I don't post on normal lesswrong because the discourse is brutal.
7
I pushed a bit for the name 'scratchpad' so that this use case was a bit clearer (or at least not subtly implied as "wrong"). Shortform had enough momentum as a name that it was a bit hard to change tho. (Meanwhile, I settled for 'shortform means either the writing is short, or it took a (relatively) short amount of time to write)
4
“I’m sorry, I didn’t have the time to write you a short email, so I wrote you a long one instead.”
