Weighing in here because this is a suboptimality I've often encountered when speaking with math oriented interlocutors (including my past self):
The issue here is an engineering problem, not a proof problem. Human minds tend to require lots of cognitive resources to take provisional definitions for things that have either no definition or drastically different definitions in their minds outside this specific context.
Structuring your argument as a series of definitions is fine when making a proof in a mathematical language, since comprehensibility is not a terminal goal, and (since each inferential step can be trusted and easily verified as such) not a high-priority instrumental goal either.
But when you're trying to accurately convey... (read more)
This is a difficult line to thread, since while I can't be sure which awakening experiences you're opposed to in particular (incidentally, see the later paragraphs re: variations between them), as a general category they seem to be the consequence of your intuitive world-model losing a mysterious "self" node to be replaced with a more gears-like representation of internal mental states and their mechanisms.
However, you might be able to make it more difficult to "look" in that direction by using vipassana-style meditations with limited time. This should lead you to disproportionately 'collapse' your anxiety and other imprinted/background thought patterns and intrusive thoughts, which would start out clamoring for your attention, and not... (read more)
I liked the parts about Moloch and human nature at the beginning, but the AI aspects seem to be unfounded anthropomorphism, applying human ideas of 'goodness' or 'arbitrarity [as an undesirable attribute]' despite the existence of anti-reasons for believing them applicable to non-human motivation.
... (read 547 more words →)But I think another scenario is plausible as well. The way the world works is… understandable. Any intelligent being can understand Meditations On Moloch or Thou Art Godshatter. They can see the way incentives work, and the fact that a superior path exists, one that does not optimize for a random X while grinding down all others. Desperate humans in broken systems might not be able to do much with that information, but
The main advantage of Intelligence Augmentation is that we know that our current minds are both generally or near-generally intelligent and more-or-less aligned with our values, and we also have some level of familiarity with how we think (edit: and likely must link our progress in IA to our understanding of our own minds, due to the neurological requirements).
So we can find smaller interventions that are certainly, or at least almost certainly, going to have no effect on our values, and then test them over long periods of time, using prior knowledge of human psychology and the small incremental differences each individual change would make to identify value drift without worrying about... (read 600 more words →)
"Like if we increased yearly economic growth by 5% (for example 2% to 2.1%), what effect would you expect that to have?"
From my personal experience, academics have a tendency and preference to work on superficially-beneficial problems; Manhattan Projects and AI alignment groups both exist (detrimental and non-obviously beneficial, respectively), but for the most part we have projects like eco-friendly technology and efficient resource allocation in specified domains.
Due to this, greater economic growth means more resources to bring to bear for other scientific/engineering problems, due to research on superficially-beneficial subjects like power-generation, efficiency, quantum computing, etc. As noted in my previous comment, the economic growth (and these increased resources as well) will also... (read 628 more words →)
"Working on global poverty seems unlikely to be a way of increasing our chances of succeeding at alignment. If anything, this would likely increase both the number of future alignment and capacity researchers. So it's unlikely to significantly increase our chances."
A fair point regarding alignment (I hadn't thought about how it would affect AI researchers as well), but I was more thinking from the perspective of X-risk in general.
AI alignment is one issue that doesn't seem to be significantly affected either way by this, but we also have things like alignment of organizations towards public interest (which is currently a fragile, kludged-together combination of laws and occasional consumer/citizenry strikes) or the increasing... (read more)
One method would be to take advantage of low-hanging fruit not directly related to X-risk. Clearly motivation isn't enough to solve these problems (and I'm not just talking about alignment), so we should be trying to optimize all our resources, and that includes getting rid of major bottlenecks like [the imagined example of] hunger killing intelligent, benevolent potential-researchers in particular areas because of a badly-designed shipping route.
A real-life example of this would be the efforts of the Rationalist community to promote more efficient methods of non-scientific analysis (i.e. cases where you don't have the effort required for scientific findings, but want a right answer anyway). This helps not only in X-risk efforts,... (read more)
As a Babble this is excellent, and many of these (e.g. optimizing income streams, motivating/participating-in groups) seem to be necessary prerequisites for being in a position to make progress on X-risk problems.
But I think the nature of such problems (as ones that have been attempted by many other individuals with at least some centralized organizations where these individuals share their experiences to avoid duplication of effort, that is) means that any undirected Babble will primarily encounter lines of inquiry that have already been addressed, as many of the more direct (non-resource-gathering) suggestions seem to be.
As a point of methodology, I would suggest trying for much larger Babble lists when approaching these problems, perhaps on the scale of a few hundred ideas, or alternatively making multiple recursive layers of Babbles for each individual point at every recursive level (e.g. 100 points, each with 100 points, each with 100 points...), so that the process is more likely to produce unique [and thus useful] approaches.
The main issue with AGI Alignment is that the AGI is more intelligent than us, meaning that making it stay within our values requires both perfect knowledge of our values and some understanding of how to constrain it to share them.
If this is truly an intractable problem, it still seems that we could escape the dilemma by focusing on efforts in Intelligence Augmentation, e.g. through Mind Uploading and meaningful encoding/recoding of digitized mind-states. Granted, it currently seems that we will develop AGI before IA, but if we could shift focus enough to reverse this trend, then AGI would not be an issue, as we ourselves would have superior intelligence to our creations.
I’ve been looking into intuition pumps recently, and I realized the MBTI system (when combined with the associated ‘function theory’) has some very general, relatively useful ones, such as the INTP (TiNe) and INTJ (NiTe) ones.
However, the mental structures each of these describe are mutually incompatible in terms of information representation and processing methods. I haven’t yet found any way to possess such mutually incompatible systems at the same time (even disregarding the infeasibility of true multitasking, simply having them both as active components in my mind, rather than inactive memories only accessible when specifically called upon, is proving difficult)
Does anyone have tips on how to keep such mentalities both available for active use despite their incompatibility?
I was going to note that this seems to be the social-interaction special-case of policy utilitarianism (which I've used for years and would attribute to giving me some quality-of-life improvements).
However, from a quick google search it seems "policy utilitarianism" doesn't exist, and I have no idea what this concept is actually called, assuming I didn't make it up.
In short, it's a mix of functional decision theory, (rule) utilitarianism, and psychology (and possibly some Buddhism), along with some handwaving for the Hard Problem of learning under bounded rationality (which I'd assert the brain is good enough at to not need an explicit algorithm for it in a human ethical framework).
To go into the... (read 687 more words →)