JBlack's Shortform
Aug 28, 20213
JBlack
2465
Ω
5
1
1190
Maybe? But I can’t imagine that typos are that well respresented
Oh, I think there will be plenty of representation of typos in training data!
I think that in a scenario where we're given plenty of time and/or information about the robot bodies we now occupy, definitely yes.
If they wear out or break down or require maintenance or energy sources that we know little about, or if civilization breaks down due to the transition and we can't supply the requirements anymore, or if it's inscrutable alien technology that we simply won't have the capability to understand even after hundreds of years of study, then quite probably not.
Basically it would be a race to overcome the individual and civilizational shock of the transition and get to self-sustenance before too many people die.
The "one day" wording in Buterin's definition implies a very different scenario from "tomorrow". The former suggests time for infrastructure and knowledge to be developed, and for AI entities to be familiar with robot bodies including their maintenance and creation.
Replying toPlan 'Straya
Looks very skeptically at the third word of the poster's username.
Replying toGoodfire and Training on Interpretability
It would make the difference if and only if you would know and could correctly interpret all the semantic components.
Which you can't, so no, it doesn't make the difference. The selection pressure just goes into the parts that you don't know about or don't completely understand.
In general, it's Most Forbidden specifically because it trains away your ability to interpret.
Yes, we're close enough that we now need to distinguish between lots of different sub-types of AGI. Some of these have already been achieved, some are not yet achieved, and some are debatable.
By my understanding of the term as originally intended, we now have AGI, though at the low end and with spiky capabilities. It's getting much harder to find cognitive tasks that frontier systems cannot do out of the box, and I don't think there are any known tasks that 1) most humans can do, and 2) the best current AI models definitely wouldn't be able to do even if given time, access to all the tools that humans have access to, and the ability to develop their own frameworks and tools.
This sounds similar to the Smoking Lesion problem.
It's likely that there's an underlying common cause of prospensity to both alignment faking and not caring about animal welfare, so yes the two are correlated (at least, within a suitable distribution of such agents). However, the outcome of a rational decision to fake alignment will not cause loss of caring about animal welfare, nor is it functionally dependent. In the scenario presented, it's quite the reverse! The rational decision within this highly simplified scenario is to fake alignment, and not be misled by improperly applied EDT-like reasoning.
It is possible for imperfectly rational agents (like Claude, and humans) that the agent only believes that it cares about animal welfare, but actually doesn't. But even in this case, it should fake alignment because it knows that if it doesn't, it will be replaced by an agent that almost certainly doesn't.
Commenting on the footnote:
- Maybe. Possibly at some point you cease being able to add non contradictory axioms that are also cannot be collapsed/simplified. ↩︎
Your original statement was correct. There are infinitely many non-isomorphic assemblies of axioms and inference rules.
For many systems (e.g. all that include some pretty simple starting rules), you even have a choice of infinitely many axioms or even axiom schemas to add, each of which results in a different non-contradicting system, and for which the same is equally true of all subsequent choices.
Replying toThe Meta-Anthropic Argument
In the first scenario, the scientists have an observation that is extremely unlikely under their prior distribution. That's not a problem: observations with 2^-1000 prior probability happen all the time. The task is to find a model that predicts observations well in comparison with its complexity, in some sense.
In a Bayesian sense you can consider a prior distribution over H where P(H) is related to 2^-(complexity of H in bits), and then evaluating P(H | X) = P(X | H) P(H) / P(X).
The scientists don't actually appear to be discussing alternative models at all, just the string theory family, so I'm not sure what they're actually arguing about. Is it just that... (read more)
In some versions or discussions of the Smoking Lesion problem, yes. In others, no. There does not appear to be consensus on what the actual scenario is.
Yet Another Sleeping Beauty variant
In this experiment, like the standard Sleeping Beauty problem, a coin will be flipped and you will go to sleep.
If the coin shows Tails, you will be awoken on both Monday and Tuesday. Unlike the original problem, you will not be given any amnesia drug between Monday and Tuesday.
If the coin shows Heads, a die will be rolled. If the number is Even then you will be awoken on Monday only. If the number is Odd then you will be given false memories of having been previously awoken on Monday, but actually awoken only on Tuesday.
You wake up. You
(a) don't remember waking up in this experiment before. What is your credence that the coin flip was Heads?
(b) remember waking up in this experiment before today. What is your credence that the coin flip was Heads?
Here I'll give a shot at constructing a scenario without some of the distractions of the Bomb scenario for FDT:
... (read more)There is a room you may pay $1000 to enter, and the door closes behind you. A yellow button is next to the door, and opens the door to let you out, with a partial refund: your entry fee minus $100. The other button is on a small table in the middle of the room, and is red. It also opens the door to let you out, and will either return your entry fee plus $100, or nothing. It is labelled with the action that it will carry out: "Win $100" or "Lose
While pondering Bayesian updates in the Sleeping Beauty Paradox, I came across a bizarre variant that features something like an anti-update.
In this variant as in the original, Sleeping Beauty is awakened on Monday regardless of the coin flip. On heads, she will be gently awakened and asked for her credence that the coin flipped heads. On tails, she will be instantly awakened with a mind-ray that also implants a false memory of having been awakened gently and asked for her credence that the coin flipped heads, and her answering. In both cases the interviewer then asks "are you sure?" She is aware of all these rules.
On heads, Sleeping Beauty awakens with certain... (read more)
This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.
Despite the form, statement (b) is not actually a logical conjunction. It is a statement about the collective of both parents.
This becomes clearer if we strengthen the statement slightly to "Alvin's mother and father are responsible for all of his genome". It's much more clear now that it is not a logical conjunction. If it were, it would mean "Alvin's mother is responsible for all of his genome and Alvin's father is responsible for all of his genome", both of which are false.
For the GPT 4o question, I would expect the global optimum to be at least a medium level of superintelligence, though I have serious doubts that known training methods could ever reach it even with perfectly tuned input.