Tetraspace — LessWrong

LESSWRONG
LW

Tetraspace1y

Just saw this and can confirm it was one of the best times of my life.

Tetraspace1y

Dominance/submission dynamics in relationships

In Act I outputs Claudes do a lot of this, e.g. this screenshot of Sonnet 3.6

Replying to"Slow" takeoff is a terrible term for "maybe even faster takeoff, actually"

Tetraspace1y

"Slow" takeoff is a terrible term for "maybe even faster takeoff, actually"

Fast/Slow takeoff

-2

-16

Tetraspace1y*

I'd like beta access. My main use case is that I intend to write up some thoughts on alignment (Manifold gives 40% that I'm proud of a write-up, I'd like that number up), and this would be helpful for literature review and finding relevant existing work. Especially so because a lot of the public agent foundations work is old and migrated from the old alignment forum, where it's low-profile compared to more recent posts.

Replying toTen arguments that AI is an existential risk

Tetraspace2y*

Ten arguments that AI is an existential risk

AI isn't dangerous because of what experts think, and the arguments that persuaded the experts themselves are not "experts think this". It would have been a misleading argument for Eliezer in 2000 being among the first people to think about it in the modern way, or for people who weren't already rats in maybe 2017 before GPT was in the news and when AI x-risk was very niche.

I also have objections to its usefulness as an argument; "experts think this" doesn't give me any inside view of the problem by which I can come up with novel solutions that the experts haven't thought of. I think this especially comes up if the... (read more)

Replying toHow I Learned To Stop Trusting Prediction Markets and Love the Arbitrage

Tetraspace2y*

How I Learned To Stop Trusting Prediction Markets and Love the Arbitrage

I learned this lesson looking at the conditional probabilities of candidates winning given they were nominated in 2016, where the candidates with less than about 10% chance of being the nominee had conditional probabilities with noise between 0 and 100%. And this was on the thickly traded real-money markets of Betfair! I personally engage in, and also recommend, just kinda throwing out any conditional probabilities that look like this, unless you have some reason to believe it's not just noise.

Another place this causes problems is in the infinitely-useful-if-they-could-possibly-work decision markets, where you want to be able to evaluate counterfactual decisions, except these are counterfactuals so you don't make the decision so there's no liquidity and it can take any value.

Replying toSimplifying Corrigibility – Subagent Corrigibility Is Not Anti-Natural

Tetraspace2y

Simplifying Corrigibility – Subagent Corrigibility Is Not Anti-Natural

Obeying it would only be natural if the AI thinks that the humans are more correct than the AI would ever be, after gathering all available evidence, where "correct" is given by the standards of the definition of the goal that the AI actually has, which arguendo is not what the humans are eventually going to pursue (otherwise you have reduced the shutdown problem to solving outer alignment, and the shutdown problem is only being considered under the theory that we won't solve outer alignment).

An agent holding a belief state that given all available information it will still want to do something other than the action it will think is best then is anti-natural; utility maximisers would want to take that action.

This is discussed on Arbital as the problem of fully updated deference.

Replying toConditional prediction markets are evidential, not causal

Tetraspace2y

Conditional prediction markets are evidential, not causal

This ends up being pretty important in practise for decision markets ("if I choose to do X, will Y?"), where by default you might e.g. only make a decision if it's a good idea (as evaluated by the market), and therefore all traders will condition on the market having a high probability which is obviously quite distortionary.

I replied on discord that I feel there's maybe something more formalisable that's like:

reality runs on math because, and is the same thing as, there's a generalised-state-transition function
because reality has a notion of what happens next, realityfluid has to give you a notion of what happens next, i.e. it normalises
the idea of a realityfluid that doesn't normalise only comes to mind at all because you learned about R^n first in elementary school instead of S^n

which I do not claim confidently because I haven't actually generated that formalisation, and am posting here because maybe there will be another Lesswronger's eyes on it that's like "ah, but...".

Replying toShutdown-Seeking AI

Tetraspace3y

Shutdown-Seeking AI

Not unexpected! I think we should want AGI to, at least until it has some nice coherent CEV target, explain at each self-improvement step exactly what it's doing, to ask for permission for each part of it, to avoid doing anything in the process that's weird, to stop when asked, and to preserve these properties.

Even more recently I bought a new laptop. This time, I made the same sheet, multiplied the score from the hard drive by $\frac{2}{3}$ because 512 GB is enough for anyone and that seemed intuitively the amount I prioritised extra hard drive space compared to RAM and processor speed, and then looked at the best laptop before sharply diminishing returns set in; this happened to be the HP ENVY 15-ep1503na 15.6" Laptop - Intel® Core™ i7, 512 GB SSD, Silver. This is because I have more money now, so I was aiming to maximise consumer surplus rather than minimise the amount I was spending.^[1]

Surprisingly, it came with a touch screen! That's just the kind... (read more)

, as a mathematical structure, is smarter than god and perfectly aligned to $U$ ; the value of $arg max U$ will never actually be $arg max V$ because $V$ is more objectively rational, or because you made a typo and it knows you meant to say $arg max V$ ; and no matter how complicated the mapping is from $a$ to $U (a)$ it will never fall short of giving the $a$ that gives the highest value of $U$ .

Which is why in principle you can align a superior being, like $arg max$ , or maybe like a superintelligence.

"The AI does our alignment homework" doesn't seem so bad - I don't have much hope for it, but because it's a prosaic alignment scheme so someone trying to implement it can't constrain where Murphy shows up, rather than because it's an "incoherent path description".

A concrete way this might be implemented is

A language model is trained on a giant text corpus to learn a bunch of adaptations that make it good at math, and then fine-tuned for honesty. It's still being trained at a safe and low level of intelligence where honesty can be checked, so this gets a policy that produces things that are mostly honest on easy questions and sometimes

... (read more)

Arbital gives a distinction between "logical decision theory" and "functional decision theory" as:

Logical decision theories are a class of decision theories that have a logical counterfactual (vs. the causal counterfactual that CDT has and the evidential counterfactual EDT has).
Functional decision theory is the type of logical decision theory where the logical counterfactual is fully specified, and correctly gives the logical consequences of "decision function X outputs action A".

More recently, I've seen in Decision theory does not imply that we get to have nice things:

Logical decision theory is the decision theory where the logical counterfactual is fully specified.
Functional decision theory is the incomplete variant of logical decision theory where the logical consequences of "decision function X outputs action A" have to be provided by the setup of the thought experiment.

Any preferences? How have you been using it?

Is there a "coherent decisions imply consistent utilities"-style argument for non-lexicographic preferences?

Tetraspace

Tetraspace, johnswentworth

The post Coherent decisions imply consistent utilities demonstrates some situations in which an agent that isn't acting as if it is maximizing a real-valued utility function over lotteries is dominated by one that does, and promised that this applies in general.

However, one intuitively plausible way to make decisions that doesn't involve a real-valued utility and that the arguments in the post don't seem to rule out is to have lexicographic preferences; say, each lottery has payoffs represented as a sequence $(u_{1}, u_{2}, u_{3}, \dots)$ and you compare them by first comparing $u_{1}$ , and if and only if their $u_{1}$ s are the same compare $u_{2}$ , and so on, with probabilities multiplying through by each $u_{n}$ and payoffs being added element-wise. The VNM axioms exclude this by requiring continuity, with a payoff evaluated like this violating it because $(0, 0, \dots) ≺ (0, 1, \dots) ≺ (1, 0, \dots)$ but there is no probability $p$ for which a $p$ probability of a $(1, 0, \dots)$ payoff and a $1 - p$ probability of a $(0, 0, \dots)$ payoff is as exactly as good as a certainty of a $(0, 1, \dots)$ payoff.

Are there coherence theorems that exclude lexicographic preferences like this also?

Recently I bought a new laptop

Tetraspace

128 GB of hard drive space, I found, was not enough to contain both the build tools for every random programming language I might want to try out and every main series Paradox grand strategy game. So I decided to get a new laptop. Also the bottom half of my screen was completely obscured by screen problems upon booting up the laptop, so it took a few minutes to "warm up" and be usable. Also I came close to accidentally my old laptop trying to install Linux. Also the casing seemed to be coming a bit loose, but that was probably fine, it just does that.

One problem, a big problem, that might... (read 635 more words →)

Smarkets is currently selling shares in Trump conceding if he loses at 57.14%. The Good Judgement Project's superforecasters predict that any major presidential candidate will concede with probability 88%. I assign <30% probability to Biden conceding* (scenarios where Biden concedes are probably overwhelmingly ones where court cases/recounts mean states were called wrong, which Betfair assigns ~10% probability to, and FTX kind of** assigns 15% probability to, and even these seem high), so I think it's a good bet to take.

* I think that the Trump concedes if he loses market is now unconditional, because by Smarkets' standards (projected electoral votes from major news networks) Biden has won.

** Kind of, because some TRUMP... (read more)

I have two questions on Metaculus that compare how good elements of a pair of cryonics techniques are: preservation by Alcor vs preservation by CI, and preservation using fixatives vs preservation without fixatives. They are forecasts of the value (% of people preserved with technique A who are revived by 2200)/(% of people preserved with technique B who are revived by 2200), which barring weird things happening with identity is the likelihood ratio of someone waking up if you learn that they've been preserved with one technique vs the other.

Interpreting these predictions in a way that's directly useful requires some extra work - you need some model for turning the ratio P(revival|technique... (read more)

You Can Do Futarchy Yourself

Tetraspace

Long ago, in the year 2000, Robin Hanson raised the question of whether we should vote on values and bet on beliefs. He proposes a form of government called futarchy, in which people vote on values and bet on beliefs.

Under futarchy, values experts propose measures of global welfare, policy-writing experts write up policies, and policy-evaluation experts bet on the outcome of those policies (in principle, anyone could do any of these things - but politics is valuable enough that I imagine experts will occupy these three roles). People vote on the measure of global welfare used, and the policies that have the best expected outcome by inputting the results of the... (read 1357 more words →)

PMarket Maker

Just under a month ago, I said "web app idea: one where you can set up a play-money prediction market with only a few clicks", because I was playing around on Hypermind and wishing that I could do my own Hypermind. It then occurred to me that I can make web apps, so after getting up to date on modern web frameworks I embarked in creating such a site.

Anyway, it's now complete enough to use, provided that you don't blow on it too hard. Here it is: pmarket-maker.herokuapp.com. Enjoy!

You can create a market, and then create a set of options within that market. Players can make buy and sell limit orders... (read more)

Thoughts on Ryan Carey's Incorrigibility in the CIRL Framework (I am going to try to post these semi-regularly).

This specific situation looks unrealistic. But it's not really trying to be too realistic, it's trying to be a counterexample. In that spirit, you could also just use $R_{2} (a, s_{d}) = 1000$ , which is a reward function parametrized by $θ$ that gives the same behavior but stops me from saying "Why Not Just set $θ = - 1$ ", which isn't the point.

How something like this might actually happen: you try to have your $R_{1}$ be a complicated neural network that can approximate any function. But you butcher the implementation and get something basically random instead, and this $R_{2}$ cannot approximate

... (read more)

Life 3.0 Liveblog/Review Thread

Prelude

The prologue begins with a short story called the Tale of the Omega Team. It's a wish-fulfilment pseudo-isekai about a bunch of effective altruist tech people working for not-Google called the Omegas who make an AGI and then use it to take over the world.

But a cybersecurity specialist on their team talked them out of the game plan [...] risk of Prometheus breaking out and seizing control of its own destiny [...] weren't sure how its goals would evolve [...] go to great lengths to keep Prometheus confined

For some reason, the Omegas in the story claim that the Prometheus (the AI) might be unsafe, and then proceed to... (read more)

In the Parable of Predict-O-Matic, a subnetwork of the titular Predict-O-Matic becomes a mesa-optimiser and begins steering the future towards its own goals, independently of the rest of Predict-O-Matic. It does so in a way that sabotages the other subnetworks.

I am reminded of one specification problem that a run of Eurisko faced:

During one run, Lenat noticed that the number in the Worth slot of one newly discovered heuristic kept rising, indicating that Eurisko had made a particularly valuable find. As it turned out the heuristic performed no useful function. It simply examined the pool of new concepts, located those with the highest Worth values, and inserted its name in their My Creator

... (read more)

Tetraspace Grouping's Shortform

Tetraspace

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.