migueltorrescosta

Is there an easy way to turn a LW sequence into an epub?

I'd really like this feature as well

Allowing Exploitability in Game Theory

If Tim tells the truth with probability $p$, you simply get that you should guess what he said if $p<\frac{1}{1000000}$, and $p>\frac{1}{1000000}$. For Tim the optimal choice is to have $p=\frac{1}{1000000}$ in order not to give you any information: Anything else is playing on psychology and human biases, which exist in reality but trying to play a "perfect" game by assuming your opponent is not also leaves you vulnerable to exploitability, as you mentioned.

It seems you are trying to get a deeper understanding of human fallibility rather than playing optimal games. Have I misunderstood it?

A method for fair bargaining over odds in 2 player bets!

Lovely idea.

Minor point: it feels to me the average bet isn’t the usual average but instead the harmonic mean of all bets taken. The difference might be small and more importantly there’s no reason why the arithmetic average is fairer than the harmonic average, but it was just a small thing I noticed 😜

Swarm AI (tool)

I’m up for this

Constructing Goodhart

Thank you habryka!

Constructing Goodhart

Note: The LaTeX is not rendering properly on this reply. Does anyone know what the reason could be?

I chose because the optimal point in that case is the set of integers , but the argument holds for any positive real constant, and by using either equality, less than or not greater than.

There is one thing we assumed which is that, given the utility function , our proxy utility function is .This is not necessarily obvious, and even more so if we think of more convoluted utility functions: if our utility was given by , what would be our proxy when we only know ?

To answer this question generally my first thought would be to build a function that maps a vector space , a utility function , the manifold S of possible points and a map from those points to a filtration that tells us the information we have available when at point to a new utility function .

However this full generality seems a lot harder to describe.

Best, Miguel

Constructing Goodhart

I think it's possible to build a Goodharts example on a 2D vector space.

Say you get to choose two parameters and . You want to maximize their sum, but you are constrained by . Then the maximum is attained when . Now assume that is hard to measure, so you use as a proxy. Then you move from the optimal point we had above to the worse situation where , but .

The key point being that you are searching for a solution in a manifold inside your vector, but since some dimensions of that vector space are too hard or even impossible to measure, you end up in sub optimal points of your manifold.

In formal terms you have a true utility function based on all the data you have, and a misaligned utility function based on the subspace of known variables , where could be obtained by integrating out the unknown dimensions if we know their probability distribution, or any other technique that might be more suitable.

Would this count as a more substantive assumption?

Best, Miguel

Edit: added the "In formal terms" paragraph

One Website To Rule Them All?

Have you seen Kialo?

In Logical Time, All Games are Iterated Games

Thank you for your post abramdemski!

I failed to understand why you can't arrive at a solution for the Single-Shot game via Iterated Play without memory of the previous game. In order to clarify my ideas let me define two concepts first:

Iterated Play with memory: We repeatedly play the game knowing the results of the previous games.

Iterated Play without memory: We repeatedly play the game, while having no memory of the previous play.

The distinction is important: With memory we can at any time search all previous games and act accordingly, allowing for strategies such as Tit-for-Tat and other history dependent strategies. Without memory we can still learn ( for example by applying some sort of Bayesian updates to our probability estimates of each move being played ), whilst not having access to the previous games before each move. That way we can "learn" how to best play the single shot version of the game by iterated play.

Does what I said above need any clarification, and is there any failure in its' logic?

Best Regards, Miguel

First of all thank you for your post, it’s very thorough :)

While I want to reread it in case I missed any arguments for this, the main issue I usually have with these trust webs is the propensity for the creation of echo chambers: by relying only on those you trust and who they trust, you might filter out others opinions not because they are less valid, but because you disagree on some fundamental axioms. Have you given any thought on how to avoid echo chambers in these webs of trust?

Best, Miguel