Swarm AI (tool)

I’m up for this

Constructing Goodhart

Thank you habryka!

Constructing Goodhart

Note: The LaTeX is not rendering properly on this reply. Does anyone know what the reason could be?

I chose because the optimal point in that case is the set of integers , but the argument holds for any positive real constant, and by using either equality, less than or not greater than.

There is one thing we assumed which is that, given the utility function , our proxy utility function is .This is not necessarily obvious, and even more so if we think of more convoluted utility functions: if our utility was given by , what would be our proxy when we only know ?

To answer this question generally my first thought would be to build a function that maps a vector space , a utility function , the manifold S of possible points and a map from those points to a filtration that tells us the information we have available when at point to a new utility function .

However this full generality seems a lot harder to describe.

Best, Miguel

Constructing Goodhart

I think it's possible to build a Goodharts example on a 2D vector space.

Say you get to choose two parameters and . You want to maximize their sum, but you are constrained by . Then the maximum is attained when . Now assume that is hard to measure, so you use as a proxy. Then you move from the optimal point we had above to the worse situation where , but .

The key point being that you are searching for a solution in a manifold inside your vector, but since some dimensions of that vector space are too hard or even impossible to measure, you end up in sub optimal points of your manifold.

In formal terms you have a true utility function based on all the data you have, and a misaligned utility function based on the subspace of known variables , where could be obtained by integrating out the unknown dimensions if we know their probability distribution, or any other technique that might be more suitable.

Would this count as a more substantive assumption?

Best, Miguel

Edit: added the "In formal terms" paragraph

One Website To Rule Them All?

Have you seen Kialo?

In Logical Time, All Games are Iterated Games

Thank you for your post abramdemski!

I failed to understand why you can't arrive at a solution for the Single-Shot game via Iterated Play without memory of the previous game. In order to clarify my ideas let me define two concepts first:

Iterated Play with memory: We repeatedly play the game knowing the results of the previous games.

Iterated Play without memory: We repeatedly play the game, while having no memory of the previous play.

The distinction is important: With memory we can at any time search all previous games and act accordingly, allowing for strategies such as Tit-for-Tat and other history dependent strategies. Without memory we can still learn ( for example by applying some sort of Bayesian updates to our probability estimates of each move being played ), whilst not having access to the previous games before each move. That way we can "learn" how to best play the single shot version of the game by iterated play.

Does what I said above need any clarification, and is there any failure in its' logic?

Best Regards, Miguel

Book Review - Probability and Finance: It's Only a Game!

You mention that a Martingale is a betting strategy where the player doubles their bet each time.

A Martingale is a fair game (i.e. the expected outcome is zero). If your outcome is given by a coin toss, and you receive only what you bet, then that is a Martingale game (you win X £ with probability and lose X £ with probability too ).

Then you could say that doubling your bet is a betting strategy on a Martingale game, BUT not that a Martingale game is a betting strategy where the player doubles their bet each time (in the same way that a dog is an animal but an animal is not a dog).

Does that make sense?

Other than that I'm very intrigued by the claim made. Definitely worth reading, but my hopes for something worthwhile are few :P

Lovely idea.

Minor point: it feels to me the average bet isn’t the usual average but instead the harmonic mean of all bets taken. The difference might be small and more importantly there’s no reason why the arithmetic average is fairer than the harmonic average, but it was just a small thing I noticed 😜