Templarrr

Software engineer and small time DS/ML practitioner.

Posts

Sorted by New

Wiki Contributions

Comments

"Science Cathedrals"

Museum of Science in Boston seems very similar. I love going there when visiting Boston.

PSA: if you are in Russia, probably move out ASAP

According to my contact in Saint-Petersburg this warning is kinda late. It has already happened. It's not in the law yet but there are already major hurdles to leave the country and it's not about sanctions - it's from the Russian side.

Is AI Alignment a pseudoscience?

I'd rather call it proto- not pseudo- science. Currently it's alchemy before chemistry was a thing.

There is a real field somewhere adjacent to the discussions lead here and people are actively searching for it. AGI is coming , you can argue the timeline, but not the event (well, unless humanity destroys itself with something else first). And artificial systems we now have often shows unexpected and difficult to predict properties. So the task "how can we increase difficulty and capabilities of AI systems, possibly to the point of AGI, while simultaneously decreasing unpredictable and unexpected side effects" is perfectly reasonable.

The problem is that current understanding of the systems and entire framework is on the level of Ptolemy astronomy. A lot of things discussed at this moment will be discarded, but some grains of gold will become new science.

TBH I have a lot of MAJOR questions to the current discourse, it's plagued by misunderstanding of what and how is possible in artificial intelligence systems, but I don't think it should stop. The only way we can find the solution is by working on it, even if 99% of the work will be meaningless in the end.

The unfalsifiable belief in (doomsday) superintelligence scenarios?

Leaving this comment just so you know - you are not alone in the assessment. A lot of reasoning on the topic, when stripped down to the core, looks like "there is nonzero chance of extinction event with AGI, any nonzero probability multiplied by infinite loss is infinite loss, the only way to survive is to make probability exactly zero, either with full alignment (whatever that term supposed to exactly mean) or just not doing AGI", which a very bad argument and essentially Pascal's wager.

And yes, there are a lot of articles here "why this isn't Pascal's wager" that do not really work to prove their point unless you already agree with it.

A good rational fiction about IT-inspired magic system?

The revelation in later chapters of why magic works like programming was especially nice

Boring machine learning is where it's at

As a person that spent last 7 years of life in the company dedicated to make "old boring algorithms" easier to apply to as frictionless as possible to many problem types - 100% agree :)

Open problem: how can we quantify player alignment in 2x2 normal-form games?

Thanks for careful analysis, I must confess that my metric does not consider the stochastic strategies, and in general works better if players actions are taken consequently, not simultaneously (which is much different from the classic description).

The reasoning being that for maximal alignment each action of P1 there exist exactly one action of P2 (and vice versa) that is Nash equilibrium. In this case the game stops in stable state after single pair of actions. And maximally unaligned game will have no nash equilibrium at all, meaning the players actions-reactions will just move over the matrix in closed loop.

Overall, my solution as is seems not fitted for the classical formulation of the game :) but thanks for considering it!

Open problem: how can we quantify player alignment in 2x2 normal-form games?
Answer by TemplarrrJun 16, 2021Ω38

So, something like "fraction of preferred states shared" ? Describe preferred states for P1 as cells in the payoff matrix that are best for P1 for each P2 action (and preferred stated for P2 in a similar manner) Fraction of P1 preferred states that are also preferred for P2 is measurement of alignment P1 to P2. Fraction of shared states between players to total number of preferred states is measure of total alignment of the game.

For 2x2 game each player will have 2 preferred states (corresponding to the 2 possible action of the opponent). If 1 of them will be the same cell that will mean that each player is 50% aligned to other (1 of 2 shared) and the game in total is 33% aligned (1 of 3), This also generalize easily to NxN case and for >2 players.

And if there are K multiple cells with the same payoff to choose from for some opponent action we can give 1/K to them instead of 1.

(it would be much easier to explain with a picture and/or table, but I'm pretty new here and wasn't able to find how to do them here yet)

Does anyone have any Data Sidequests?

Have you considered applying to DataRobot? Our company does a lot of research in DS/ML with focus on transparency and explainability of resulting models in scalable and reproducible way. Your skills and interests looks exactly like the people we are looking for and we are constantly hiring.