Burny — LessWrong

LESSWRONG
LW

Burny — LessWrong

How I think about alignment and ethics as a cooperation protocol software

4mo

I think alignment by itself is complex phenomenon, but big part of it is sharing the same, or similar ethics.

And ethics itself is also very complex phenomenon and often fuzzy and inconsistent, but to a certain approximation from a certain perspective, I see a lot of ethics in big part as software protocol evolved by evolution for mutual cooperation between "same enough kinds".

And different ethical frameworks define "same enough kind" differently, like if you include people from your neighborhood, country, earth, ideology, personality cluster, interests cluster, etc., or also animals (also which animals you include and exclude), more aligned or less aligned AGIs, etc., in your ethics as someone you want to... (read more)

Burny's Shortform

Burny

8mo

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Why is Gemini telling the user to die?

Burny

https://gemini.google.com/share/6d141b742a13 My favorite theory is that the whole conversation is too much like a scifi plot, where someone asks an AI repetitive questions, until the AI snaps, so this general pattern was pattern matched from the training data, because while training, the RLHF, or whatever they use for alignment, didn't squeeze the region that corresponds to this antihuman persona in the latent space sufficiently enough.

Possible OpenAI's Q* breakthrough and DeepMind's AlphaGo-type systems plus LLMs

Burny

tl;dr: OpenAI leaked AI breakthrough called Q*, acing grade-school math. It is hypothesized combination of Q-learning and A*. It was then refuted. DeepMind is working on something similar with Gemini, AlphaGo-style Monte Carlo Tree Search. Scaling these might be crux of planning for increasingly abstract goals and agentic behavior. Academic community has been circling around these ideas for a while.

Reuters: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster, sources say

Michael Trazzi, on Twitter:

"Ahead of OpenAI CEO Sam Altman’s four days in exile, several staff researchers sent the board of directors a letter warning of a powerful artificial intelligence discovery that they said could threaten humanity
Mira Murati told employees on

... (read 575 more words →)

Human-like systematic generalization through a meta-learning neural network

Burny

Step closer to AGI?

The classic argument made over 30 years ago by Fodor and Pylyshyn - that neural networks fundamentally lack the systematic compositional skills of humans due to their statistical nature - has cast a long shadow over neural network research. Their critique framed doubts about the viability of connectionist models in cognitive science. This new research finally puts those doubts to rest.

Through an innovative meta-learning approach called MLC, the authors demonstrate that a standard neural network model can exhibit impressive systematic abilities given the right kind of training regimen. MLC optimizes networks for compositional skills by generating a diverse curriculum of small but challenging compositional reasoning tasks. This training nurtures... (read 359 more words →)