This is a special post for quick takes by the gears to ascension. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
post ideas, ascending order of "I think causes good things":
-
(lowest value, possibly quite negative) my prompting techniques
-
velocity of action as a primary measurement of impact (how long until this, how long until that)
sketch: people often measure goodness/badness in probabilities. latencies, or probability of moving to next step per time?, might be underused, for macro scale systems. if you're trying to do differential improvement of things, you want to change the expected time until a thing happens - which, looking at the dynamics of the systems involved, means changing how fast relevant things happen. possibly obvious to many, weird I'd need to even say it for some, but a useful insight for others?
-
goodhart slightly protective against ppl optimizing for badbehavior benchmarks?
sketch: people make benchmark of bad thing. optimizing for benchmark doesn't produce as much bad thing as ai that accidentally scores highly. so, benchmark of bad thing not as bad as it seems. especially if dataset small. standard misalignment argument, but may be mildly protective if dataset is of doing bad things instead of good things
-
my favorite research plans and why you should want to contribute or use them (todo: move post's point into title as much as possible).
sketch: you must eventually "solve alignment"-as-in-make-a-metric-that-can-be-optimized-~indefinitely-and-provably-does-good-things-if-you-do, this remains true for deep learning based asi and it remains true if "solving alignment"-as-in-never-have-to-do-anything-again isn't a thing
- a metric like that needs to care about the world around it; so, see/please assist wentworth, kosoy
- a metric like that needs to care about the agency of the beings in the world already, so, see/please assist kosoy, ngo, and other "what formal property is agency as we care about it, really? is there anything wrong with or ActInf?" research
- a metric like that needs to have a provable relationship to your learning system; so, see/please assist kosoy and other learning theory, katz and other formal verification
in order to do this, we want to have a piece of math (theorem with a natural language hole, perhaps) whose correctness can be checked (eg, by humans being able to reliably check whether a resulting fleshed-out theorem represents the right philosophical thing the natural language described) such that, if completed and proved, means we have such a metric and we're ready to plug it in. need to be able to avoid slop hard enough to not get fooled about whether the metric is really the right one
2