RP — LessWrong

LESSWRONG
LW

RP — LessWrong

A starting point for making sense of task structure (in machine learning)

ML models can perform a range of tasks and subtasks, some of which are more closely related to one another than are others. In this post, we set out two very initial starting points. First, we motivate reverse engineering models’ task decompositions. We think this can be helpful for interpretability and for understanding generalization. Second, we provide a (potentially non-exhaustive, initial) list of techniques that could be used to quantify the ‘distance’ between two tasks or inputs. We hope these distances might help us identify the task decomposition of a particular model. We close by briefly considering analogues in humans and by suggesting a toy model.

Epistemic status: We didn’t spend much... (read 3362 more words →)

Transfer learning and generalization-qua-capability in Babbage and Davinci (or, why division is better than Spanish)

RP, agg

Tl/Dr:

Generalisation is a capability in its own right, and in our experiments it scales less quickly than specific capabilities.
Our experiments show that on a range of tasks, finetuning Babbage on task A transfers more to task B than finetuning Davinci on task A transfers to task B. This is relevant for identifying the nature of the alignment problem we might encounter (for things in this vague cluster see here, here, and here).
Of independent interest is the fact that a transfer learning metric can give us a fairly non-arbitrary way to measure how similar any two tasks are.

Epistemic status: written quickly, but oh look we did 4×8^2 experiments and we have a lovely graphic

This post makes... (read 775 more words →)

Replying toNietzsche's Morality in Plain English

RP2y

Nietzsche's Morality in Plain English

Thanks for writing this—I think it’s a fairly useful summary. I’m not a Nietzsche expert but it tracks well with what I know. I also find the L&L Darrow reference well-chosen: when I was younger I remember being excited the first time I read the Darrow plea, just because Leopold, whatever his other flaws, seemed to ‘take ideas seriously’—although I didn’t know the words for that then. I think one of Nietzsche’s contribution to culture is in some sense instantiating part of the essence of taking ideas seriously.

My main worry with your post is that, as with all analytic distillations of continental philosophy, it reduces a worldview to some claims about that... (read more)

-2

Apply to a small iteration of MLAB in Oxford

RP, MariaK, OliverHayman

TLDR: We’re running a small iteration of MLAB (~10 participants) in Oxford towards the end of September. If you’re interested in participating, apply here by 7 September. If you’re interested in being a TA, please email us directly at oxfordmlab@gmail.com

Background

MLAB is a program, originally designed by Redwood Research, to help people upskill for alignment work. We think it’s a good use of time if you want to eventually get into technical alignment work, or if you want to work on theoretical alignment or related fields and think understanding ML would be useful. The program we’re running is slightly shorter than the full MLAB—two weeks instead of three. We’ve condensed the curriculum similarly... (read 184 more words →)

Apply to a small iteration of MLAB to be run in Oxford

RP, MariaK, OliverHayman

Edit: The dates are now confirmed for the 23 September- 7 October.

Background

Crystal Healing — or the Origins of Expected Utility Maximizers

Alexander Gietelink Oldenziel

Alexander Gietelink Oldenziel, RP, Kaarel

(Note: John discusses similar ideas here. We drafted this before he published his post, so some of the concepts might jar if you read it with his framing in mind. )

Traditionally, focus in Agent Foundations has been around the characterization of ideal agents, often in terms of coherence theorems that state that, under certain conditions capturing rational decision-making, an agent satisfying these conditions must behave as if it maximizes a utility function. In this post, we are not so much interested in characterizing ideal agents — at least not directly. Rather, we are interested in how true agents and not-so true agents may be classified and taxonomized, how agents and pseudo-agents may... (read 1452 more words →)

Oxford Essay Writing

Flourish Journal

Flourish Journal, RP, Jemima

[Originally a post, duplicated as an event.]

Have an idea? Come to Oxford to discuss and write (20 – 24 March).

We’re running a workshop (20 – 24 March) in Oxford with three goals.

Go in with only a vague idea; come out with a drafted academic essay that you could eventually publish.
Build tacit knowledge about the research/doing-good-things sphere.
Meet other people in that sphere, for peers and mentoring.

We encourage you to apply if you think those goals seem useful or entertaining and if you fit the below criteria.

You’re between 18 – 23,
Interested in research or writing,
Have ideas (even vague or bare-bones ones) that, if taken seriously, might make the world better, and
Are available between 20

RP, Flourish Journal, Jemima

We’re running a workshop (20 – 24 March) in Oxford with three goals.

Go in with only a vague idea; come out with a drafted academic essay that you could eventually publish.
Build tacit knowledge about the research/doing-good-things sphere.
Meet other people in that sphere, for peers and mentoring.

We encourage you to apply if you think those goals seem useful or entertaining and if you fit the below criteria.

You’re between 18 and 25^[1],
Interested in research or writing,
Have ideas (even vague or bare-bones ones) that, if taken seriously, might make the world better, and
Are available between 20 – 24 March.

We welcome applications from people interested in all academic disciplines, from maths and computer science to politics... (read more)

Replying toNotes on writing

RP3y

Notes on writing

My main takeaway from your comment is that not all thoughts are of an expressible form, and that there’s a pre-writing step where inexpressible thoughts sometimes become expressible ones.

Before your comment, I would’ve considered the step you discuss part of step (1) of the intro (‘have a thought’). But, I think you make a good point about the end of idea formation being separable from the end of clarifying that idea into a potentially communicable thing—and about both being separable from the act of actually communicating the thing (e.g. writing).

Obviously, the borders between the three steps (or kinds of steps?) are vague and somewhat arbitrary. But, you make an interesting point, and... (read more)

Replying toNotes on writing

RP3y

Notes on writing

The point of point 10 is that overusing words like ‘like’ or choosing simple, possibly-inexact words can cause problems. In particular, it can cause writing to stray from being about the idea to being about the interaction with the idea (see point 9), it can set the wrong kind of vibe (communicate the wrong emotive thought to the reader), and it can occasionally obscure the logical content. I think using ‘like’ and deliberately simplified language is great when one is thinking through one’s ideas, but I think it often makes writing less sharp (less bell-like, if that simile resonates).

Notes on writing

Cross-posted from my new blog.

Recently, I gave a few people some advice on how to write well. Below is that advice.

What is it to write well?

Have a thought.
Get that thought from your head to someone else’s.

What is a thought?

In this community, a ‘thought’ tends to be some idea. But, thoughts are not synonymous with ideas. Some thoughts are vibes, forms of inspiration, emotions, etc.

Can’t writing itself come before step (1) of the above?

You can write to think, but that kind of writing is profoundly different from the writing I discuss here: writing to clarify your own thought is not writing to get your thought to someone else.

Must I have thoughts to write... (read 783 more words →)