jsteinhardt

Wiki Contributions

Comments

How much slower is remote work?

It is definitely useful in some settings! For instance it's much easier to collaborate with people not at Berkeley, and in some cases those people have valuable specialized skills that easily outweigh the productivity hit.

How much slower is remote work?

I personally have Wednesdays, plus Thursday mornings, as "no meeting days". I think it works pretty well and I know other faculty who do something similar (sometimes just setting mornings as meeting-free). So this does seem like a generally good idea!

Film Study for Research

Thanks, those are really cool!

Where do your eyes go?

I enjoyed this quite a bit. Vision is very important in sports as well, but I hadn't thought to apply it to other areas, despite generally being into applying sports lessons to research (i.e. https://bounded-regret.ghost.io/film-study/).

In sports, you have to choose between watching the person you're guarding and watching the ball / center of play. Or if you're on offense, between watching where you're going and watching the ball. Eye contact is also important for (some) passing.

What's most interesting is the second-level version of this, where good players watch their opponent's gaze (for instance, making a move exactly when the opponent's gaze moves somewhere else). I wonder if there's an analog in video games / research?

Let Us Do Our Work As Well

Thanks, really appreciate the references!

Economic AI Safety

If there was a feasible way to make the algorithm open, I think that would be good (of course FB would probably strongly oppose this). As you say, people wouldn't directly design / early adopt new algorithms, but once early adopters found an alternative algorithm that they really liked, word of mouth would lead many more people to adopt it. So I think you could eventually get widespread change this way.

Film Study for Research

Thanks for the feedback!

I haven't really digged into Gelman's blog, but the format you mention is a perfect example of the expertise of understanding some research. Very important skill, but not the same as actually conducting the research that goes into a paper.

Research consists of many skills put together. Understanding prior work and developing the taste to judge it is one of the more important individual skills in research (moreso than programming, at least in most fields). So I think the blog example is indeed a central one.

In research, especially in a weird new field like alignment, it's rare to find another researcher who want to conduct precisely the same research. But that's the basis of every sport and game: people want to win the same game. It make the whole "learning from other" slightly more difficult IMO. You can't just look for what works, you constantly have to repurpose ideas that work in slightly different field and/or approaches and check for the loss in translation.

I agree with this, although I think creative new ideas often come from people who have also mastered the "standard" skills. And indeed, most research is precisely about coming up with new ideas, which is a skill that you can cultivate my studying how others generate ideas.

More tangentially, you may be underestimating the amount of innovation in sports. Harden and Jokic both innovate in basketball (among others), but I am pretty sure they also do lots of film study. Jokic's innovation probably comes from having mastered other sports like water polo and the resulting skill transfer. I would guess that mastery of fruitfully adjacent fields is a productive way to generate ideas.

Experimentally evaluating whether honesty generalizes

Actually, another issue is that unsupervised translation isn't "that hard" relative to supervised translation--I think that you can get pretty far with simple heuristics, such that I'd guess making the model 10x bigger matters more than making the objective more aligned with getting the answer right (and that this will be true for at least a couple more 10x-ing of model size, although at some point the objective will matter more).

This might not matter as much if you're actually outputting explanations and not just translating from one language to another. Although it is probably true that for tasks that are far away from the ceiling, "naive objective + 10x larger model" will outperform "correct objective".

Experimentally evaluating whether honesty generalizes

Thanks Paul, I generally like this idea.

Aside from the potential concerns you bring up, here is the most likely way I could see this experiment failing to be informative: rather than having checks and question marks in your tables above, really the model's ability to solve each task is a question of degree--each table entry will be a real number between 0 and 1. For, say, tone, GPT-3 probably doesn't have a perfect model of tone, and would get <100% performance on a sentiment classification task, especially if done few-shot.

The issue, then, is that the "fine-tuning for correctness" and "fine-tuning for coherence" processes are not really equivalent--fine-tuning for correctness is in fact giving GPT-3 additional information about tone, which improves its capabilities. In addition, GPT-3 might not "know" exactly what humans mean by the word tone, and so fine-tuning for correctness also helps GPT-3 to better understand the question.

Given these considerations, my modal expectation is that fine-tuning for correctness will provide moderately better results than just doing coherence, but it won't be clear how to interpret the difference--maybe in both cases GPT-3 provides incoherent outputs 10% of the time, and then additionally coherent but wrong outputs 10% of the time when fine-tuned for correctness, but 17% of the time when fine-tuned only for coherence. What would you conclude from a result like that? I would still have found the experiment interesting, but I'm not sure I would be able to draw a firm conclusion.

So perhaps my main feedback would be to think about how likely you think such an outcome is, how much you mind that, and if there are alternative tasks that avoid this issue without being significantly more complicated.

Load More