naterush — LessWrong

LESSWRONG
LW

naterush's Shortform

naterush

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

naterush1yQuick Take

For anyone too lazy to look through SWELancer themselves:

There's some sub benchmarks, but the main coding benchmark is pretty much like "here's a real issue description, fix it." There are also real dollar values attached to the issues (based on bounties the open source maintainers put on the issue).
The paper claims: these issues are filtered less than SWE-Bench Verified (which e.g. filters for PRs that have automatic tests). But it's not clear this is true. For example, if you look through the issues that e.g. Expensify has in the benchmark, they say things like "I'm not sure this is a great issue for external contractors. @expensify does this feel well-contained enough for

naterush3y

Shh, don't tell the AI it's likely to be evil

How do you define transformative AI? If ChatGPT gets 10x better (e.g. it can write most code, answer most questions as good as the experts in a subject, etc) -- would this qualify?

How would you even force an AI to use the weights (simplification) that correspond to the fact vs. fiction anyways?

Also, what really is the difference between our history textbooks and our fiction to an AI that's just reading a bunch of text? I'm not being flippant. I'm genuinely wondering here! If you don't imbue these models with an explicit world-model, why would one be always privileged over the other?

Shh, don't tell the AI it's likely to be evil

naterush

If you haven't been online in the past week: ChatGPT is OpenAI's new fine-tuned language model that is really much better at doing all sorts things that you'd expect. Here's it debugging some code, or writing awful-sounding Seinfeld episodes, or (my favorite) writing a verse from the King James Bible about taking a PB sandwich out of a VCR.

My fairly-basic understanding of large language models is that they are trying to predict the most-likely-next token in a sequence. So if you start a sentence with "I love peanut butter and..." the large language model will finish it with "jelly." It's clear that the relationships be captured by "most-likely-token" are way more complex... (read 283 more words →)

Machines vs. Memes 2: Memetically-Motivated Model Extensions

naterush

[This is post 2 of 3 in a series written as the output for the 2022 AI Safety Camp, for the ‘Impact of Memetics on Alignment’ team, coached by Daniel Kokotajlo and comprising Harriet Farlow, Nate Rush and Claudio Ceruti. You can view the first post in this series here. Feedback so appreciated I'll do a jig for every comment.]

Overview

This post is a beginner’s attempt at extending the generally-accepted model of reinforcement learning agents (see this post).

This extension encourages us to think about artificial agents in a more suggestive and (potentially) more productive way. Specifically, this extensions captures an agent’s ability to change the policies it uses over time, while abstracting the decision to... (read 1182 more words →)