You heard the LLM, alignment is solved!
But seriously, it definitely has a lot of unwarranted confidence in its accomplishments.
I guess the connection to the real world is what will throw off such systems until they are trained on more real-world-like data.
I wouldn't phrase it that it needs to be trained on more data. More like it needs to be retrained within an actual R&D loop. Have it actually write and execute its own code, test its hypotheses, evaluate the results, and iterate. Use RLHF to evaluate its assessments and a debugger to evaluate its code. It doesn't matter whether this involves interacting with the "real world," only that it learns to make its beliefs pay rent.
Anyway, that would help with its capabilities in this area, but it might be just a teensy bit dangerous to teach an LLM to do R&D like this without putting it in an air-gapped virtual sandbox, unless you can figure out how to solve alignment first.
I agree. A while back, I asked Does non-access to outputs prevent recursive self-improvement? I think that letting such systems learn from experiments with the real world is very dangerous.
fwiw I'm kinda less interested in prompts like this (which... just obviously won't work, and are maybe funny but not particularly informative) and more interested in people experimenting with what sort of simpler tasks it can actually reliably do.
Maybe I should have posted this as a Shortform. Sorry. I don't have much time these days to polish a post but I thought: Better post it than not.
They ask the user to provide their own OpenAI API key. Is that wise? I am not a paid subscriber to OpenAI, so I haven't experienced how it works, but handing over one's key... won't that become risky at some point?
The code is open source. You can download and run it yourself, using your private API key. The statement about giving them your API key is confusing, since that's not actually how it works.
No, you can download the wrapper code, and interface your copy of the wrapper code with GPT4 using your private API key. Meaning you don't give your API key to anyone else, and you can modify that wrapper code however you see fit.
I saw this announcement on Twitter:
https://twitter.com/asimdotshrestha/status/1644883727707959296
I gave it a trial run and asked it to Solve the hard AI Alignment problem:
At this point the trial had reached 30 seconds and expired.
I think it is funny that it "thinks" the task executed successfully. I guess that's because in most texts ChatGPT has read how researchers describe their successes but not the actual work that is backing it up. I guess the connection to the real world is what will throw off such systems until they are trained on more real-world-like data.