Tao Lin

Wiki Contributions

Comments

If automating alignment research worked, but took 1000+ tokens per researcher-second, it would be much more difficult to develop, because you'd need to run the system for 50k-1M tokens between each "reward signal or such". Once it's 10 or less tokens per researcher second, it'll be easy to develop and improve quickly.

AIs could learn to cooperate with perfect selfishness, but humans and AIs usually learn easier to compute heuristics / "value shards" early in training, which persist to some extent after the agent discovers the true optimal policy, although reflection or continued training could stamp out the value shards later.

The big reason why humans are cosmopolitan might be that we evolved in multipolar environments, where helping others is instrumental. If so, just training AIs in multipolar environments that incentivize cooperation could be all it takes to get some amount of instrumental-made-terminal-by-optimization-failure cosmopolitanism. 

if I were doing this, I'd use gpt-4 to translate it into the style of a specific person, preferably a deceased public figure, then edit the result. I'd guess GPTs are better at translating to a specific style than removing style

I think the fact that some process produced the image and showed it to you is a lot of evidence. Your theories need to be compatible with something intelligent deciding to produce the image and show it to you. Therefore you could in principle (although I think unlikely) arrive at GR from a render of a simulated apple, by considering universes that support intelligence where said intelligence would make an image of an apple.

One takeaway from this would be "CoT is more accurate at the limit of the model's capabilities".  Given this, you could have a policy of only using the least capable model for every task to make CoT more influential. Of course this means you always get barely-passable model performance, which is unfortunate. Also, people often use CoT in cases where it intuitively wouldn't help, such as common sense NLP questions, where I'd expect influential CoT to be pretty unnatural.

based on semi-private rumors and such, i think their current best model is significantly behind gpt4

>Why did you decide to go with the equivalence of 1 token = 1 bit? Since a token can usually take on the order of 10k to 100k possible values, wouldn't 1 token equal 13-17 bits a more accurate equivalence?

 

LLMs make very inneficient use of their context size because they're writing human-like text which is predictable. Human text is like 0.6 bits/byte, so maybe 2.5 bits per token. Text used in language model scaffolding and such tends to be even more predictable (by maybe 30%)

Why measure flops at FP32? All the big training runs in the last 2 years are FP16 right?

Foundation Models tend to have a more limited type of orthogonality - they're good at pursuing any goal that's plausible under the training distribution, meaning they can pursue any goal that humans would plausibly have (with some caveats I guess). This is most true without outcome-based RL on top of the foundation model, but I'd guess some of the orthogonality transfers through RL.

Load More