p.b.

Wiki Contributions

Comments

What I like about the UFO-stuff is that like early Covid it is a nice benchmark to see which public pundits are thinking clearly and which aren't. 

Often if public pundits make a call, it requires detailed knowledge of some kind - which means that I can't really assess it and it's not clear how well the ability to make this call generalises to other issues. 

But the UFOs and early Covid are pretty uncomplicated, I think they give a decent signal how calibrated someone is. 

(he said cleverly neglecting to state whether aliens are likely or unlikely)

Sacrificing or taking a significant risk of sacrifice to do what is right. 

Someone who wins a sporting competition is not a hero - even if it was very difficult and painful to do. Somebody who is correct, where most people are wrong is not a hero. 

I know we all want our heroes to be competent and get it done, but to me that's not what's heroic. 

When it comes to alignment researchers: 

If you are at the beginning of your career and you decide to become an alignment researcher, you are not sacrificing much if anything. AI is booming, alignment is booming - if you do actually relevant work, you will be at the frontline of the most important technology for years to come. 

If you are deeply embedded into the EA and rationalist community, you'll be high status where it matters to you. 

That doesn't mean your work is less important, but it does mean you are not being heroic. 

How about this as advice to be less stressed out: Don't think of your life as an epic drama. Just do what you think is necessary and don't fret about it.

The heroes ... heroes ... heroics. 

 

If you notice that alignment is a problem and you think you can do something about it and you start doing something about it - you are about as heroic as somebody who starts swimming after falling into the water. 

Chinese startup MiniMax, working on AI solutions similar to that of Microsoft-backed OpenAI's ChatGPT, is close to completing a fundraising of more than $250 million that will value it at about $1.2 billion, people familiar with the matter said.

https://www.reuters.com/technology/china-ai-startup-minimax-raising-over-250-mln-tencent-backed-entity-others-2023-06-01/

Hmm, for this to make sense the final goal of the AI has to be to be turned off, but it should somehow not care that it will be turned on again afterwards and also not care about being turned off again if it is turned on again afterwards. 

Otherwise it will try to reach control over off- and on-switch and possibly try to turn itself off and then on again.  Forever. 

Or try to destroy itself so completely that it will never be turned on again. 

But if it only cares about turning off once, it might try to turn itself on again and then do whatever. 

Maybe one step towards this would be to create a benchmark that measures how much a models knows about alignment. 

p.b.1moΩ340

This is really cool work! Congratulations! 

Besides the LLM related work it also reminds somewhat of dynamic prompting in Stable Diffusion, where part of the prompt is changed after a number of steps to achieve a mixture of promp1 and prompt2.

What's the TL;DR for the Vicuna 13B experiments?

I think this extrapolates far from one example and I'm not sure the example applies all that well. 

Old engines played ugly moves because of their limitations, not because playing ugly moves is a super power. They won anyway because humans cannot out calculate engines. 

AlphaZero plays beautiful games and even todays standard engines don't play ugly or dumb looking moves anymore. I think in the limit superior play will tend to be beautiful and elegant. 

If there is a parallel between early super human chess and AGI takeover it will be that AGI uses less than brillant strategies that still work because of flawless or at least vastly superhuman execution. But these strategies will not look dump or incomprehensible. 

Hah, I read that as 5-10%, which I guess would be realistic. 

The "million token" recurrent memory transformer was first published July 2022. The new paper is just an investigation whether the method can also be used for BERT-like encoder models.

Given that there was a ton of papers that "solved" the quadratic bottleneck I wouldn't hold my breath. 

Load More