A new open-source model has been announced by the Chinese lab DeepSeek: DeepSeek-V3. It reportedly outperforms both Sonnet 3.5 and GPT-4o on most tasks and is almost certainly the most capable fully open-source model to date.

Beyond the implications of open-sourcing a model of this caliber, I was surprised to learn that they trained it using only 2,000 H800 GPUs! This suggests that, with an exceptionally competent team of researchers, it’s possible to overcome computational limitations.

Here are two potential implications:

1. Sanctioning China may not be effective if they are already capable of training cutting-edge models without relying on massive computational resources.
2. We could be in a serious hardware overhang scenario, where we already have sufficient compute to build AGI, and the only limiting factor is engineering talent.
   
   (I am extremely uncertain of this, it was just my reaction after reading about it)
    

Perhaps Randolph Carter was right about losing access to dreamlands after your twenties:

When Randolph Carter was thirty he lost the key of the gate of dreams. Prior to that time he had made up for the prosiness of life by nightly excursions to strange and ancient cities beyond space, and lovely, unbelievable garden lands across ethereal seas; but as middle age hardened upon him he felt these liberties slipping away little by little, until at last he was cut off altogether. No more could his galleys sail up the river Oukranos past the gilded spires of Thran, or his elephant caravans tramp through perfumed jungles in Kled, where forgotten palaces with veined ivory columns sleep lovely and unbroken under the moon.

Btw, have you heard about PropheticAI?  They are working on device that is supposed to help you with lucid dreaming?

Still think it will be hard to defend against determined and competent adversaries committed to sabotaging the collective epistemic. I wonder if prediction markets can be utilised somehow? 

My dark horse bet is on 3d country trying desperately to catch up to US/China just when they will be close to reaching agreement on slowing down progress. Most likely: France. 

Why so? My understanding is that, if AGI will arrives in 2026 it will be based on the current paradigm of training increasingly large LLMs on massive clusters of advanced GPUs. Given that US has banned selling advanced GPUs to China, how do you expect them to catch up that soon?

To add to this point, author in question is infamous for doxxing Scott Alexander and writing a hit piece on rationalist community before.

https://slatestarcodex.com/2020/09/11/update-on-my-situation/

 

Also agree about not promoting political content on LW but would love to read your writings on some other platform if possible.