I feel like "Big-World Intuitions" implies that the main breaking point of those intuitions is the size of the environment and this can be misleading. In one of the examples given: "share scientific knowledge freely" the intuition might break down even if you are in a Big-World situation. You might discover information hazards accidentally anyway. For those cases, it is better to share with specific institutions first before revealing to the wider public. Similarly, in the chess example: how you develop in the opening might depend on what your opponent is ... (read more)
Why are yours and most other predictions for take-off timelines years long? My personal median for take-off via RSI (from a reliable auto-coder to ASI) is a single day. The primary reasons I expect it to be faster:
The potential for intelligence in modern hardware is extreme: modern computers can reach over 3 GHz, whereas humans neurons can't fire more than 1000 times per second. With enough processing cores and similar strategies for reasoning an AI could potentially think 6 OOMs faster than a human (assuming there are no other major bottlenecks for reason
I feel like "Big-World Intuitions" implies that the main breaking point of those intuitions is the size of the environment and this can be misleading. In one of the examples given: "share scientific knowledge freely" the intuition might break down even if you are in a Big-World situation. You might discover information hazards accidentally anyway. For those cases, it is better to share with specific institutions first before revealing to the wider public. Similarly, in the chess example: how you develop in the opening might depend on what your opponent is ... (read more)