This is a special post for quick takes by Raemon. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
20
and obviously in some sense the underlying distribution is kinda smooth
I actually disagree that the output distribution has to be smooth, even assuming the inputs are smoothly increasing.
The reason for this is phase changes, and an example is when you heat up water from say 20 degrees Celsius to 100 degrees Celsius in increments of say 0.01 degrees per minute, and while it is true that increasing the heat by 0.01 Celsius per minute matters in and of itself, at 100 degrees Celsius, water heating behaves discontinuously and boiling starts to happen.
And the impact of AI is also likely to be a phase change, because of long-tail effects meaning that 99% automation of something is vastly less valuable than 100% automation of something.
More generally, even assuming the inputs are continously increasing, you cannot assume that downstream consequences/output will continuously increase.
This likely has important implications for how to think about AI.
1
This is an experiment in short-form content on LW2.0. I'll be using the comment section of this post as a repository of short, sometimes-half-baked posts that either:
I ask people not to create top-level comments here, but feel free to reply to comments like you would a FB post.