Assuming you had a respectable metric for it, I wouldn't expect general intelligence to improve exponentially forever, but that doesn't mean it can't follow something like a logistic curve, with us near the bottom and some omega point at the top. That's still a singularity from where we're sitting. If something is a million times smarter than I am, I'm not sure it matters to me that it's not a billion times smarter.

I think this is plausible, but maybe a bit misleading in terms of real-world implications for AGI power/importance.

Looking at the scaling laws observed for language model pretraining performance vs model size, we see strongly sublinear increases in pretraining performance for linear increases in model size. In figure 3.8 of the GPT-3 paper, we also see that zero/few/many shot transfer learning to SuperGLUE benchmarks also scale sublinearly with model size.

However, the economic usefulness of a system depends on a lot more than just parameter count. Consider that Gorillas have 56% as many cortical neurons as humans (9.1 vs 16.3 billion; see this list), but a human is much more than twice as economically useful as a gorilla. Similarly, a merely human level AGI that was completely dedicated to accomplishing a given goal would likely be far more effective than a human. E.g., see the appendix of this Gwern post (under "On the absence of true fanatics") for an example of how 100 perfectly dedicated (but otherwise ordinary) fanatics could likely destroy Goldman Sachs, if each were fully willing to dedicate years of hard work and sacrifice their lives to do so.

The first two points in (1) are plausible from what we know so far. I'd hardly put it at more than 90% that there's no way around them, but let's go with that for now. How do you get 1 EUR per 10^22 FLOPs as a fundamental physical limit? The link has conservative assumptions based on current human technology. My prior on those holding is negligible, somewhere below 1%.

But that aside, even the most pessimistic assumption in this post don't imply that a singularity won't happen.

We know that it is possible to train a 200 IQ equivalent intelligence for at most 3 MW-hr, energy that costs at most $30. We also know that once trained, it is possible for it to do at least the equivalent of a decade of thought by the most expert human that has ever lived for a similar cost. We have very good reason to expect that it could be capable of thinking very much faster than the equivalent human.

Those alone are sufficient for a superintelligence take-off that renders human economic activity (and possibly the human species) irrelevant.