Here's a high-level argument against very short timelines:

1. LLMs clearly lack some of the important cognitive algorithms that humans possess, and there's been no sign of them grokking those algorithms/meaningfully "getting better" at them over the years. Given their stratospheric growth in other aspects, and how far their practical capabilities have been pushed, this suggests (though by no means proves) that they're not going to ever learn those algorithms (for any number of reasons). And this presents a legitimate objective limitation: they are in fact orders of magnitude less efficient at some important cognitive tasks than humans. Scaffolds won't fix it, multi-agent swarms won't fix it, other in-paradigm tricks won't fix it, no matter how many of them you stack.
2. For the entities executing it, pushing the Singularity/general-purpose strong RSI forward will be at least as hard as the hardest cognitive project humanity has historically accomplished. If it implies compressing several next centuries of counterfactual human science and engineering into a few scant years, those entities would need to solve problems at least as hard and diverse as the last several centuries of human scientific and engineering progress combined. Thus, a computationally tractable implementation of this process would likely require using every major trick for making cognition efficient that human brains implement. Trying to compensate for a missing trick by quantitatively scaling other tricks won't work: the gulf between those approaches may be of (handwave, handwave) exponential-time vs. polynomial-time implementation of "the Singularity algorithm".
3. Therefore, nothing within the LLM paradigm can power the Singularity.

Of those:

• I am pretty certain of (1).
• I am not entirely confident in (2), but I do think it's pretty likely. (Note that this is not a "you have to build a bird to fly" argument that's validly countered by planes. Rather, the argument is that the current would-be planes in fact aren't taking advantage of all properties of physics that they have to take advantage of in order to achieve sustained flight. Extending the analogy, LLMs to humans and AGIs are what gliders are to birds and planes.)
• I am most unsure of the inference in (3). E. g., it's possible that the last remaining tricks are "conceptually close" to the tricks already known, and that rapidly mining out those tricks using exponentially inefficient algorithms of LLMs is nevertheless doable in the next couple years. Or some other schlep along those lines. (If you want to reach a given high altitude, maybe you have to build a plane... or maybe it's enough to climb a very tall mountain and then build a very tall tower on top.)

Still, all together, I think something of this form is the main underlying reason I have my "medium timelines" of 2030-2035.