What's the chance that AI doesn't have that much of an impact on the world by 2100?

Honestly, that one belongs in the settled-questions bin next to theism. Making intellectual progress requires having spaces where the basics can be taken for granted, for a definition of "the basics" that's for people trying to contribute at the intellectual frontier, rather than for the human population at large.

How well does the epistemic process on LW work? Are there any changes you would make to LW's epistemic processes?

This is never going to be perfect, anywhere, and people should always be on the lookout for epistemic problems. But there's a pretty strong outside-view reason to think LW's epistemics will outperform those of the rest of the world: it's full of people investing heavily in improving their epistemics, and having abstract discussions about them. 

What's the chance that we do have massive impacts, but alignment is so easy that standard ML techniques work?

I think this is the core question, but is a slightly incorrect framing. I also think this is the core point of disagreement between the AGI Ruin perspective and the AI Accelerationist perspective.

How hard alignment is, is a continuous variable, not a boolean. The edges of the range are "it's borderline impossible to solve before time runs out" and "it's trivial and will solve itself". The same applies to framing specific research as capabilities research or as alignment research: a lot of things live in the border research, where it makes more sense to think in terms of things having a ratio between those two.

I don't think the people leading and working in AGI research programs think alignment is easy. I do think that they think that it's easier, by a large enough amount to change their view of the cost-benefit of accelerating the timelines. And because this is a continuous variable with a lot of inputs, expanding it out doesn't yield a single large crux that distinguishes the two camps, but rather a large number of smaller, unshared differences in belief and intuition.

(I lean more towards the "it's hard" side, but am decidedly not on the edge of the scale; I think it's likely to be close enough that individual insights into alignment, and modest changes to research timelines, could potentially be decisive. I also think that my difficulty-estimation could move substantially in either direction without changing my beliefs about the correct course of action, due to a "playing to outs" argument.)

1. This probability is the probability of non AI apocalypse. (Large asteroid impacts, nuclear war, alien invasion, vacuum collapse, etc). Basically assuming nothing stopping humans from continuing to improve AI, the chance of "not much impact" is precisely 0. It's 0 because either it already had an impact or will in the very near future with just slight and obvious improvements to the AI systems that already exist. What sort of future history would have "no significant impact" and HOW? This is like asking after the first Trinity fission weapon test what the probability by 2022 there would be "no significant impact" from nuclear weapons. It's 0 - already the atmosphere of the earth was contaminated we just didn't know it.

2. This is very possible. Complex deception and unstoppable plans to conquer the planet and so on require specific setups for the agent, like "long term reward". Actual models have myopia inherently, due to how they are trained and limitations on their computational resources. This means a "paperclip production agent" is probably more likely to spend all it's compute optimizing for small variables like air temperature differences and other parameters to accelerate the robots producing paperclips than to invest in a multi year plan to take over the planet that will let it tile the solar system in paperclip plants after it wins a world war.

3. I think it isn't productive to say "let's not talk of how we would improve capabilities.". Modeling how future systems are likely to actually work helps to model how you might restrict their behavior effectively.

Imagine LessWrong started with an obsessive focus on the dangers of time-travel. 

Because the writers are persuasive there are all kinds of posts filled with references that are indeed very persuasive regarding the idea that time-travel is ridiculously dangerous, will wipe out all human life and we must make all attempts to stop time-travel.

So we see some new quantum entanglement experiment treated with a kind of horror. People would breathlessly "update their horizon" like this matters at all. Physicists completing certain problems or working in certain areas would be mentioned by name and some people would try to reach out to them to convince them how dangerous time-travel and what they're doing is.

Meanwhile, from someone not taken in by very persuasive writing, vast holes are blindingly obvious. When those vast holes are discussed... well, they're not discussed. They get nil traction, are ignored, aren't treated with any seriousness. 

Examples of magical thinking (they're going to find unobtainium and that'll be it, they'll have a working time-machine within five years) are rife but rarely challenged.

I view a lot of LessWrong like this. 

I'll provide two examples.

1. AI will improve itself very quickly, becoming the most intelligent being that can exist and then will have the power to wipe humans out.
2. AI will be able to make massive technological jumps, here come nanites, bye humans

For 1 - we don't have any examples of this in nature. We have evolution over enormous timelines which has eventually produced intelligence in humans and varying degrees of it in other species. We don't have any strong examples of computers improving code which in turn improves code which in turn improves code. ChatGPT for all the amazing things it can do -- okay, so here's the source code for Winzip, make compression better. I do agree "this slow thing but done faster" is possible but it is an extraordinarily weak claim that self-improvement can exist at all. Just because learning exists, does not mean fundamental architecture upgrades can be made self-recursively. 

For 2 - AI seems to always be given near godlike magical powers. It will be able to "hack" any computer system. Oh, so it worked out how to break all cryptography? It will be able to take over manufacturing to make things to kill people? How exactly? It'll be able to work up a virus to kill all humans and then hire some lab to make it... are we really sure about this? 

I wrote about the "reality of the real world" recently. So many technologies and processes aren't written down. They're stored in meat minds, not in patents, and embodied in plant equipment and vast, intricate supply chains. Just trying to take over Taiwan chip manufacturing would be near impossible because they're so far out on the the cutting edge they jealously guard their processes.

I love sci-fi but there are more than a few posts that are pretty close to sci-fi fan fiction than actual real problems.

The risk of humans using ChatGPT and so on to distort narratives, destroy opponents, screw with political processes and so on seems vastly more deadly and serious than an AI will self-improve and kill us all. 

Going back to the idea of LessWrong obsessed with time-travel - what would you think of such a place? It would have all the predictions, and persuasive posts, and people very dedicated to it... and they could all just be wrong. 

For what it's worth, I strongly support the premise that anything possible in nature is possible for humans to replicate with technology. X-rays exist, we learn how to make and use them. Fusion exists, we will learn how to make fusion. Intelligence/sentience/sapience exists - we will learn how to do this. But I rarely see anyone touch on the idea of "what if we only make something as smart as us?"