Stephen: I think it's just an inability to see around an illusion. People think, "It seems like I have free will; I decide to do something and then I do it. How can that not be the case?" But this is just one of many illusions built into our brains, and it's hard to let go of like so many other illusions about time and space and so on that have come up in this series. As far as I can tell, and I've gotten into this discussion with many people, the sticking point is always that illusion.
The other thing that happens is people start to worry about how we can enforce laws/punish criminals and so forth if there's no free will, which just shows how entrenched the illusion is--these people are still imagining that we can somehow step outside the system to make that decision--like criminals don't have free will, but the government still does? They're somehow missing that no-free-will goes all the way up.
But "if your actions are determined by prior causes" then whether or not you think those actions are blameworthy is determined by prior causes too. The act of punishing criminals is subject to the same physics that crime is. So is talking about the act of punishing criminals. And so on.