it makes cartoonish errors reasoning about various AI related situations and can't do such reasoning except aloud

Wouldn't reasoning aloud be enough though, if it was good enough? Also, I expect reasoning aloud first to be the modal scenario, given theoretical results on Chain of Thought and the like.