ITT: link tries to distinguish between three often conflated (and highly correlated) states of mind, because the distinction is Important.
First of all, let me define my terms.
Hubris, I'm using entirely in its original context. The word comes from Ancient Greece; Google defines it as "excessive pride toward or defiance of the gods" - the state of daring to do - or even to attempt - that which should be the exclusive province of divinity. It is the sin of Icarus, who tried to touch the sun and so melted his wax wings; of Prometheus, who stole the heavenly fire from Zeus himself; of Arachne, who was in fact a better weaver than Athena but whose place in the world was not one where she was permitted to try.
I don't really need to spend much time on this one. Even if this post was on a less transhumanistic forum, Eliezer's done an excellent job of it already in Inadequate Equilibria and "Hero Licensing". Suffice it to say that we casually fly on metal wings, that we chain the lightning and use it to watch cat videos, that we make occasional attempts to see just how close to the sun we can get. Insofar as hubris is a useful concept at all, hubris is a virtue.
So is pride. Google-sensei again: "a feeling of deep pleasure or satisfaction derived from one's own achievements." Put more poetically, pride is the emotional result of succeeding at something hubristic - at accomplishing something you "shouldn't have been able" to accomplish. We don't get proud for being able to walk, unless we were once unable. -- In other words, pride is a built-in reward button for doing something we need to do anyway, an internal reward for achieving what we set out to do, and as such one that's much more robust as an incentive than most externally imposed. It's one of the things (sometimes, it seems the few things) that evolution got right in our brains: success is fun, especially when it's only just within our reach - and so we absolutely want "feeling and showing pride justly earned" to be entirely respectable.
I mention these because I want to be very, very clear I'm not talking about either of them.
Where hubris and pride are both focused on yourself, yourself and your ability to do the task before you, arrogance is about others. Arrogance, as I'm defining it, is sneer culture, is mockery; arrogance is the step from knowing you're more competent than most (or all) other people to pointing it out, rudely, to their face.
For example: compare and contrast Pat Modesto to Simplicio. (Sorry, EY, I know you get a lot of this but you really did hand me an excellent set of examples here.)
Pat Modesto is very well done. He is erudite, well-spoken; he's near enough to his author's and audience's beliefs to not be dismissed out of hand; he understands just enough rationality to come across as reasonable; he makes good-sounding points and strong-sounding arguments. Even though the narration is clear that he's meant to be read as wrong - that his mindset is entirely wrongheaded - it still respects him, as an intelligent man who happens to be epistemologically unlucky. I am not a modest man, and so I cannot say if he passes the ideological Turing test for modest epistemology -- but he certainly seems like he's within that ballpark.
Simplicio is none of these. Certainly, it is true that he holds beliefs that are truly held by the reference group he's meant to represent - but they are presented subtly drenched in biting ridicule; his use of common far-left memes, buzzwords, and even his choice of punctuation (exclamation points, everywhere) clearly meant to sound overwrought, lacking in insight or original thought, and entirely undeserving of attention. Even his name reflects this: to call a man "simple" is not a compliment. He is, in a word, a butt monkey. Whether or not he is in fact a strawman is irrelevant - his arguments have been presented in a way intended to elicit an emotional reaction, and that system 1 response is long done before any system 2 analysis of his words can take place. He does, I grant, get some good points in towards the end - but by that point the first impression set by his name and his dialogue has long since been cast.
That is what I mean by "arrogance" - the deliberate mockery of other people's beliefs and ideas, regardless of how worthy of mockery they may seem. Because, completely aside from any epistemological problems this may pose, arrogance as defined here alienates people. I'm not just talking about the people directly mocked - an argument can be made that they're not too likely to support or care about our soteriology, so we don't really care if they're alienated. And on the other end, the people we really want, the ones who already have the intellectual firepower to make a difference, will see through it and ask of themselves: "okay, so they're sort of jerks, but do they actually have a point, yes or no?"
The people I'm concerned about are a sort of "intellectual middle class": the ones who haven't quite bootstrapped themselves up to the point where we want them at MIRI or CFAR yesterday, but might one day reach that point, given motivation and inclination. Our "seed corn," one might say - proto-potential, the people who might become researchers and pioneers ten years from now. Because this set will be just insightful enough to notice that people are being mocked - and whether or not they are being mocked themselves, they will understand that they do not wish to associate themselves with that sort of mockery and that sort of person, and they will not think twice to see if we have a point.
They'll just stand by, or stand against us, and we'll lose them. And they do matter - or they will, ten years from now. Indeed, even on our own forum, I note a milder form of this point - that many people, in the comments to "Hero Licensing", found Modesto more fun to read, more engaging, than the comparable argument in Inadequate Equilibria, even though Eliezer's spokesman within the text is significantly more biting in his reproach - which my model predicts is a result of Modesto being seen as a proper worthy opponent.
And so this is my point. Hubris is more than useful: it is required, we can't get anything done if we're not willing to do the impossible. Insofar as our current society opposes that, with its sword called "modesty" and its dragging swamp called "status," it is right and proper to ignore it, to cut it aside and pierce through. Pride is not required, but it is useful: it is a purely internal, self-optimizing incentive gradient that encourages us to achieve and overachieve especially when things are most difficult. It is right and proper to feel proud when we have done the allegedly impossible.
But arrogance - sneer culture, strawmanning, failure to constantly, always write an opposition that passes an ITT...
Arrogance might get us all killed, someday.