Aidan O'Gara

Comments

Is voting theory important? An attempt to check my bias.

OpenPhil supported the Center for Election Science once, but they're much more a political action group than a voting theory research group. They primarily do ballot initiatives and public education on what we already know.

If enacting your policies is the real bottleneck, then it makes sense that 90% of your argument is true, but it still doesn't matter because you can't enact political change.

I don't know if I believe that, but it's imaginable.

EDIT: After seeing that you know way more about this than I do, I'll leave my thought here, but definitely defer to you.

Why do you reject negative utilitarianism?

I haven't seriously evaluated the arguments, but my intuition is that suffering and happiness are opposite sides of the same scale, not separate values. Utility is the measure of how good or bad something is, and happiness and suffering correspond to positive and negative values of utility.

terminal value monism (suffering is the only thing that motivates us “by itself”)

So I'd say that I value only utility.

it is a straw man argument that NUs don’t value life or positive states, because NUs value them instrumentally

Again, not having thought too much about it, I find that my intuition better matches a system that cares about positive utility even when it doesn't avert negative utility. E.g., I want paradise forever, not mild pleasantness forever.

Is there a good reason to suspect this is wrong?

Drexler on AI Risk
a realistic example where I expect the delay would generate a strong incentive for using an agent AGI

I'd guess high speed stock trading. Right now, we already have AI trading stock to maximize profits over significant time horizons way faster than humans can effectively supervise.

We might already have examples of these AIs being misaligned and causing harm. (Maybe.) The 2010 Flash Crash is poorly understood, and few blame it entirely on high frequency trading algorithms. But regulators say that HFTs operating without human supervision were "clearly a contributing factor" to the crash because:

  • HFTs sold big and fast as soon as the market began dipping, faster than humans likely could've, and:
  • (Probably not a major factor) HFTs may have clogged and confused markets with quote stuffing--"placing and then almost immediately cancelling large numbers of rapid-fire orders to buy or sell stocks".

To be fair, others say that HFTs were a big part of why the crash was quickly reversed and the market returned to normal.

In any case, all of this happened without any human supervision, and was so opaque that we still don't understand what happened. That seems like evidence for opaque, unsupervised AIs with broad goals.

[Link] Did AlphaStar just click faster?

I don't know anything about StarCraft, but the impression I got was that a few seconds of superhuman clicking in high leverage situations can mean a lot.

Agreed that this is a big improvement on previous StarCraft AIs no matter its clicking speed, but this seems like reason to doubt that AI has surpassed human strategy in StarCraft.