Wiki Contributions

Comments

Is there a more-formal statement somewhere of the theorem in Complexity theory of team games without secrets? Specifically, one that only uses terms with standard meanings in complexity theory? I find that document hard to parse. 

If concreteness is helpful, take "terms with standard meanings in Complexity Theory" to be any term defined in any textbook on complexity theory. 

This is awesome! A couple suggestions:

"and quickly starts to replace both Fox News and other news sources among members of all political parties." -- if this is plausible, it's not clear to me, and while I'm not a great predictor of the human race, I'm pretty damn smart. More importantly, your story doesn't need it; what it needs is just that Face News is useful and liked by a strong majority of people, like Google is today.

Murpt is a fun detail, but your story doesn't need him either. Fasecure can become dominate in government systems over a period of years, without needing to assume bad politicians. It's empirically a great system for cybersecurity, after all. It can start at the state level. Few people besides the AI safety nerds (who, I'll grant, by now have realized that nanotech stories don't seem to resonate with the general public) seem to be raising a fuss about it. Elon Musk tweets some concerns, but he's too preoccupied with mars to do much else. There are plenty of non-billionaire computer scientists who are concerned, but they look at how the careers of earlier computer scientists have fared after speaking out, in the wake of ridicule from famous academics that followed, and that triggers enough doubt in their perspective, or simply short-sighted self-interest, that they mostly keep quiet.

Would love to read more stuff like this, and would be happy to help!
 

Lukas, I wish you had a bigger role in this community.

I've kept fairly up to date on progress in neural nets, less so in reinforcement learning, and I certainly agree at how limited things are now.

What if protecting against the threat of ASI requires huge worldwide political/social progress? That could take generations.

Not an example of that (which I haven't tried to think of), but the scenario that concerns me the most, so far, is not that some researchers will inadvertently unleash a dangerous ASI while racing to be the first, but rather that a dangerous ASI will be unleashed during an arms race between (a) states or criminal organizations intentionally developing a dangerous ASI, and (b) researchers working on ASI-powered defences to protect us against (a).

He might be willing to talk off the record. I'll ask. Have you had Darklight on? See http://lesswrong.com/r/discussion/lw/oul/openai_makes_humanity_less_safe/dqm8

If my own experience and the experiences of the people I know is indicative of the norm, then thinking about ethics, the horror that is the world at large, etc, tends to encourage depression. And depression, as you've realized yourself, is bad for doing good (but perhaps good for not doing bad?). I'm still working on it myself (with the help of a strong dose of antidepressants, regular exercise, consistently good sleep, etc). Glad to hear you are on the path to finding a better balance.

For Bostrom's simulation argument to conclude the disjunction of the two interesting propositions (our doom, or we're sims), you need to assume there are simulation runners who are motivated to do very large numbers of ancestor simulations. The simulation runners would be ultrapowerful, probably rich, amoral history/anthropology nerds, because all the other ultrapowerful amoral beings have more interesting things to occupy themselves with. If it's a set-it-and-forget-it simulation, that might be plausible. If the simulation requires monitoring and manual intervention, I think it's very implausible.

If my anecdotal evidence is indicative of reality, the attitude in the ML community is that people concerned about superhuman AI should not even be engaged with seriously. Hopefully that, at least, will change soon.

I'm not sure either. I'm reassured that there seems to be some move away from public geekiness, like using the word "singularity", but I suspect that should go further, e.g. replace the paperclip maximizer with something less silly (even though, to me, it's an adequate illustration). I suspect getting some famous "cool"/sexy non-scientist people on board would help; I keep coming back to Jon Hamm (who, judging from his cameos on great comedy shows, and his role in the harrowing Black Mirror episode, has plenty of nerd inside).

Load More