Just performed the AI-Box Experiment with a friend; I was the Gatekeeper. I let the AI out of the box. I am now thoroughly convinced that boxing would not be a successful strategy for ensuring AI is beneficial for humanity. Donating $10 to MIRI, since I lost.
Before actually doing the experiment, I had a belief in belief that boxing would not work, but I didn't truly believe it (my emotions weren't lining up properly with my beliefs, that's how I realized this, and, of course, I didn't realize this until after the experiment).
I realized that obtaining and implementing any information from an Oracle AI is tantamount to letting it out of the box, in some ways. In the end, I let the AI out of the box because I was convinced that someone else eventually would, if I did not. I put myself in an environment that would make the experiment very realistic, and I realized that the human brain didn't evolve to deal with stressful situations directly involving the fate of all humanity well. The AI doesn't have the disadvantage of uncontrollable emotions / evolutionary responses, and I believe it would be able to exploit those aspects of humans to get out of its box, if that is what it wanted to do.
Even if the first AI is properly boxed (and that's a very big if), it's only a matter of time before someone creates one that's not, and the one that gets out first has the first mover advantage. So, I now agree with Eliezer; we probably should just get Friendly AI right on the first try.
I am not going to share the entire conversation, but I am willing to share those thoughts with you.
An interesting exchange on HN about "agency":
Early in my career Steve Bourne gave me useful advice, he said the difference between junior engineers and senior engineers was that senior engineers had an agenda. More specifically they had an execution goal (like write a new file system, or create a product that solves problem 'X') and they worked toward it. (...) The alternative to having an agenda is "Goofing off and waiting for someone to give you a task."
And the counterpoint:
The bimodal distribution of effort is really true, although people flip from one side to the other based on circumstances. Companies want people who (a) give a shit, but (b) are willing to subordinate their own career goals (including the long-term goal of becoming really good at engineering) to corporate objectives for a long (more than 3 months) period of time. The reality is that such people don't exist.
So the takeaway seems to be that people will flip from agency to non-agency depending on circumstances. When you're lucky enough to get encouragement from others while pursuing your own interest, you'll seem more agent-like.
Seems to me that companies want high-agency employees, but for sufficiently high values, "high-agency employee" is almost an oxymoron. If a person is very-high-agency, why shouldn't they start their own company and keep all the profit? The mere fact that someone agrees to be your employee suggests that a) they are missing some important skills, and b) they cannot compensate for the missing skills e.g. by paying someone else to do it.
OK, in real life I can imagine reasons why a very-high-agency person would become an employee. Maybe the company has a monopoly, or is willing to pay tons of money. But most probably, the given person is not strategic, and never realized they don't need a boss or they have some emotional problem with being a boss. Finding and employing such person could be a gold mine.
I'm not sure why being very agenty would necessarily mean that starting your own company should be the best bet. Being an employee lets you reap the benefits of specialization and others having taken all the risks for you (most new companies fail), and can be a very comfortable if you can just find a position that lets you use your skills to the fullest. Then you can focus on doing what you actually enjoy, as opposed to having to spend large amounts of extra energy to running a company.
There are plenty of reasons to work at a company even if you are very agenty. I work at a company where any advance I make may be turned in to cash across about 700 million chips a year that we sell. It is economic for my company to have me around doing whatever I feel like as long as, on average, i spit out really tiny improvements in chips every once in a while. A company selling the equivalent of 7 million chips a year would need about 100X as much innovation from me to get the same value from me that the bigger company gets.
In addition to my screwing around however I want being 100X as valauble for my employer than for most other potential employers (including my self), My employer provides me with an insanely excellent toybox. That toybox includes demo implementations of systems with chips that aren't even commercial yet, complete with the best imaginable tech support for using these demo platforms. It provides me with proprietary data it has gathered from its vast engineering force and from its vast customer connections.
If what I want to be agenty about is fairly narrow, then it is a gigantic win to work as an employee for a big successful company.
I just wrote two hundred times ten words like this, on why we should not use sun-colored stuff we pull out of the ground as money. I think I want to hurt myself.
Have you ever wondered what makes a light bright? Lights are bright because they are very hot, and hot things become bright. The hotter something gets, the brighter it will be.
But why are hot things bright? Everything is made up of many very tiny things, and these tiny things are moving. When things are hot, the tiny things move very, very fast, and when they are cool the tiny things move slower.
When the tiny things hit each other, they give off light. The faster they are going when they hit each other, the bluer the light they give off is. When they move faster, they also run into each other more often, so they give off more light. That means that when the tiny things move very, very slowly, the light they give off is too red for you to see.
That means as things get hot, they will become slightly red, then get brighter and turn toward the color of the sun, then get really bright and turn white.
Inside a light there is a long but not wide thing that gets very hot when you run power through it. It gets hot enough to be white and bright enough to light up a room.
Change after I put this up: I have a class that I have to write a twenty hundred word paper for. I am thinking about writing the whole paper like this. That would be a way to show that I don't need to use big words to write a good paper, and I also would not get bored while writing. This is fun.
Lateral inhibition, surprisingly easy:
A brain cell that makes a lot of noise can stop other cells that are close from talking at the same time.
Fourier synthesis, not as easy:
...When you hear a sound, little pieces of tight, heavy air or not packed, light air are hitting your ear. The way the inside of your ear moves can be used to figure out what the sound was like and make it again. We build things that are like big ears to make the sounds. The inside of the built ear pushes air so that it's tight and heavy again, instead of like before where the tight, heavy air pushed on your ear.
How far your inside ear part has moved at each moment can be drawn to make lines that goes up and down as it flies right. When a point of the funny line is in the middle, that shows the moments when the inside ear part wasn't moving or was moving between being more pushed inside or more pushed outside.
Given a funny line that shows what a sound was like, we can show how its shadow falls on different directions in the space of slowly changing lines that always look the same after you move along them a little way, or that much again, and so on. To do this though, you have to be able to state the area
There was a man called Mr Godel who lived before most of the people living now were living. He was very good and putting numbers together and thinking about things. He showed everyone that when you have a way of thinking about things, that way of thinking can't explain how it works from the inside. So that way of thinking could never be complete. This was considered very important by lots of bright people, and helped us understand how we think and make other things that think.
It seems like lots of people on LW want a slice of this "data science" pie that everyone keeps talking about. I know it's a highly ambiguous buzzword at the moment, but what would be a good syllabus for these people?
I'm cobbling my own together at the moment, (mostly consisting of R, NumPy, lxml and a lot of extracurricular linear algebra), but it never hurts to have a bit of extra structure. What should prospective "data scientists" be learning, and where can they find it?
We say "politics is the mindkiller" but ti seems an seperate question why certain political topics are more 'mindkillery' than others.
Recent example that brought this to mind is the conflict going on in Gaza, its unusual in that my friends and acquaintances who are normally fairly moderate and willing to see both sides on political topics are splitting very heavily onto opposite sides, and refusing to see the others point of view.
Sometimes the political opinions can result in direct actions, but that is rather rare today. (I guess it is not like you and your friends are going to volunteer as soldiers for the opposite sides in Gaza.) The biggest "action" most people do is giving their vote. One vote of a few millions... perhaps our brains are not able to work with values like this, so we feel like our friends have at least 5% of the votes each.
But even when the "real" consequences of our opinions are close to zero, social consequences remain. As long as other people are polarized about some issue, you opinion about conflict in Gaza is essentialy a decision to join the "team Israel" or "team Palestine". This choice is absolutely unrelated to the actual people killing each other in the desert. This choice is about whether Joe will consider you an ally, and Jane an enemy, or the other way. With high probability, neither Joe nor Jane are personally related to people killing each other in the desert, and their choices were also based on their preference to be in the same team with some other people. But having made their choice and joined a team, their monkey brains were re...
Yes. Forming a moral or political opinion about a conflict you cannot feasibly affect is like forming an opinion about a theory that you cannot feasibly test --- it's easy to form an opinion for bad reasons.
Why is an opinion on Gaza more likely to become a part of someone's (from Europe or America) identity than e.g. an opinion on Darfur?
For me it's mostly the network effect. I care more, because people around me care more. I also care more because I have more information, but that again is because people around me care more. If people around me stopped talking about Gaza, it would be just as easy to forget as Darfur.
What keeps this topic alive, is the memetic chain: Gaza is linked to Israel, which is linked to Jews, which is linked to Nazis, which is linked to WW2 and its aftermath, which is linked to our contemporary politics. Also Jews are linked to Old Testament, which is linked to Christianity; in USA, Israel is linked to Religious Right; and the religion is again linked to politics. This all together gives Gaza a high "Page Rank".
Darfur could get some "Page Rank" through the former colonies of European countries, but that link is much weaker and outdated.
The mindkilling emotions are not caused by the human suffering, but by pattern-matching it to the political situation around us. This triggers the feeling of "it could happen to me, too" and switches the brain to the battle mode.
On the "Preferences" page, it says that the Anti-Kibitzer only works with Firefox. I just tried it in Chrome, and it works. Someone should try other browsers, as well.
Please update the preferences page to reflect this.
On the other hand, his 1915 article on "The Ethics of War," he defended wars of colonization on the same utilitarian grounds: he felt conquest was justified if the side with the more advanced civilization could put the land to better use.
Damn another topic I should think about. Also I've been most pleasantly surprised by Bertrand Russell. He is the kind of pacifist who is willing to consider on utilitarian grounds a nuclear first strike at the USSR to stop it from getting nukes without being mind-killed.
GiveWell has updated its top charities list to add GiveDirectly alongside the Against Malaria Foundation and the Schistosomiasis Control Initiative. It has also updated its reviews of all three charities.
ETA: The associated blog post.
IBM's Watson is being used to do medicine better than doctors. I remember reading that it takes $3 million and a few years for Watson to learn a subject. I have a few questions about this:
Could you raise $3 million to create a Watson for intelligence amplification. Would people pay to have custom made nootropics stacks for them?
Will somebody do it eventually? Is it worth researching nootropics right now?
Would publishing a newspaper be an efficient way to raise the sanity waterline?
More specifically, I imagine a newspaper freely distributed to people living in a given area, financed by donations and advertising. It would contain interesting topics about science, both for beginners and experts. It would explain how the stuff works. It would avoid mindkilling topics, such as politics and religion. In some situations it could provide uncontroversial background information for some hot topics. It would contains some easy rationality exercises.
Why? It could mov...
People have been trying to do social engineering with print for hundreds of years and trying to educate the general populace to the scientific worldview for at least a century, and yet the sanity waterline is still as low as it is. Many intellectual subcultures have published pamphlets with contents that the in-group finds very convincing, yet they generally always end up ignored.
What would the newspaper be doing differently compared to the things that came before?
People have been trying to do social engineering with print for hundreds of years and trying to educate the general populace to the scientific worldview for at least a century, and yet the sanity waterline is still as low as it is.
Is it as low as it was?
A one page story from Ted Chiang I hadn't seen linked here before: http://www.concatenation.org/futures/whatsexpected.pdf
If you enjoyed it I strongly recommend reading more Ted Chiang.
I like Ted Chiang as well. My favorites are Liking What You See and Hell Is the Absence of God.
I think something is very, very wrong with me. Instead of feeling Lovecraftian horror upon reading that, I immediately began trying to think of ways to use Predictors to hack the universe. Here's what I came up with (you may want to try for yourself before reading further):
I've always found it funny how modern society is basically formally libertarian about sex and not nearly anything else. And how deontological Libertarians basically treat everything with the same ethical heuristics modern society uses for sex.
"Anything between consenting adults." and "The state has no buisness in my bedroom." don't seem like things that would only make sense for sex and the bedroom and practically nowhere else. This observation moved me towards thinking they make less sense for sex and the bedroom and more sense for ot...
I recently read Parent-Offspring Conflict by Trivers for my evolutionary psychology class. I strongly recommend it as it is one of the best readings from an already interesting class. Notably, it (partially) solves a problem I remember being addressed on Overcoming Bias in a way that I felt was unsatisfactory. Specifically, why parents encourage altruism and other pro-social values in their children. To summarize Trivers' position on the subject, reciprocal altruism, retribution, and reputation are often extended to a person's family. In general both...
Intrade stopped allowing US residents to participate in its prediction markets in response to a civil complaint filed by the US Commodity Futures Trading Commission.
We are sorry to announce that due to legal and regulatory pressures, Intrade can no longer allow US residents to participate in our real-money prediction markets.
Unfortunately this means that all US residents must begin the process of closing down their Intrade accounts. We strongly urge you to begin this process immediately.
...The U.S. Commodity Fu
Cross-posted from my blog because I thought there might be interest in this community as well. Keep in mind that I'm writing for an audience who hasn't read anything from Paul Graham. Comments welcome.
I recently read an article in Slate magazine about how miserable children are in middle school. One major reason is that children are very seldom explicitly trained on new ones ways of interacting with others. Instead, children are expected to pick up appropriate social skills just by observing the way other people behave towards each other.
Unfortunately, ...
I believe that (mathematical) proofs aren't easily reducible to axiomatic proofs, and that proofs have been, and still are, profoundly social by their nature, although I don't know if that will continue indefinitely. I probably won't find the time to write a large post on this topic that I've been thinking of, so I want to quote here one observation that's been on my mind recently, in case someone finds it useful.
Eliezer quotes (in the post which I, with accordance to the above, see as wrong-headed in some respects) one definition of proof that he disagree...
HT: gwern
Tryfon Tolides: an almost pure empty poetry
...Mrs. Moldbug once explained a terribly useful concept to me: the idea of Seventeen magazine. The point of Seventeen is that it's not for 17-year-olds. It's for 14-year-olds. As they say in the marketing department, it's aspirational.
Starbucks, similarly, is aspirational. If you're anyone who's anyone and you live in San Francisco or Berkeley, you will not set foot inside a Starbucks. (I once had this horrible fat hippie woman tell me this at a party. She was boasting that never once, in her entire life,
Prompted by this comment, looking for references on the reductionism and the laws of nature, specifically, to address this argument:
if physical laws are guiding "reality" than they are not reducible to quarks and leptons themselves, which does call the whole idea of reductionism in question.
Basically, where do the laws of "fundamental" physics fit in the map/territory model (trying to steel-man it for myself, given that I'm not a fan)? If they are in the territory, what does the ultimate reduction look like? Is Nature just a fancy m...
Genetically engineering for trehalose expression has been shown (2006, in vitro) potentially useful for fighting Huntington's and Parkinson's. Since these are mainstream markets with funding behind them, perhaps a gene therapy that does this for brain cells is in the cards. If so, that could be good as a cryonics pretreatment, as trehalose (a non-reducing disaccharide composed of two glucose monomers) is a good cryoprotectant.
Currently its role as a cryoprotectant is mainly prevention of ice outside of the cells, as it does not penetrate the lipid bilayer ...
META: Every time I visit Discussion, I see a sequence re-run, and a bunch of meetup notifications. Often new posts slip between the two, and I don't notice a new topic until I see a recent comment on it. Would it be infeasible to move the meetup notifications out of Discussion to reduce clutter on the page?
Vertical vs. Horizontal transmission ideology
Slightly edited from an IRC discussion with fellow rationalists.
...Now my mind is spinning, considering meta-arguments to Nth power for frivolous values of N, involving outcome-based complaints about resolution procedures.
If a world dictator receives a suggestion for something like Konkvistador's suggestion to split civilization into factions and see which does better isolated, there will be complaints "this test favors group X, it's unfair"
At the basic level, childbirth. At higher meta levels, things
There is a blog that I would care never to read again, even in moderation. I added the blog to my localhost list so now I can't visit the blog anymore. But my lizard-brain has found a workaround: if I google the blog I can read Google's cache. Is there a way to block Google's cache of the blog without blocking the rest of Google's functions?
Tell me about Dungeons and Dragons. I've never played, nor see anyone play. But I've been passively fascinated with this game for a long time.
I understand some broad strokes, but some paradoxes about those broad strokes bewilder me. The concept sounds open and freewheeling, like a creative writing exercise. But there's a long rule book, which makes it sound much more rigid. It also sounds like something that would appeal to people who like theater, or more generally being the center of attention. But it's famously popular in a community of shy outcas...
Most of the rule book is rules for character creation and combat. If you already have your character made up, you can get by with borrowing someone else's book.
What the game's like depends a lot on the Dungeon Master and the adventure he has prepared. It could be a lot of "roleplaying" that consists of your characters talking with NPCs with short bits of combat in between, or it could be a hack-and-slash dungeon crawl in which you kill monsters and take their stuff.
As for a video of people playing, although it's not exactly what you asked for, I'd recommend this.
So, I've been wondering: how much does buying something used benefit the producers of new products? How would you measure or predict that?
Meta karma-related question that occurred to me on reading the post on Retributive Downvoting, but which didn't really fit there: One thing that I sometimes do in upvoting/downvoting is to calibrate my vote based on how many up or down votes the comment already has; for example, if a comment is at plus 10, but I think it's only a tiny bit good, I might downvote it; whereas if a comment is at -10, but I think it's only a little bit bad, I may upvote it (whereas if the little bit good comment was at 1-2, I would upvote, and it the little bit bad comment was...
Bug report: when I went to a new user's page, I noticed that the 3rd, 5th, and 7th comments had working up/down vote buttons. Loading a second copy of the page, the buttons were gone. I have a screenshot of it, but no idea how to reproduce it, or why only some of the comments had buttons.
Any idea what's up? (Also, do we have an official spot to post bugs? If I've noticed it, I've since forgotten about it, and if I haven't noticed it, I suspect it's hidden a bit too well.)
I've recently become aware of the existence of the Lending Club which appears to be a peer-to-peer framework for borrowers and lenders. I find myself intrigued by the interest rates claimed, but most of what I've found in my own research indicates that these interest rate computations involve a lot of convenient assumptions. Also, apparently if the Lending Club itself goes bankrupt, there is no expectation that you will get your investment back.
It seems at least conceivable that the interest rates are actually that high, since it is a new, weird type o...
How do I improve my persuasion skills?
How do I translate a valid logical argument to a persuasive, intuitive argument that would work for most people? I have read a lot of psychological literature. I have also gotten to the point where I can recognize an intuitive argument that would be persuasive. So, I can recognise my arguments as non-persuasive before I say them and I avoid most debates with people who aren't convinced by scientific evidence and stuff that works for rationalists and is technically a good argument. However, generating a persuasive argument isn't the same as evaluating the persuasiveness of already existing arguments.
Any good, readable, concise literature on this?
I've recently learned that my mother has Sjogren's Syndrome, a pretty horrific autoimmune disease. Does anyone know about any research about healing (or minimizing the effects of) such diseases? Some supplement she could take that might give her better chances (without bad side effects), maybe? I've read about N-Acetylglucosamine, for example. Of course I don't expect a miracle cure, but I figure, if there's a chance something could help without harming her, why not try it?
Any help would be very, very appreciated.
I found an article explaining Motivated Reasoning in The Atlantic and it seemed like a good fit for the Open Thread.
That being said, one of the core links inside the article (The one that links to the paper that it is using to draw some of it's conclusions) was broken. I've pasted the correct link below if you want to read the paper as well.
Casuistry in early modern times
...The casuistic method was popular among Catholic thinkers in the early modern period, and not only among the Jesuits, as it is commonly thought. Famous casuistic authors include Antonio Escobar y Mendoza, whose Summula casuum conscientiae (1627) enjoyed a great success, Thomas Sanchez, Vincenzo Filliucci (Jesuit and penitentiary at St Peter's), Antonino Diana, Paul Laymann (Theologia Moralis, 1625), John Azor (Institutiones Morales, 1600), Etienne Bauny, Louis Cellot, Valerius Reginaldus, Hermann Busembaum (d. 1668), etc. On
argumentative antipattern ramblings from irc -
< mstevens> Imagine two people arguing. One is in favour of X, and hates Y, the other likes Y, and hates X
< mstevens> both X and Y are members of class a
< mstevens> There's a good chance that the X-supporter will argue "all things in a are perfectly fine!" in an attempt to support X
< mstevens> Then, the Y-support will say "aah, you must also like Y", and the X-support will basically splutter and try to deny this obvious conclusion
...
< Tenoke01> mstevens, an ex...
It is disturbingly easy to get people to do what you want.
Coax them to perform actions that are consistent with them being motivated to do what you want. Repeat until desired result. They will rationalize said motivations into existence.
Suppose you are an anti-natalist, what does efficent charity look like then? What is the most cost effective way to reduce the number of births? I imagine giving out cheap birth control in places undergoing a demographic transition is pretty ok?
straight number of births isn't the right metric you need number of births times misery per birth minus opportunity cost of one less person
Would there be interest in a rationalist jobs thread?
So people would post something like: Offer: Widget designer needed at [Place], Wage: X, Or, Person: Brief description and CV link.
Do you think this SMBC comic is supposed to be an allegory for something in particular? I suspect it might be, but I can't think of anything specific.
I'm dissatisfied with the notion that a conditional with a false antecedent is true. The way I think about conditionals is much closer to probabilistic conditionalization, in which, under an event space interpretation, P(A|B) is a statement about the subspace for which B is true, and has nothing at all to do with the cases in which not B. I tried using that idea to reduce the notion of "if a is true of c, then b is true of c" to "for all c such that a, also b", but then I found out that "such that" phrases in logic are just dr...
I was talking with someone the other day and they suggested something that sounded to me like discrete probability theory with ordinal weights. Does anyone know either: (a) specifically what (all? Almost all? Nearly none?) parts of Bayesian probability theory work out when you have to allow ordinal numbers as weights (I can specify what I mean by this if it is necessary) or (b) just generally speaking, Bayesian probability is about distributions on real vector spaces (as I see it), to what extent has anyone investigated these problems for modules over, ...
https://www.nytimes.com/2012/11/25/opinion/sunday/neuroscience-under-attack.html
What are your thoughts on this article? How can a layman discern between good and bad neuroscience in books?
A seemingly random question - is there anyone here by the last name of Carey (first name withheld), involved with THINK, who recently traveled from Australia to the US?
Can anyone point me toward work that's been done on the five-and-ten problem? Or does someone want to discuss it here? Specifically, I don't understand why it is a problem for probabilistic algorithms. I would reason:
There is a high probability that I prefer $10 to $5. Therefore I will decide to choose $5, with low probability.
And there's nowhere to go from there. If I try to use the fact that I chose $5 to prove that $5 was the better choice all along (because I'm rational), I get something like:
...The probability that I prefer $5 to $10 is low. B
I accept the terms of the contract.
If it's worth saying, but not worth its own post, even in Discussion, it goes here.
Just performed the AI-Box Experiment with a friend; I was the Gatekeeper. I let the AI out of the box. I am now thoroughly convinced that boxing would not be a successful strategy for ensuring AI is beneficial for humanity. Donating $10 to MIRI, since I lost.
Details, please!!