# All of selylindi's Comments + Replies

"Other people are wrong" vs "I am right"

Regarding the title problem,

I have historically been too hasty to go from “other people seem very wrong on this topic” to “I am right on this topic”

I think it's helpful here to switch from binary wrong/right language to continuous language. We can talk of degrees of wrongness and rightness.

Consider people who are smarter than those they usually argue with, in the specific sense of "smarter" where we mean they produce more-correct, better-informed, or more-logical arguments and objections. These people probably have some (binarily) wrong ideas. The pe

Contrite Strategies and The Need For Standards

Yes. Page 287 of the paper affirms your interpretation: "REMORSE does not exploit suckers, i.e. AllC players, whereas PAVLOV does."

The OP has a mistake:

Remorse is more aggressive; unlike cTFT, it can attack cooperators

Neither Remorse nor cTFT will attack cooperators.

The Pavlov Strategy
If Pavlov accidentally defects against TFTWF, the result is
D/C -> D/C -> D/D -> C/D -> D/D -> C/C,

Can you explain this sequence? I'm puzzled by it as it doesn't follow the definitions that I know about. My understanding of TFTWF is that it is "Tit for Tat with a small randomised possibility of forgiving a defaulter by cooperating anyway." What seems to be happening in the above sequence is Pavlov on the left and, on the right, TFT with a delay of 1.

4ErickBall3yI think what's being called "TFTWF" here is what some other places call "Tit for Two Tats [https://en.wikipedia.org/wiki/Tit_for_tat#Tit_for_two_tats]", that is, it defects in response to two defections in a row.
Complete Class: Consequentialist Foundations
in Critch's framework, agents bet their voting stake rather than money! The more bets you win, the more control you have over the system; the more bets you lose, the less your preferences will be taken into account.

If I may be a one-note piano (as it's all I've talked about lately on LW), this sounds extremely similar to the "ophelimist democracy" I was pushing. I've since streamlined the design and will try to publish a functional tool for it online next year, and then aim to get some small organizations to test it out.

An optimization process for democratic organizations
score voting is immune to the Gibbard-Satterthwaite theorem

I was basing this off the description in Wikipedia; please correct that entry if you think I was in error. As of this time it still explicitly states, "While the scope of this theorem is limited to ordinal voting, Gibbard's theorem is more general, in that it deals with processes of collective decision that may not be ordinal: for example, voting systems where voters assign grades to candidates."

any proportional method is subject to free riding strategy. And since this system is des
5Jameson Quinn3yOn Gibbard-Satterthwaite, you are wrong. Please read the original papers; Wikipedia is not definitive here. There is a sense in which the sentence you quote from Wikipedia is not quite wrong, but that sense is so limited that the conclusion you draw from it is not supported. In terms of the "craziest possible option" strategy: people may deliberately vote for something they believe will not win in order to "build up" voting power for later. When they decided to actually spend this built-up power, they would not vote for something crazy. Insofar as this strategy artificially increases their overall voting power over that of other voters, it undermines the fairness of the system. And in the worst case, it could backfire by actually electing a crazy option. In case of backfire, this would obviously not be a rational strategy ex post, but I believe the collective risk of such failed rationality is unacceptably high. As for the "rich irony" of me calling something a nonstarter politically: just this week, approval voting passed in Fargo; and STAR voting came within a few percent of passing in Lane County, OR. Last summer, thousands of people voted on the Hugo Awards which had been nominated through E Pluribus Hugo. In British Columbia, voters are currently deciding between four election methods, three of which are proportional and two to three of which have never been used. I personally played a meaningful role in each of these efforts, and a pivotal role in some cases. All of these are clearly far beyond "nonstarter politically". So yes, I'm not afraid to tilt at windmills sometimes, but sometimes the windmills actually are giants, and sometimes the giants lose. I believe I've earned some right to express an opinion about when that might be, and when it might not.
An optimization process for democratic organizations

Regarding exploitability by well-funded and well-connected entities - I'm not sure how to tell without an empirical test. My understanding is that research into funding of electoral campaigns doesn't show the funding as having any effect on vote totals. If that is accurate, then I'd expect it's still true under alternate voting methods.

An optimization process for democratic organizations

Fully agreed - the intention is to start with small-scale clubs and parts of private organizations that are open to experimenting.

An optimization process for democratic organizations
increased voting power given to those whose bills are not passed risks giving undue power to stupid or inhumane voters.

True. Equalizing the influence of all parties (over the long term at least) doesn't just risk giving such people power; it outright does give them power. At the time of the design, I justified it on the grounds that (1) it forces either compromise or power-sharing, (2) I haven't found a good way to technocratically distinguish humane-but-dumb voters from inhumane-but-smart ones, or rightly-reviled inhumane minorities from wrong... (read more)

1Aiyen3y"True. Equalizing the influence of all parties (over the long term at least) doesn't just risk giving such people power; it outright does give them power. At the time of the design, I justified it on the grounds that (1) it forces either compromise or power-sharing, (2) I haven't found a good way to technocratically distinguish humane-but-dumb voters from inhumane-but-smart ones, or rightly-reviled inhumane minorities from wrongly-reviled humane minorities, and (3) the worry that if a group's interests are excluded, then they have no stake in the system, and so they have reason to fight against the system in a costly way. Do any alternatives come to your mind?" 1. True, but is the compromise beneficial? Normally one wants to compromise either to gain useful input from good decision makers, or else to avoid conflict. The people one would be compromising with here would (assuming wisdom of crowds) be poor decision makers, and conventional democracy seems quite peaceful. 2. Why are you interested in distinguishing humane-but-dumb voters from inhumane-but-smart ones? Neither one is likely to give you good policy. Wrongly-reviled humane minorities deserve power, certainly, but rebalancing votes to give it to them (when you can't reliably distinguish them) is injecting noise into the system and hoping it helps. 3. True, but this has always been a trade-off in governance-how much do you compromise with someone to keep the peace vs. promote your own values at the risk of conflict? Again, conventional democracy seems quite good at maintaining peace; while one might propose a system that seeks to produce better policy, it seems odd to propose a system that offers worse policy in exchange for averting conflict when we don't have much conflict. "I may have been unduly influenced by my anarchist youth: I'm more worried about the negative effects of concentrating power than about the negative effects of distributing it. Is there any objective way to compare those effect
An optimization process for democratic organizations

Thanks for your thoughts. Your questions are quite valid but I'm inclined to punt on them, as you'll see:

For #3, it depends on the group. If a government were to use it, they could provide access via terminals in public libraries, schools, and other government facilities. If a private group were to use it, they'd probably just exclude the poor.

For #4, 6, 7, 8: It's intended for use in any democratic organization for the equivalent of ordinary legislation and bylaws, but not intended to replace their constitutions or founding documents... (read more)

An optimization process for democratic organizations

I’m seeking critique of this design. It combines SSC- and LessWrong-influenced thinking about optimization processes and utilitarianism with a long personal history of dabbling in groups that want to reform electoral processes. In my unquestionably rose-tinted judgment (it’s my baby!), I think Ophelimo has much in it that could be desired by everyone from the far right to the far left.

If there’s an error, I want to correct it. (Or to give up on it quickly, if there’s no way to correct it). If there’s an important criticism or technical limitation to addres... (read more)

Moloch's Toolbox (2/2)
This means that the organization with the best machine learning algorithm to estimate the bill score gets a lot of political power. ... I would expect a few big organisations to arrise that have heavy machine learning capabilities and that hold the power about bill making.

It's true I omitted the possibility of expending votes at a Vickrey auction level instead of an actual-bid level, so I grant the possibility that, if only one side had good polling data (implausible as that is), then they might buy votes a small fraction more cheaply. However,... (read more)

Moloch's Toolbox (2/2)

Er, is that agreement or an objection? It reads like an objection to me, though that could be the lack of body language. But the content agrees with the post, which explicitly states at both the beginning and the end that the system is designed to start very small and then grow.

Moloch's Toolbox (2/2)
SIMPLICIO: But then what, on your view, is the better way?

I'm not sure if I'm more Simplicio or more the Visitor, but ... the political side of it doesn't seem that hard to fix. At least, I've been advocating a set of improvements to futarchy that address all the political-structure inadequacies discussed here, as well as several others not discussed here. I know it's too much to hope that it's free of such inadequacies ... though I still hope, because I explicitly designed it with the intent to be free of them. So I decided ... (read more)

4ChristianKl4yThis means that the organization with the best machine learning algorithm to estimate the bill score gets a lot of political power. The same goes for the actual making of predicitions. This looks like short-term effects of the bill become more important than it's long-term effects. There are likely some interesting Goodhard's law problems where issues that benefit special interest groups aren't asked about in the polling questions and thus there's favors to be exchanged with those interests groups. I would expect a few big organisations to arrise that have heavy machine learning capabilities and that hold the power about bill making. In general the system is likely sufficiently intransparent that it's hard to understand whats happening and how the big organisations use their power to get benefits but they likely will be able to get benefits.
6ChristianKl4yWhen creating a new governance system, don't focus on creating something that's supposed to work at the scale of a country. Focus on something that works on a smaller scale. Student self-goverance at universities should be the perfect test bed for new governance models.
Why is the surprisingly popular answer correct?

In the Nature Podcast from January 26th, the author of the paper, Dražen Prelec, said that he developed the hypothesis for this paper by means of some fairly involved math, but that he discovered afterwards he needed only a simple syllogism of a sort that Aristotle would have recognized. Unfortunately, the interviewer didn't ask him what the syllogism was. I spent ~20 minutes googling to satisfy my curiosity, but I found nothing.

If you happen to know what syllogism he meant, I'd be thrilled to hear it. Also it would suit the headline here well.

0Stuart_Armstrong4yMaybe something like "there is one truth, but many ways of being wrong, so those who don't know the truth will spread their population estimates too widely?"

I think the most fun and empirical way to dissolve this confusion would be to hold a tourney. Remember the Prisoner's Dilemma competitions that were famously won, not by complex algorithms, but by simple variations on Tit-for-Tat? If somebody can host, the rules would be something like this:

1. Players can submit scripts which take only one input (their current money) and produce only one output (whether to accept the bet again). The host has infinite money since it's just virtual.
2. Each script gets run N times where N isn't told to the players in advance. The script with the highest winnings is declared Interesting.
2016 LessWrong Diaspora Survey Analysis: Part Four (Politics, Calibration & Probability, Futurology, Charity & Effective Altruism)

2) On political opinions, HBD is about an objective, falsifiable, scientifically mesureable characteristic of the world, whereas the other opinions are opinions.

That's not how I interpreted the item. This was a political quiz, so to my mind it's not about "Does X exist?", but "Do you favor more X than the status quo?" For example, liberal abortion laws exist, and feminism exists, and higher taxes exist. Similarly, HBD may exist depending on its precise definition. But what's politically relevant is whether you favor more liberal ... (read more)

0skeptical_lurker5yilliberal abortion laws exist too, in some countries. Maybe in future tests HBD should be split into "is HBD true?" and "do you want HBD to be true?" to avoid this sort of confusion. The government can raise taxes, it can't raise HBD.
The AI in Mary's room

"Within a week to a few months after surgery, the children could match felt objects to their visual counterparts."

i.e. not immediate, but rather requiring the development of experience

The AI in Mary's room

Let's take the AI example in a slightly different direction: Consider an AI built as a neural net with many input lines and output effectors, and a few well-chosen reward signals. One of the input lines goes to a Red Detector; the other input lines go to many other types of sensors but none of them distinguish red things from non-red things. This AI then gets named Mary and put into a black and white room to learn about optics, color theory, and machine learning. (Also assume this AI has no ability to alter its own design.)

Speculation: At the moment when... (read more)

0TheAncientGeek5yWe know what happens when blind people gain sight, and it isn't nothing,
Principia Compat. The potential Importance of Multiverse Theory

Hypothetical Independent Co-inventors, we're pretty sure you exist. Compat wouldn't be a very good acausal pact if you didn't. Show yourselves.

I'm one - but while the ideas have enough plausibility to be interesting, they necessarily lack the direct scientific evidence I'd need to feel comfortable taking them seriously. I was religious for too long, and I think my hardware was hopelessly corrupted by that experience. I need direct scientific evidence as an anchor to reality. So now I try to be extra-cautious about avoiding faith of any kind lest I be trapped again in a mental tarpit.

0MakoYass5yUnderstandable. You could definitely be right about that. Compat is a theory that boosts its own probability just for being thought about. Such theories are dangerous for rationalists with histories of spiritualism. Which, unfortunately for me, is a category that includes a large number of rationalists. Right now it's dangling by a few multiverse immortality precepts and some very extreme measure differentials and I'm not sure it's going to hold together in the end. You'll probably hear about it if it does. I might make a post even if it doesn't(could make an interesting parable about the tenacity of religion even under rationalism). Either way, when that happens, I'll update this thread. But don't check it too often. This can be practice for you in ignoring your triggers.
Linguistic mechanisms for less wrong cognition

"The father of my mother feels (passively) that that my left ringfinger touches him 2 centimeters in inferior direction from his right earlobe" (At the present he lies on his back, so inferior is not the direction towards the center of the earth).

tê ömmilek audyal íčawëla tê adlaisakenniňk qe oeksrâ’as oimřalik akpʰialîk êntô’alakuňk

There you go. :) It's a very literal translation but it's overly redundant. A hypothetical native speaker would probably drop the "audyal" verb, deframe "íčawëla", and rely more on Ithkuil's extensive case system.

1ChristianKl6yThanks, I update on it's capacity.
Linguistic mechanisms for less wrong cognition

responded to wrong person

[This comment is no longer endorsed by its author]Reply
Doomsday Argument Map

This is probably the wrong place to ask, but I'm confused by one point in the DA.

For reference, here's Wikipedia's current version:

Denoting by N the total number of humans who were ever or will ever be born, the Copernican principle suggests that humans are equally likely (along with the other N − 1 humans) to find themselves at any position n of the total population N, so humans assume that our fractional position f = n/N is uniformly distributed on the interval [0, 1] prior to learning our absolute position.

f is uniformly distributed on (0, 1) even aft

1Vaniver6yWhat would this imply about the total number of humans? If you knew that you were the 50th percentile human, for example, that would give you the total number of humans, and the same is true for all percentiles. I think the 'continuous' approach to the DA, which does not rely on the 'naturalness' of 1/20th, goes like the following: 1. Suppose across all of time there are a finite number of humans, and that we can order them by time of birth. 2. To normalize the birth orders, we can divide each person's position in the ordering by the total number of people, meaning each person corresponds to a fractile between 0.0 and 1.0. 3. My prior should be that my fractile is uniformly distributed between 0.0 and 1.0. 4. Upon observing that I am human number 108 billion, I can now combine this with my prior on the fractile to compute the estimated human population. 5. There is a 1% chance that my fractile is between 0.99 and 1.0, which would imply there is a 1% chance the total number of humans is between 109B and 108B. (The larger number is earlier because it corresponds to being the 99th percentile human instead of the 100th percentile human.) We now add this up for all possible fractiles, to get a distribution and a mean. 6. This is a tool for integration. If I'm number N and my fractile is f, then the total number of humans is N/f. So we integrate , which... does not converge [http://www.wolframalpha.com/input/?i=integrate+108000000000%2Ff+df+from+0+to+1] . The expected number of future humans is infinite! But while the expectation diverges, that doesn't mean the most likely value is infinite humans. The median value is determined by f=0.5, where there are only 108B more humans. In fact, for any finite number of humans, one can calculate the probability that there will be that many or fewer humans--which is why the last 95% of humans is relevant. Those
0turchin6yAs we are interested in future time of humanity existence we choose this interval. If we have different question, we may choose different intervals.
An overview of the mental model theory

FWIW, I still got the question wrong with the new wording because I interpreted it as "One ... is true [and the other is unknown]" whereas the intended interpretation was "One ... is true [and the other is false]".

In one sense this is a communication failure, because people normally mean the first and not the second. On the other hand, the fact that people normally mean the first proves the point - we usually prefer not to reason based on false statements.

The Brain as a Universal Learning Machine

New AI designs (world design + architectural priors + training/education system) should be tested first in the safest virtual worlds: which in simplification are simply low tech worlds without computer technology. Design combinations that work well in safe low-tech sandboxes are promoted to less safe high-tech VR worlds, and then finally the real world.

A key principle of a secure code sandbox is that the code you are testing should not be aware that it is in a sandbox.

So you're saying that I'm secretly an AI being trained to be friendly for a more advanced world? ;)

0jacob_cannell6yThat's possible given the sim argument. The eastern idea of reincarnation and the western idea of afterlife map to two main possibilities: in the reincarnation model all that is transferred between worlds is the architectural seed or hyperparameters. In the afterlife model the creator has some additional moral obligation or desire to save and transfer whole minds out.
Open Thread, Jun. 22 - Jun. 28, 2015

Does that name come from the old game of asking people to draw a bike, and then checking who drew bike gears that could actually work?

Open Thread, Jun. 22 - Jun. 28, 2015

Inspired by terrible, terrible Facebook political arguments I've observed, I started making a list of heuristic "best practices" for constructing a good argument. My key assumptions are that (1) it's unreasonable to expect most people to acquire a good understanding of skepticism, logic, statistics, or the ways the LW-crowd thinks of as how to use words rightly, and (2) lists of fallacies to watch out for aren't actually much help in constructing a good argument.

One heuristic captured my imagination as it seems to encapsulate most of the other he... (read more)

1[anonymous]6yOne thing you might want to consider is the reason people or posting on Facebook... usually, it's NOT to create a good argument, and in fact, sometimes a good, logical argument is counterproductive to the goal people have (to show their allegiance to a tribe).
0Elo6yyou might like www.yourlogicalfallacy.com
4ChristianKl6yAs far as I understand CFAR teaches this heuristic under the name "Gears-Thinking".

In theory, an annoyed person would have called "point of order", asked to move on, and the group would vote up or down. The problem didn't occur while I was present.

Futarchy and Unfriendly AI

There's no room for human feedback between setting the values and implementing the optimal strategy.

Here and elsewhere I've advocated* that, rather than using Hanson's idea of target-values that are objectively verifiable like GDP, futarchy would do better to add human feedback in the stage of the process where it gets decided whether the goals were met or not. Whoever proposed the goal would decide after the prediction deadline expired, and thus could respond to any improper optimizing by refusing to declare the goal "met" even if it technica... (read more)

Against the internal locus of control

Internals seem to do better at life, pace obvious confounding: maybe instead of internals doing better by virtue of their internal locus of control, being successful inclines you to attribute success internal factors and so become more internal, and vice versa if you fail. If you don't think the relationship is wholly confounded, then there is some prudential benefit for becoming more internal.

I'm willing to bet that Internals think there's a prudential benefit to becoming more internal and Externals think the relationship is wholly confounded.

In large formal groups: Robert's Rules of Order.

Large organizations, and organizations which have to remain unified despite bitter disagreements, developed social technologies such as RRoO. These typically feature meetings that have formal, pre-specified agendas plus a chairperson who is responsible for making sure each person has a chance to speak in an orderly fashion. Of course, RRoO are overkill for a small group with plenty of goodwill toward each other.

In small formal groups: Nonce agendas and rotating speakers

The best-organized small meetings I've e... (read more)

0gwillen6yWhat happens in the anarchist group if someone does not wish to relinquish the stick? (Perhaps the very ethos of the group makes this unlikely. But I'm curious if there's a method for dealing with people who, as you put it in the third part, "are too fond of their own voices".)
Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113

Hm, Harry can't lie in Parseltongue, meaning he can't claim what he doesn't believe, but he can probably state something of unclear truth if he is sufficiently motivated to believe it.

It'd be a nice irony if part of Harry's ultimate "rationality" test involves deliberately motivated reasoning. :D

Stupid Questions February 2015

Background: Statistics. Something about the Welch–Satterthwaite equation is so counterintuitive that I must have a mental block, but the equation comes up often in my work, and it drives me batty. For example, the degrees of freedom decrease as the sample size increases beyond a certain point. All the online documentation I can find for it gives the same information as Wikipedia, in which k = 1/n. I looked up the original derivation and, in it, the k are scaling factors of a linear combination of random variables. So at some point in the literature after t... (read more)

0Lumifer6yI am guessing that this is the default assumption of equal weighting or equal scaling of the variances that you are pooling. If you want to assign non-equal weights you should have some specific reason to do so. I don't think it's "superior", it's just the simplest default in the absence of any additional information.
The buildup to Jaynes?

familiar with applied mathematics at the advanced undergraduate level or preferably higher

In working through the text, I have found that my undergraduate engineering degree and mathematics minor would not have been sufficient to understand the details of Jaynes' arguments, following the derivations and solving the problems. I took some graduate courses in math and statistics, and more importantly I've picked up a smattering of many fields of math after my formal education, and these plus Google have sufficed.

Help us name the Sequences ebook

Think To Win: The Hard Part is Actually Changing Your Mind

(It's even catchier, and actively phrased, and gives a motivation for why we should bother with the hard part.)

How many words do we have and how many distinct concepts do we have?

That's not really how word usages spread in English. Policing usage is almost a guaranteed failure. What would work much better would be for you to use these words consistently with your ideals, and then if doing so helps you achieve things or write things that people want to mimic, they will also mimic your words. Compare to how this community has adopted all manner of jargon due to the influence of EY's weirdly-written but thought-reshaping Sequences! SSC is now spreading Yvain's linguistic habits among us, too, in a similar way: by creating new associations between them and some good ideas.

0[anonymous]7yYes, indeed. And I hope I have and do. However, how would I ever know if my use of "reality" is exactly what others mean by "universe" or "nature", if I didn't ask the question. How do you put reality together? Where does meaurement fit in? How does objectivity get accounted for? What does such a view have in common with physics and philosophy contemporarily and historically? I guess that's where I was headed with this discussion. Compare the structure of reality I've come up with to others here.
Superintelligence 14: Motivation selection methods

Bostrom's philosophical outlook shows. He's defined the four categories to be mutually exclusive, and with the obvious fifth case they're exhaustive, too.

1. Select motivations directly. (e.g. Asimov's 3 laws)
2. Select motivations indirectly. (e.g. CEV)
3. Don't select motivations, but use ones believed to be friendly. (e.g. Augment a nice person.)
4. Don't select motivations, and use ones not believed to be friendly. (i.e. Constrain them with domesticity constraints.)
5. (Combinations of 1-4.)

In one sense, then, there aren't other general motivation selection metho... (read more)

1William_S7yAnother approach might be to dump in a whole bunch of data, and hope that the simplest model that fits the data is a good model of human values (this is like Paul Christiano's hack to attempt to specify a whole brain emulation as part of an indirect normativity if we haven't achieved whole brain emulation capability yet: http://ordinaryideas.wordpress.com/2012/04/21/indirect-normativity-write-up/ [http://ordinaryideas.wordpress.com/2012/04/21/indirect-normativity-write-up/]). There might be other sets of data that could be used in this way, ie. run a massive survey on philosophical problems, record a bunch of people's brains while they watch stories play out in television, dump in DNA and hope it encodes stuff that points to brain regions relevant to morality etc. (I don't trust this method though).
Superintelligence 14: Motivation selection methods

If it really is a full AI, then it will be able to choose its own values.

I think this idea relies on mixing together two distinct concepts of values. An AI, or a human in their more rational moments for that matter, acts to achieve certain ends. Whatever the agent wants to achieve, we call these "values". For a human, particularly in their less rational moments, there is also a kind of emotion that feels as if it impels us toward certain actions, and we can reasonably call these "values" also. The two meanings of "values&quo... (read more)

Superintelligence 13: Capability control methods

Why is anthropic capture considered more likely than misanthropic capture? If the AI supposes it may be in a simulation and wants to please the simulators, it doesn't follow that the simulators have the same values as we do.

Superintelligence 12: Malignant failure modes

That would depend on it knowing what real-world physics to expect.

Superintelligence 12: Malignant failure modes

I feel like there are malignant failure modes beyond the categories mentioned by Bostrom. Perhaps it would be sensible to try to break down the topic systematically. Here's one attempt.

1. Design by fools: the AI does what you ask, but you asked for something clearly unfriendly.

2. Perverse instantiation & infrastructure profusion: the AI does what you ask, but what you ask turns out to be most satisfiable in unforeseen destructive ways, such as redirecting most resources to its infrastructure at our expense.

3. Partial perverse instantiation & mind cr

Superintelligence 11: The treacherous turn

Regarding the "treacherous turn" and the "conception of deception", I've previously proposed the following empirical method of testing friendliness:

• If only one AI is available, use a double-blind protocol, such that the friendliness of an AI is evaluated by someone who does not know that is her task, on an AI that is forbidden from conditioning its actions on whether it is being judged worthy of existing.
• Much, much better is to make many different AIs with many different attempts at friendly utility functions, inform them that you ha
Unemployment explanations

The specific example you gave doesn't sound promising, but you're entirely correct in the broader sense that my original post was unimaginative regarding possible solutions.

EDIT: It was worth an empirical try, so I tried your recommendation on a subset of applications. Zero responses from that group of companies.

CEV: coherence versus extrapolation

I've gone ahead and tried to flesh out this idea. It became so different than CEV that it needed a different name, so for now I'm calling it Constrained Universal Altruism. (This is the second revision.) Unfortunately I can't indent, but I've tried to organize the text as the comment formatting allows.

If anyone wants to criticize it by giving an example of how an AI operating on it could go horribly wrong, I'd be much obliged.

Constrained Universal Altruism:

• (0) For each group of one or more things, do what the group's actual and ideal mind (AIM) would
Open thread, Nov. 17 - Nov. 23, 2014

On the "all arguments are soldiers" metaphorical battlefield, I often find myself in a repetition of a particular fight. One person whom I like, generally trust, and so have mentally marked as an Ally, directs me to arguments advanced by one of their Allies. Before reading the arguments or even fully recognizing the topic, I find myself seeking any reason, any charitable interpretation of the text, to accept the arguments. And in the contrary case, in a discussion with a person whose judgment I generally do not trust, and whom I have therefore... (read more)

3Viliam_Bur7yJust wondering if it would make sense to consider everyone a Stupid Ally. That is, a good person who is just really really bad at understanding arguments. So the arguments they forward to you are worth examining, but must be examined carefully.
Unemployment explanations

Another friction is the stickiness of nominal wages. People seem very unwilling to accept a nominal pay cut, taking this as an attack on their status.

Salary negotiation is a complicated signalling process, indeed. I'm currently an unemployed bioengineer and have been far longer than I would have liked, and consequently I would be willing and eager to offer my services to an employer at a cut rate so that I could prove my worth to them, and then later request substantial raises. But this is impossible, because salary negotiations only occur after the c... (read more)

127chaos7yAre you very confident that this is the inevitable signal? I'd imagine that if you gave a well worded explanation of your circumstances this would not be so likely. Consider that your resume is not the sole means of communication available to you, this is not necessarily a one shot exchange of information. You could, for example, ask within your resume for them to use the interview to verify your claims of competency despite your willingness to accept a low salary. Or, you could try to speak to someone in person. If I am naively other-optimizing, please let me know. Apologies if so. I hope that's not the case, and that you find this potentially helpful.
A few thoughts on a Friendly AGI (safe vs friendly, other minds problem, ETs and more)

Basically its a challenge for people to briefly describe an FAI goal-set, and for others to respond by telling them how that will all go horribly wrong. ... We should encourage a slightly more serious version of this.

Thanks for the link. I reposted the idea currently on my mind hoping to get some criticism.

But more importantly, what features would you be looking for in a more serious version of that game?

0the-citizen7yI think I'd like the comments to be broadly organised and developed as the common themes and main arguments emerge. Apart from that a little more detail. I don't think it has to go into much implementation specifics, because that's a separate issue and requires a more highly developed set of math/CS skills. But I think we can make use of a broader set of smart brains by having this kind of discussion.
Superintelligence 5: Forms of Superintelligence

A higher quality intelligence than us might, among other things, use better heuristics and more difficult analytical concepts than we can, recognize more complex relationships than we can, evaluate its expected utility in a more consistent and unbiased manner than we can, envision more deeply nested plans and contingencies than we can, possess more control over the manner in which it thinks than we can, and so on.

A more general intelligence than us might simply have more hardware dedicated to general computation, regardless of what it does with that general ability.

1SteveG7yI am trying to turn this concept of Quality Intelligence into something more precise. Here are some items from history which most people will think of as improvements in quality intelligence. I am thinking about quality with the context of collective intelligence. The concept of AGI = the intelligence of a single human I do not find useful for predicting a recursively improving system, for reasons we can look at later. Development of symbolic language from pictographs Development of the number zero Development of set theory Invention of calculus Development of Newton's method for approximating functions Invention of Bayes' Rule Matrix theory Closed-form solutions to many kinnds of partial differential equations Procedural programming languages Approximations to vast numbers of functions using Newton's method on computers (Quality or Quantity?) These are advances in reasoning and improve intelligence quality. I am not sure whether to chalk up the following to advances in quality intelligence, or not: Formulation of gravity Development of the periodic table General relativity Demonstration of nuclear fission Development of the transistor Discovery of DNA Development of the microprocessor (Quality or quantity, or both?) Mechanisms of transcription and translation within the cell. Certainly, figuring all of these things out about the real world advanced our ability to solve practical problems. I am inclined to consider the distinction between them and the discoveries in logic, computer programming and applied math somewhat arbitrary.
Questions on Theism

The switch flipped for me when I was reading Jim Holt's "Why Does The World Exist?" and spent a while envisioning and working out the implications of Vilenkin's proposal that the universe may have started from a spherical volume of zero radius, zero mass, zero energy, zero any other property that might distinguish it from nothingness. It made clear to me that one could propose answers to the question "Why is there something rather than nothing?" without anything remotely like a deity.

Polymath-style attack on the Parliamentary Model for moral uncertainty

To avoid the timelessness issue, the parliament could be envisioned as voting on complete courses of action over the foreseeable future, rather than separate votes taken on each action. Then the deontologists' utility function could return 0 for all unacceptable courses of action and 1 for all acceptable courses of action.

SRG 4: Biological Cognition, BCIs, Organizations

Remember that effect where you read a newspaper and mostly trust what it says, at least until one of the stories is about a subject you have expertise in, and then you notice that it's completely full of errors? It makes it very difficult to trust the newspaper on any subject after that. I started Bostrom's book very skeptical of how well he would be handling the material, since it covers many different fields of expertise that he cannot hope to have mastered.

My personal field of expertise is BCI. I did my doctoral work in that field, 2006-2011. I endo... (read more)