All of dlthomas's Comments + Replies

Eliezer Yudkowsky Facts

I don't see that that makes other formulations "not Occam's razor", it just makes them less useful attempts at formalizing Occam's razor. If an alternative formalization was found to work better, it would not be MDL - would MDL cease to be "Occam's razor"? Or would the new, better formalization "not be Occam's razor"? Of the latter, by what metric, since the new one "works better"?

For the record, I certainly agree that "space complexity alone" is a poor metric. I just don't see that it should clearly be... (read more)

3[anonymous]7yThere's an intent behind Occam's razor. When Einstein improved on Newton's gravity, gravity itself didn't change. Rather, our understanding of gravity was improved by a better model. We could say though that Newton's model is not gravity because we have found instances where gravity does not behave the way Newton predicted. Underlying Occam's razor is the simple idea that we should prefer simple ideas. Over time we have found ways to formalize this statement in ways that are universally applicable. These formalizations are getting closer and closer to what Occam's razor is.
Eliezer Yudkowsky Facts

Is there anything in particular that leads you to claim Minimum Description Length is the only legitimate claimaint to the title "Occam's razor"? It was introduced much later, and the wikipedia article claims it is "a forumlation of Occam's razor".

Certainly, William of Occam wasn't dealing in terms of information compression.

2[anonymous]7yThe answer seems circular: because it works. The experience of people using Occam's razor (e.g. scientists) find MDL to be more likely to lead to correct answers than any other formulation.
Eliezer Yudkowsky Facts

What particular gold-standard "Occam's razor" are you adhering to, then? It seems to fit well with "entities must not be multiplied beyond necessity" and "pluralities must never be posited without necessity".

Note that I'm not saying there is no gold-standard "Occam's razor" to which we should be adhering (in terms of denotation of the term or more generally); I'm just unaware of an interpretaton that clearly lays out how "entities" or "assumptions" are counted, or how the complexity of a hypothes... (read more)

5[anonymous]7yMinimum description length [http://en.wikipedia.org/wiki/Minimum_description_length]. The MWI requires fewer rules than Copenhagen, and therefore its description is smaller, and therefore it is the strictly simpler theory.
The AI in a box boxes you

But good reason to expect it not to torture people at greater than the maximum rate its hardware was capable of, so if you can bound that there exist some positive values of belief that cannot be inflated into something meaningful by upping copies.

The Fabric of Real Things

I am not saying, "You value her continued existence, therefore you should believe in it." I am rather saying that your values may extend to things you do not (and will not, ever) know about, and therefore it may be necessary to make estimations about likelihoods of things that you do not (and will not, ever) know about. In this case, the epistemological work is being done by an assumption of regularity and a non-privileging of your particular position in the physical laws of the universe, which make it seem more likely that there is not anything special about crossing your light cone as opposed to just moving somewhere else where she will happen to have no communication with you in the future.

-2chaosmosis9yIt seems like a waste of time to think about things I can't ever know about. It seems like make-believe to place objective value on the existence of things that I'll never be able to experience or interact with. I don't understand why I should care about things that I will never ever know or experience. My values are broken insofar as they lead me to value abstract concepts in and of themselves, as opposed to physical things that I can interact with. I'd like to point out that my interpretation means I'll fight like hell to keep my daughter inside my light-cone, because I don't want to lose her. Your interpretation means you'll be content with the idea of your daughters existence in the abstract, and to me that's no different than belief in an afterlife. I point this out because I think the initial example emphasizes the "downsides" to my position while ignoring any corresponding "upsides" that it might entail. I thought about this. It turns out I've been using a modified version of Hume's problem of induction as the basis for my argument, in the back of my head. When it comes to real life and my future, I'm willing to temporarily discard the problem of induction because doing so brings me rewards. When it comes to things beyond my light cone and my experiences, I'm not, because there is no such reward and never could be. In other words, I have a heuristic in my head that says paradoxes only go away when it is pragmatic to ignore them, because otherwise you sacrifice mental accuracy. This heuristic means that I'm not willing to discard the problem of induction when it comes to experiences and existences beyond my range of interaction. Hopefully that makes my position clearer to you. It's not that I'm privileging my own position; it's that my position is the only one I have to work from and that I have no idea how things would work outside my light cone.
The Fabric of Real Things

Values aren't things which have predictive power. I don't necessarily have to be able to verify it to prefer one state of the universe over another.

-2chaosmosis9yYou can prefer that state, sure. But that doesn't mean that it is an accurate reflection of reality. The abstract idea of my daughters existence beyond the light cone is comforting, and would make me happy. But the abstract idea of my daughters existence in heaven is also comforting and would make me happy. I wish it were true that she existed. But I don't believe things just because they would be nice to believe. This is what I meant when I said that thought experiments were a bad way to think about these things. You've confused values and epistemology as a result of the ludicrously abstract nature of this discussion and the emotionally charged thought experiment that I had thrust upon me.
Firewalling the Optimal from the Rational

You're assuming that display of loyalty can radically increase your influence. My model was that your initial influence is determined situationally, and your disposition can decrease it more easily than increase it.

That said, let's run with your interpretation; Bot-swa-na! Bot-swa-na!

Firewalling the Optimal from the Rational

Because states are still a powerful force for (or against) change in this world, you are limited in the number of them you can directly affect (determined largely by where you and relatives were born), and for political and psychological reasons that ability is diminished when you fail to display loyalty (of the appropriate sort, which varies by group) to those states.

Also, apple pie is delicious.

7MixedNuts9yThen the obvious strategy is to start feeling lots of loyalty toward Easily Affected Country, and donate lots to organizations in Powerful Country that effect change in Easily Affected Country. This diminishes your political bonus but the extra leverage compensates. Bot-swa-na! Bot-swa-na! I actually think the apple pie reason is an unusually good one. There's nothing wrong with cheering for things.
Rationality Quotes October 2012

Irrelevant. The quote is not "If goods do cross borders, armies won't."

Bayes for Schizophrenics: Reasoning in Delusional Disorders

But one or more drawing-inferences-from-states-of-other-modules module could certainly exist, without invoking any separate homunculus. Whether they do and, if so, whether they are organized in a way that is relevant here are empirical questions that I lack the data to address.

What Is Signaling, Really?

What are the costs associated with flowers?

0CronoDAS9yYou have to spend time taking care of them.

I don't know enough about gardening to have a reasonable opinion on that, but here are some possibilities:

  • It takes resources in the form of studying fashion, or hiring someone else to do it for you, in order to know which flowers are in, and they're not the ones you expect. Compare to fashion in clothing; I'm probably unfashionable, and this correctly signals that I don't have enough time to keep up with trends or enough hip friends to advise me on them.

  • The high-status flowers are harder/more expensive to grow than the low status flowers. Compare to hi

... (read more)
1TheOtherDave9yNote that "more likely or less costly" is a disjunction. Which means it may be, on this article's account, that high-status flowers are not costlier than low-status flowers, merely that they are reliably less common among high-status flower-displayers. Of course, this also raises the possibility that this account is exactly backwards, and the only thing that makes the flowers "high-status" is the fact that high-status people display them; if high-status people started displaying bright red flowers in rotting wheelbarrows, that would shortly thereafter become a status signal. On that account, the flowers aren't a status signal at all. This is an empirical dispute; we can look at what happens when high-status people display low-status flowers. On the first account, we would expect the status of the people to go down (that is, third-party observers would think less well of them, keeping all other factors fixed). On the second account, we would expect the status of the flowers to go up. (My own expectation is that we would actually find it depends on several other factors, because both accounts are woefully oversimplified, but that something like what Yvain describes is in fact going on.)
1OphilaDros9yPossibly the the time spent in figuring out which ones are classy as opposed to 'wannabe' or 'cheesy' or 'trying too hard' or 'lower class'. Probably difficult to figure out for any given group to which you are signalling you belong, unless you actually do belong to that group.
2MileyCyrus9yThey're expensive and easy to misuse.
Problematic Problems for TDT

Could you construct an agent which was itself disadvantaged relative to TDT?

"Take only the box with $1000."

Which itself is inferior to "Take no box."

0[anonymous]9yOh, neat. Agents in "lowest terms", whose definitions don't refer to other agents, can't react to any agent's decision, so they're all at an epistemic disadvantage relative to each other, and to themselves, and to every other agent across all games.
0[anonymous]9yHow is agent epistemically inferior to agent ? They're both in "lowest terms" in the sense that their definitions don't make reference to other agents / other facts whose values depend on how environments depend on their values, so they're functionally incapable of reacting to other agents' decisions, and are on equivalent footing.
0[anonymous]9yHow is agent epistemically inferior to agent ? They're both constant decisions across all games, both functionally incapable of reacting to any other agent's actual decisions. Even if we broaden the definition of "react" so that constant programs are reacting to other constant programs, your two agents still have equivalent conditonal power / epistemic vantage / reactive footing.
Holden's Objection 1: Friendliness is dangerous

My point is that we currently have methods of preventing this that don't require an AI, and which do pretty well. Why do we need the AI to do it? Or more specifically, why should we reject an AI that won't, but may do other useful things?

0DanArmak9yThere have been, and are, many mass killings of minority groups and of enemy populations and conscripted soldiers at war. If we cure death and diseases, this will become the biggest cause of death and suffering in the world. It's important and we'll have to deal with it eventually. The AI under discussion not just won't solve the problem, it would (I contend) become a singleton and prevent me from building another AI that does solve the problem. (If it chooses not to become a singleton, it will quickly be supplanted by an AI that does try to become one.)
Holden's Objection 1: Friendliness is dangerous

Because there's no consensus, your version of CEV would not interfere, and the 80% would be free to kill the 20%.

There may be a distinction between "the AI will not prevent the 80% from killing the 20%" and "nothing will prevent the 80% from killing the 20%" that is getting lost in your phrasing. I am not convinced that the math doesn't make them equivalent, in the long run - but I'm definitely not convinced otherwise.

0DanArmak9yI'm assuming the 80% are capable of killing the 20% unless the AI interferes. That's part of the thought experiment. It's not unreasonable, since they are 4 times as numerous. But if you find this problematic, suppose it's 99% killing 1% at a time. It doesn't really matter.
Problematic Problems for TDT

If I'm right, we may have shown the impossibility of a "best' decision theory, no matter how meta you get (in a close analogy to Godelian incompleteness). If I'm wrong, what have I missed?

I would say that any such problem doesn't show that there is no best decision theory, it shows that that class of problem cannot be used in the ranking.

Edited to add: Unless, perhaps, one can show that an instantiation of the problem with particular choice of (in this case decision theory, but whatever is varied) is particularly likely to be encountered.

Problematic Problems for TDT

But doesn't that make cliquebots, in general?

Thoughts on the Singularity Institute (SI)

On the one hand a real distinction which makes a huge difference in feasibility. On the other hand, either way we're boned, so it makes not a lot of difference in the context of the original question (as I understand it). On balance, it's a cute digression but still a digression, and so I'm torn.

2Strange79yActually in the case of removing all oxygen atoms from Earth's gravity well, not necessarily. The AI might decide that the most expedient method is to persuade all the humans that the sun's about to go nova, construct some space elevators and Orion Heavy Lifters, pump the first few nines of ocean water up into orbit, freeze it into a thousand-mile-long hollow cigar with a fusion rocket on one end, load the colony ship with all the carbon-based life it can find, and point the nose at some nearby potentially-habitable star. Under this scenario, it would be indifferent to our actual prospects for survival, but gain enough advantage by our willing cooperation to justify the effort of constructing an evacuation plan that can stand up to scientific analysis, and a vehicle which can actually propel the oxygenated mass out to stellar escape velocity to keep it from landing back on the surface.
When is Winning not Winning?

Azathoth should probably link here. I think using our jargon is fine, but links to the source help keep it discoverable for newcomers.

Thoughts on the Singularity Institute (SI)

I wish I could vote you up and down at the same time.

2Strange79yPlease clarify the reason for your sidewaysvote.
Holden's Objection 1: Friendliness is dangerous

As long as we're using sci-fi to inform our thinking on criminality and corrections, The Demolished Man is an interesting read.

Holden's Objection 1: Friendliness is dangerous

Thank you. So, not quite consensus but similarly biased in favor if inaction.

Holden's Objection 1: Friendliness is dangerous

Assuming we have no other checks on behavior, yes. I'm not sure, pending more reflection, whether that's a fair assumption or not...

Holden's Objection 1: Friendliness is dangerous

It's a potential outcome, I suppose, in that

[T]here's nothing I prefer/antiprefer exist, but merely things that I prefer/antiprefer to be aware of.

is a conceivable extrapolation from a starting point where you antiprefer something's existence (in the extreme, with MWI you may not have much say what does/doesn't exist, just how much of it in which branches).

It's also possible that you hold both preferences (prefer X not exist, prefer not to be aware of X) and the existence preference gets dropped for being incompatible with other values held by other people while the awareness preference does not.

Holden's Objection 1: Friendliness is dangerous

My understanding is that CEV is based on consensus, in which case the majority is meaningless.

2DanArmak9yThere is absolutely no reason to think that the values of all humans, extrapolated in some way, will arrive at a consensus.

Some quotes from the CEV document:

Coherence is not a simple question of a majority vote. Coherence will reflect the balance, concentration, and strength of individual volitions. A minor, muddled preference of 60% of humanity might be countered by a strong, unmuddled preference of 10% of humanity. The variables are quantitative, not qualitative.

(...)

It should be easier to counter coherence than to create coherence.

(...)

In qualitative terms, our unimaginably alien, powerful, and humane future selves should have a strong ability to say "Wait! Stop! You

... (read more)
9Wei_Dai9yIf CEV doesn't positively value some minority group not being killed (i.e., if it's just indifferent due to not having a consensus), then the majority would be free to try to kill that group. So we really do need CEV to saying something about this, instead of nothing.
Holden's Objection 1: Friendliness is dangerous

Um, if you would object to your friends being killed (even if you knew more, thought faster, and grew up further with others), then it wouldn't be coherent to value killing them.

4[anonymous]9yJust because I wouldn't value that, doesn't mean that the majority of the world wouldn't. Which is my whole point.
Thoughts on the Singularity Institute (SI)

I am not the one who is making positive claims here.

You did in the original post I responded to.

All I'm saying is that what has happened before is likely to happen again.

Strictly speaking, that is a positive claim. It is not one I disagree with, for a proper translation of "likely" into probability, but it is also not what you said.

"It can't deduce how to create nanorobots" is a concrete, specific, positive claim about the (in)abilities of an AI. Don't misinterpret this as me expecting certainty - of course certainty doesn't ex... (read more)

2kalla7249yYou are correct. I did not phrase my original posts carefully. I hope that my further comments have made my position more clear?
Thoughts on the Singularity Institute (SI)

No, my criticism is "you haven't argued that it's sufficiently unlikely, you've simply stated that it is." You made a positive claim; I asked that you back it up.

With regard to the claim itself, it may very well be that AI-making-nanostuff isn't a big worry. For any inference, the stacking of error in integration that you refer to is certainly a limiting factor - I don't know how limiting. I also don't know how incomplete our data is, with regard to producing nanomagic stuff. We've already built some nanoscale machines, albeit very simple one... (read more)

8kalla7249yScaling it up is absolutely dependent on currently nonexistent information. This is not my area, but a lot of my work revolves around control of kinesin and dynein (molecular motors that carry cargoes via microtubule tracks), and the problems are often similar in nature. Essentially, we can make small pieces. Putting them together is an entirely different thing. But let's make this more general. The process of discovery has, so far throughout history, followed a very irregular path. 1- there is a general idea 2- some progress is made 3- progress runs into an unpredicted and previously unknown obstacle, which is uncovered by experimentation. 4- work is done to overcome this obstacle. 5- goto 2, for many cycles, until a goal is achieved - which may or may not be close to the original idea. I am not the one who is making positive claims here. All I'm saying is that what has happened before is likely to happen again. A team of human researchers or an AGI can use currently available information to build something (anything, nanoscale or macroscale) to the place to which it has already been built. Pushing it beyond that point almost invariably runs into previously unforeseen problems. Being unforeseen, these problems were not part of models or simulations; they have to be accounted for independently. A positive claim is that an AI will have a magical-like power to somehow avoid this - that it will be able to simulate even those steps that haven't been attempted yet so perfectly, that all possible problems will be overcome at the simulation step. I find that to be unlikely.
Thoughts on the Singularity Institute (SI)

It can't deduce how to create nanorobots[.]

How do you know that?

3kalla7249yWith absolute certainty, I don't. If absolute certainty is what you are talking about, then this discussion has nothing to do with science. If you aren't talking about absolutes, then you can make your own estimation of likelihood that somehow an AI can derive correct conclusions from incomplete data (and then correct second order conclusions from those first conclusions, and third order, and so on). And our current data is woefully incomplete, many of our basic measurements imprecise. In other words, your criticism here seems to boil down to saying "I believe that an AI can take an incomplete dataset and, by using some AI-magic we cannot conceive of, infer how to END THE WORLD." Color me unimpressed.
Thoughts on the Singularity Institute (SI)

But in the end, it simply would not have enough information to design a system that would allow it to reach its objective.

I don't think you know that.

Thoughts on the Singularity Institute (SI)

You'll have to forgive Eliezer for not responding; he's busy dispatching death squads.

0[anonymous]9yNot funny.
Thoughts on the Singularity Institute (SI)

The answer from the sequences is that yes, there is a limit to how much an AI can infer based on limited sensory data, but you should be careful not to assume that just because it is limited, it's limited to something near our expectations. Until you've demonstrated that FOOM cannot lie below that limit, you have to assume that it might (if you're trying to carefully avoid FOOMing).

7kalla7249yI'm not talking about limited sensory data here (although that would fall under point 2). The issue is much broader: * We humans have limited data on how the universe work * Only a limited subset of that limited data is available to any intelligence, real or artificial Say that you make a FOOM-ing AI that has decided to make all humans dopaminergic systems work in a particular, "better" way. This AI would have to figure out how to do so from the available data on the dopaminergic system. It could analyze that data millions of times more effectively than any human. It could integrate many seemingly irrelevant details. But in the end, it simply would not have enough information to design a system that would allow it to reach its objective. It could probably suggest some awesome and to-the-point experiments, but these experiments would then require time to do (as they are limited by the growth and development time of humans, and by the experimental methodologies involved). This process, in my mind, limits the FOOM-ing speed to far below what seems to be implied by the SI. This also limits bootstrapping speed. Say an AI develops a much better substrate for itself, and has access to the technology to create such a substrate. At best, this substrate will be a bit better and faster than anything humanity currently has. The AI does not have access to the precise data about basic laws of universe it needs to develop even better substrates, for the simple reason that nobody has done the experiments and precise enough measurements. The AI can design such experiments, but they will take real time (not computational time) to perform. Even if we imagine an AI that can calculate anything from the first principles, it is limited by the precision of our knowledge of those first principles. Once it hits upon those limitations, it would have to experimentally produce new rounds of data.
Thoughts on the Singularity Institute (SI)

Of those who attempted, fewer thought they were close, but fifty still seems very generous.

But Somebody Would Have Noticed

Why isn't it a minor nitpick? I mean, we use dimensioned constants in other areas; why, in principle, couldn't the equation be E=mc (1 m/s)? If that was the only objection, and the theory made better predictions (which, obviously, it didn't, but bear with me), then I don't see any reason not to adopt it. Given that, I'm not sure why it should be a significant* objection.

Edited to add: Although I suppose that would privilege the meter and second (actually, the ratio between them) in a universal law, which would be very surprising. Just saying that the... (read more)

0mistercow9yThe whole point of dimensional analysis as a method of error checking is that fudging the units doesn't work. If you have to use an arbitrary constant with no justification besides "making the units check out", then that is a very bad sign. If I say "you can measure speed by dividing force by area", and you point out that that gives you a unit of pressure rather than speed, then I can't just accuse you of nitpicking and say "well obviously you have to multiply by a constant of 1 m²s/kg". You wouldn't have to tell me why that operation isn't allowed. I would have to explain why it's justified.
Thoughts on the Singularity Institute (SI)

I don't think "certainty minus epsilon" improves much. It moves it from theoretical impossibility to practical - but looking that far out, I expect "likelihood" might be best.

2Bugmaster9yI don't understand your comment... what's the practical difference between "extremely high likelihood" and "extremely high certainty" ?
GAZP vs. GLUT

Things that are true "by definition" are generally not very interesting.

If consciousness is defined by referring solely to behavior (which may well be reasonable, but is itself an assumption) then yes, it is true that something that behaves exactly like a human will be conscious IFF humans are conscious.

But what we are trying to ask, at the high level, is whether there is something coherent in conceptspace that partitions objects into "conscious" and "unconscious" in something that resembles what we understand when we talk abo... (read more)

I Stand by the Sequences

Ah, fair. So in this case, we are imagining a sequence of additional observations (from a privileged position we cannot occupy) to explain.

Can You Prove Two Particles Are Identical?

In the macro scale, spin (ie rotation) is definitely quantitative - any object is rotating at a particular rate about a particular axis. This can be measured, integrated to yield (change in) orientation, etc.

In QM, my understanding is that (much like "flavor" and "color") the term is just re-purposed for something else.

Can You Prove Two Particles Are Identical?

Spin is qualitative. Qm is dealing with a degree of spin (spin up or spin down) which is quantitative.

I believe the distinction you want is "continuous" vs. "discrete", rather than "quantitative" vs. "qualitative".

I Stand by the Sequences

I think this might be the most strongly contrarian post here in a while...

5Normal_Anomaly9yAnd look! It's been upvoted! We are neither an Eliezer-following cult nor an Eliezer-hating cult! :)
I Stand by the Sequences

Not all formalizations that give the same observed predictions have the same Kolmogorov complexity[.]

Is that true? I thought Kolmogorov complexity was "the length of the shortest program that produces the observations" - how can that not be a one place function of the observations?

2JoshuaZ9yYes. In so far as the output is larger than the set of observations. Take MWI for example- the output includes all the parts of the wavebranch that we can't see. In contrast, Copenhagen only has outputs that we by and large do see. So the key issue here is that outputs and observable outputs aren't the same thing.
I Stand by the Sequences

(and there's the whole big-endian/little-endian question).

That's cleared up by:

I am number 25 school member, since I agree with the last and two more.

Thoughts on the Singularity Institute (SI)

There are only 7 billion people on the planet, even if all of them gained internet access that would still be fewer than 13 billion. In this case, instead of looking at the exponential graph, consider where it needs to level off.

People are a lot more complicated than neurons, and it's not just people that are connected to the internet - there are many devices acting autonomously with varying levels of sophistication, and both the number of people and the number of internet connected devices are increasing.

If the question is "are there points in sup... (read more)

0Monkeymind9yFYI ...A recent study by Cysco (I think) says something like: The internet is currently around 5 million terabytes with 75 million servers world wide. On average, one billion people use the internet per week. Internet use consumes enough information per hour to fill 7 million DVDs and growing, so an internet AI would need the capabilities of handling 966 exabytes of information by 2015. An Exabyte is 1,000,000,000,000,000,000 bytes. Every word ever spoken by human beings could be stored in 5 exabytes. Counting smart phones, robotic arms, cameras, GPS systems, clocks, home security systems, personal computers, satellites, cars, parking meters, ATMs, and everything else, there are more things connected to the internet than there are human beings on the planet. In a few years there will be over 50 billion with enough possible internet connections for 100 connections for each atom comprising the surface of the earth.
0JoshuaZ9yThat was essentially what I had in mind. Of course, getting that detailed a map of a brain would by itself already be way beyond what we have today.
Thoughts on the Singularity Institute (SI)

My point was just that there's a whole lot of little issues that pull in various directions if you're striving for ideal. What is/isn't close enough can depend very much on context. Certainly, for any particular purpose something less than that will be acceptable; how gracefully it degrades no doubt depends on context, and likely won't be uniform across various types of difference.

2Bugmaster9yAgreed, but my point was that I'd settle for an AI who can translate texts as well as a human could (though hopefully a lot faster). You seem to be thinking in terms of an AI who can do this much better than a human could, and while this is a worthy goal, it's not what I had in mind.
Thoughts on the Singularity Institute (SI)

One complication here is that you ideally want it to be vague in the same ways the original was vague; I am not convinced this is always possible while still having the results feel natural/idomatic.

2Bugmaster9yIMO it would be enough to translate the original text in such a fashion that some large proportion (say, 90%) of humans who are fluent in both languages would look at both texts and say, "meh... close enough".
Thoughts on the Singularity Institute (SI)

What if we added a module that sat around and was really interested in everything going on?

Thoughts on the Singularity Institute (SI)

They are a bit rambly in places, but they're entertaining and interesting.

I also found this to be true.

That's not what "realist" means in philosophy.

Load More