All of wuncidunci's Comments + Replies

Rationality Quotes April 2014

Hodges claims that Turing at least had some interest in telepathy and prophesies:

These disturbing phenomena seem to deny all our usual scientific ideas. How we should like to discredit them! Unfortunately the statistical evidence, at least for telepathy, is overwhelming. It is very difficult to rearrange one’s ideas so as to fit these new facts in. Once one has accepted them it does not seem a very big step to believe in ghosts and bogies. The idea that our bodies move simply according to the known laws of physics, together with some others not yet disc

... (read more)
Rationality Quotes April 2014

A video of the whole talk is available here.

6khafra8yAnd his textbook on the new univalent foundations of mathematics in homotopy type theory is here [].
2013 Survey Results

Did you mean Saint Boole?

And whence the blasphemy?

3AlexMennen8yYes, thanks. Fixed. I endorse Vaniver's explanation of the blasphemy.
6Vaniver8y1265 people are in group A. 947 are in group B, which is completely contained in A. Of all the people in group A, 450 satisfy property C, whereas this is true for 602 people in group B, all of whom are also in group A. 602 is larger than 450, so something has gone wrong.
I Will Pay $500 To Anyone Who Can Convince Me To Cancel My Cryonics Subscription

If someone believes they have a really good argument against cryonics, even if it only has a 10% chance of working, that is $50 in expected gain for maybe an hour of work writing it up really well. Sounds to me like quite worth their time.

Dr. Jubjub predicts a crisis

Quite possible. I didn't intend for that sentence to come across in a hostile way.

Since in Swedish we usually talk about the 1800s and the 1900s instead of the 19th and 20th century, I thought something could have been lost in translation somewhere between the original sources, the book by Kelly and gwern's comment, which is itself ambiguous as to whether it is intended as (set aside an island for growing big trees for making wooden warships) (in the 1900s) or as (set aside an island for growing big trees for (making wooden warships in the 1900s)). (I assumed the former)

Dr. Jubjub predicts a crisis

If we assume a scenario without AGI and without a Hansonian upload economy, it seems quite likely that there are large currently unexpected obstacles for both AGI and uploading. Computing power seems to be just about sufficient right now (if we look at supercomputers), so it probably isn't the problem. So it will probably be a conceptual limitation for AGI and a scanning or conceptual limitation for uploads.

Conceptual limitation for uploads seems unlikely, because were just taking a system cutting it up into smaller pieces and and solving differential equa... (read more)

Dr. Jubjub predicts a crisis

a Scandinavian country which set aside an island for growing big trees for making wooden warships in the 1900s, which was completely wrong since by that point, warships had switched to metal, and so the island became a nature preserve;

This was probably Sweden planting lots of oaks in the early 19th century. 34 000 oaks were planted on Djurgården for shipbuilding in 1830. As it takes over a hundred years for the oak to mature, they weren't used and that bit of the Island is now a nature preserve. Quite funny is that when the parliament was deciding this ... (read more)

4Pfft8yI guess gwern meant the construction was planned to take place in the 1900s.
The mechanics of my recent productivity

and the dark arts that I use to maintain productivity.

Yes! Please tell us more about these!

5So8res8yDelivered [].
Defining causal isomorphism

Two points of relevance that I see are:

If we care about the nature of morphisms of computations only because of some computations being people, the question is fundamentally what our concept of people refers to, and if it can refer to anything at all.

If we view isomorphic as a kind of extension of our naïve view of equals, we can ask what the appropriate generalisation is when we discover that equals does not correspond to reality and we need a new ontology as in the linked paper.

0badtheatre8yActually, I started thinking about computations containing people (in this context) because I was interested in the idea of one computation simulating another, not the other way around. Specifically, I started thinking about this while reading Scott Aaronson's review of Stephen Wolfram's book. In it, he makes a claim something like: the rule 110 cellular automata hasn't been proved to be turing complete because the simulation has an exponential slowdown. I'm not sure if the claim was that strong, but definitely it was claimed later by others that turing completeness hadn't been proved for that reason. I felt this was wrong, and justified my feeling by the thought experiment: suppose we had an intelligence that was contained in a computer program and we simulated this program in rule 110, with the exponential slowdown. Assuming the original program contained a consciousness, would the simulation also? And I felt strongly, and still do, that it would. It was later shown, If i'm remembering right, that there was a simulation with only polynomial slowdown, but I still think it's a useful question to ask, although the notion it captures, if it does so at all, seems to me to be a slippery one.
Book Review: Computability and Logic

van Dalen's Logic and Structure has a chapter on second order logic, but it's only 10 pages long.

Shapiro's Foundations without Foundationalism has as its main purpose to argue in favour of SOL, I've only read the first two chapters which give philosophical arguments for SOL, which were quite good, but a bit too chatty for my tastes. Chapters 3 to 5 is where the actual logic lives, and I can't say much about them.

1Gvaerg8yThanks, I'll check them out.
Book Review: Computability and Logic

Which edition did you read? The image in the post is of the fifth edition, and some people (eg Peter Smith in his Teach Yourself Logic (§2.7 p24)) claim that the earlier editions by just Boolos and Jeffrey are better.

Cutland's Computability and Mendelson's Introduction to Mathematical Logic between them look like they cover everything in this one, and they are both in MIRI's reading list. What is the advantage of adding Computability and Logic to them? (ie is it easier to start out with, does it cover some of the ground between them that both miss, or is it just good with alternatives?)

1So8res8yThe fifth -- I had not heard that. Thanks for the tip. I bet the Computability and Logic books on the course list cover similar subject matter. I read this book instead on the recommendation of Luke: I wanted to read up on provability logic specifically, and this book came recommended (presumably because it has an explicit introduction to provability logic at the end). I am now following it up with another of Luke's recommendations, which covers provability logic more specifically. I should probably refrain from suggestions about the content of the course list until after I read the suggested books on Logic & Computability, but I was quite impressed by the way this book took you from zero to Löb's theorem and made it all seem easy.
2013 Census/Survey: call for changes and additions

The questions on Smoking and Nicotine distinctly lack a middle question "Do you use some kind of smokeless tobacco?" (eg I don't smoke but use snuff almost daily).

Very Basic Model Theory

Cantor who first did the first work on infinite cardinals and ordinals seemed to have a somewhat mystic point of view some times. He thought his ideas about transfinite numbers were communicated to him from god, whom he also identified with the absolute infinite (the cardinality of the cardinals which is too big to itself be a cardinal). This was during the 19th century so quite recently.

I'd say that much mysticism about foundational issues like what numbers really are, or what these possible infinities actually mean, have been abandoned by mathematicians ... (read more)

Is it immoral to have children?

Coffee purchases seem to be done by near-mode thinking (at least for me), while having children is usually quite planned.

Personally I like giving myself quite a bit of leniency when it comes to impulsive purchases in order to direct my cognitive energy to long-term issues with higher returns. Compare and contrast to the idea of premature optimization in computer science.

Relevance of the STEPS project to (F)AI teams

Understanding the OS to be able to optimize better sounds somewhat useful to a self-improving AI.

Understanding the OS to be able to reason properly about probabilities of hardware/software failure sounds very important to a self-improving AI that does reflection properly. (obviously it needs to understand hardware as well, but you can't understand all the steps between AI and hardware if you don't understand the OS)

0Gunnar_Zarncke8yIf I remember correctly then a VHDL specification of hardware was also part of the STEPS project.
Engaging Intellectual Elites at Less Wrong

Private bittorrent trackers come to mind. Though over there, "good enough" is not measured by quality of conversation, but by your ability to keep up a decent ratio.

Harry Potter and the Methods of Rationality discussion thread, part 25, chapter 96

I've read it but didn't consider the possibility of a twist like that here as well.

Harry Potter and the Methods of Rationality discussion thread, part 25, chapter 96

My largest problem with the Dark Lord == Death theory is that it doesn't really square with Quirrelmort being another super-rationalist and Eliezer's First Law of Fanfiction (You can't make Frodo a Jedi unless you give Sauron the Death Star). Either Quirrelmort is a henchman or personification of Death, which is unlikely considering he is afraid of dying and the dementor try to frighten him in the Humanism arch. Or Quirrelmort is not the Sauron of this story but will help Harry to defeat the main bad guy Death. This could be a really cool ending, but I doubt that it would fit in the remaining arch.

3Vaniver8yI don't know, I think turning Sauron into death is comparable to giving Sauron the Death Star (i.e. your 'Quirrelmort is not Sauron' interpretation).
2ThrustVectoring8yRead Eliezer's short story "The Sword of Good". I half-expect a "The 'good' wizard is only playing the role and really isn't helping make the world be a better place, while the 'evil' wizard is actually the righteous one".
Harry Potter and the Methods of Rationality discussion thread, part 25, chapter 96

Dementors symbolise death. Dementors can destroy humans (by their kiss), and Harry can destroy dementors (by True Patronus). That if anything marks him as Death's equal. If not, dementors obeying him can be understood as him being Death's equal.

2hairyfigment8yYes, I was going to point out that "Make him go away," surely marked him as a monster or source of terror in someone's eyes.
Prisoner's Dilemma (with visible source code) Tournament

If they run your function from within theirs they simply tell the computer to start reading those instructions, possibly with a timer for stopping detailed in other parts of the comments. If they implement a VM from scratch they can mess with how the library functions work, for instance giving you a time that moves much faster so that your simulation must stop within 0.1s instead of 10 and they can run your code 100 different times to deal with randomness. Now implementing your own VM is probably not the optimal way to do this, you probably just want to do a transformation of the source code to use your own secret functions instead of the standard time ones.

0Decius9yI was considering simply measuring the behavior of my current opponent against programs that aren't me and determining their behavior as cooperatebot, mutualbot, defectbot, cooperate if they simulate me, cooperate against long programs, cooperate IFF they cooperate IFF I cooperate, or some other beast. That allows their behavior to depend on my behavior versus them, but my behavior only depends on their behavior versus third parties. I can't see a way to check another program for the desired IFF behavior without going beyond my skill level, but I think a mutualbot with a tiny chance of cooperating followed by a mutualbot with a tiny chance of defecting comes close; the first puts a lot of copies on the stack and then the top one cooperates unilaterally; if the response of the opponent is mutuality, it will be cooperate all the way down. If my opponent defects at his top level because I didn't cooperate for the right reason, it yields a defection... all the way down. A perfect program wouldn't do that, because it could see that it was probably in a simulation.
Prisoner's Dilemma (with visible source code) Tournament

Unless one of the contestants have time limits on their VM (or on their simulations in general). You can of clearly implement a VM where time goes faster just by pretending they have a slower processor than you really run on.

0Decius9yHrm- is it different if they run my function from within theirs instead of constructing a full VM? I was considering ways to communicate with a copy of myself that my simulation of my opponent was running that it was a simulation, but couldn't find any good ideas.
Prisoner's Dilemma (with visible source code) Tournament

Unless the other contestant wrote a virtual machine in which they are running you. Something which I think would be quite doable considering the ridiculously large time you've got (10s gives ~10^10 instructions).

1Decius9yWhen their VM runs your VM running their VM... it times out and everybody loses.
Post ridiculous munchkin ideas!

Hasn't been very consistent lately. Might try this later.

Welcome to Less Wrong! (5th thread, March 2013)

Not strictly speaking. Warning, what follows is pure speculation about possibilities which may have little to no relation to how a computational multiverse would actually work. It could be possible that there are three computable universes A, B & C, such that the beings in A run a simulation of B appearing as gods to the intelligences therein, the beings in B do the same with C, and finally the beings in C do the same with A. It would probably be very hard to recognize such a structure if you were in it because of the enormous slowdowns in the simulati... (read more)

0CCC9y...congratulations. I thought time travel would be a neccesity, I certainly didn't expect that intuition to be disproved so quickly. It may be speculative, but I don't see any glaring reason to disprove your hypothesised structure.
Probability is in the Mind

Your question is not well specified. Event though you might think that the proposition "its favorite ball is blue" is something that has a clear meaning, it is highly dependent on to which precision it will be able to see colours, how wide the interval defined as blue is, and how it considers multicoloured objects. If we suppose it would categorise the observed wavelength into one of 27 possible colours (one of those being blue), and further suppose that it knew the ball to be of a single colour and not patterned, and further not have any backgro... (read more)

Welcome to Less Wrong! (5th thread, March 2013)

You seem to be neglecting the possibility of a cyclical god structure. Something which might very well be possible in Tegmark level IV if all the gods are computable.

0CCC9yHuh. You are right; I had neglected such a cyclical god structure. That would appear to require time travel, at least once, to get the cycle started.
Boring Advice Repository

Note, according to my 30 seconds google scholar search, it is dipping/oral snuff that causes a higher risk of oral cancer. Nasal snuff seems safer (or perhaps less well researched).

Post ridiculous munchkin ideas!

That is true. However according to my experience you don't need to spend much time in the library itself if you know what you're looking for (you can always stay for the atmosphere). What takes time is going to and from the library. The value of this time obviously depends on a lot of parameters: is the library close to your route to/from some other place, are you currently very busy, do you enjoy city walks/bike-rides, etc.

Boring Advice Repository

Have you tried snuff? It smells quite nice and can help clear your nose as well as deliver nicotine.

0OrphanWilde9yI haven't. I did switch to a pipe, however, which works marvelously at delivering nicotine, in addition to smelling better, and carrying better social connotations. (Like snuff, it does carry a higher risk of oral cancer, but that's not -quite- as deadly.)
Post ridiculous munchkin ideas!

I've now tried f.lux for the past week or so. And now I'm disabling it. I like working late at night, and being a student in a term of revising but no lectures, I'm very flexible about what times I have to wake up. So it made me tired when I didn't want to be which was annoying. It did work very well at getting me to bed though, so I'll definitely reenable it when I want to go to bed earlier.

2Unnamed9yIf you have a late but fairly consistent bedtime, you can set your location several time zones to the west. f.lux kicks in at sunset in your reported location.
Post ridiculous munchkin ideas!

Many library catalogues are searchable online. So you just have to search a different site to wether they have it or not. If they have it, it's probably quicker to take a trip to the library than to wait for shipping.

1DSimon9yBut it requires active, exclusive use of time to go to a library, loan out a book, and bring it back (and additional time to return it), whereas I can do whatever while the book is en route.
LW Study Group Facebook Page

You could always create a throw-away FB account. But then of course it could be quite inconvenient to sign out of your usual account to sign in on this alternative one.

Let's make a "Rational Immortalist Sequence". Suggested Structure.

How is this in any way relevant?

If someone were to write the same proposal from the point of view of a sequence on how to most effectively maximize animal welfare through research and optimal philanthropy, it would hardly be relevant to discuss whether it is unconditionally good to maximize animal welfare. Sure this discussion might be useful to have, but when an article starts with "Suppose that" you don't start by fighting this hypothetical.

9shminux9yYou know what, you are right. I let my aversion to diego's writings get the better of me. I still don't like it when rationality is replaced by advocacy, but he did state it as a hypothetical.
Open Thread, February 1-14, 2013

Might work, depends on how inconspicuous and patient your were. Certainly not the first time people have been trying to recruit from/take over another organisation. Writing about it on the internet however will make what you're doing so much more obvious if someone started noticing.

3Viliam_Bur9yOne of the reasons I wrote this in Open Thread, instead of a separate article. :D But even if they notice, can they prevent it? I don't think so. If a group of rationalists decides to become Mensa members, who can stop them? If they pass the entry test (in my estimate, 9 of 10 would pass), they cannot be stopped from becoming Mensa members. If they are Mensa members, they cannot be denied information about the new tests, and they cannot be denied contact with the new members. Nothing in the current rules of Mensa prevents this. Actually, the whole "special interest group" system encourages this -- of course, assuming that the group wants to remain a subset of Mensa. So we just need to have a subset of rationalists who are both rationalists and Mensa members, and this subset is a completely valid group within Mensa. This is not even exceptional; for example there is a group of Mensa members who love classical music, and I assume nobody expects them to avoid non-Mensans who share the same hobby. In the same way, rationalist Mensans could have meetups with rationalist non-Mensans, and ignore the whole Mensa, except for fishing for new members. Sure, the same strategy could by used by... well, anyone, unless they are strongly anticorrelated with IQ. But it would be most useful for groups strongly correlated with IQ. Seems to me that rationalists are such group. (I assume that most high-IQ people are not rationalists, but most rationalists are high-IQ people.) Any other groups like this? Probably many of them, for example entrepreneurs, programmers, mathematicians, etc. But each of them already have their specialized communities, probably larger than Mensa, so it does not make sense for them. When we will have local rationalist communities of size comparable with local Mensa, it will stop making sense for us too. But today, we are not there yet (at least in my country).
Singularity Institute is now Machine Intelligence Research Institute

The same, though Star Trek comes up second. Though google uses a lot of other info about your computer to determine the results (like IP-adress and browser details).

8Xachariah9yI had overlooked that I was trying to hide from Google using their own software. Of course they wouldn't have provided it if it worked against them. Silly of me in retrospect. A handful of proxies verify that it is indeed Malaysia first and Star Trek second.
Singularity Institute is now Machine Intelligence Research Institute

When I searched the first hit was the Malaysian town called Miri. Looks like an example of filter bubbles.

2Xachariah9yWhat do you get when you use incognito mode? I checked with that and got the same Star Trek result. I think incognito pops Google's filter bubble, although I'm not certain. Though other search engines do give me the Malaysian town.
0FiftyTwo9ySame, followed by "Mid-Infrared Instrument "
4[anonymous]9yddg agrees on "miri" being the Malaysian town. []
DRAFT:Ethical Zombies - A Post On Reality-Fluid

Let N=3^^^^^^3, surely N nice world + another nice world is better than N nice worlds + a torture world. Why? Because another nice world is better than a torture world, and the prior existence of the N previous worlds shouldn't matter to that decision.

What about the probability of actually being in the torture world which is tiny 1/(N+1), the expected negative utility from this must surely be so small it can be neglected? Sure, but equally the expected utility of being the master of a torture world with probability 1/(N+1) can be neglected.

What this post tells me is that I'm still very very confused about reality fluid.

-2MugaSofer9yThe torture world, in this case, is being used to satisfy the whims of the Niceworld's residents. Lots of Niceworld copies = lots of Reality = lots of utility. So goes the logic. Since they are all the same, they can share a torture world.
Second-Order Logic: The Controversy

I would have done the following if I had been asked that: calculate which numbers I would have time to count up to before I was thrown out/got bored/died/earth ended/universe ran out of negentropy. I would probably have to answer I don't know, or I think X is a number for some of them, but it's still an answer, and until recently people could not say wether "the smallest n>2 such that there are integers a,b,c satisfying a^n + b^n = c^n" was a number or not.

I'm not advocating any kind of finitism, but I agree that the position should be taken seriously.

Godel's Completeness and Incompleteness Theorems

The standard approach in foundations of mathematics is to consider a special first order theory called ZFC, it describes sets, whose elements are themselves sets. Inside this theory you can encode all other mathematics using sets for example by the Von Neumann construction of ordinals. Then you can restrict yourself to the finite ordinals and verify the Peano axioms, including the principle of induction which you can now formulate using sets. So everything turns out to be unique and pinned down inside your set theory.

What about pinning down your set theory... (read more)

1[anonymous]9yThe thing about ZFC is that it doesn't feel like "the definition" of a set. It seems like the notion of a "set" or a "property" came first, and then we came up with ZFC as a way of approximating that notion. There are statements about sets that are independent of ZFC, and this seems more like a shortcoming of ZFC than a shortcoming of the very concept of a set; perhaps we could come up with a philosophical definition of the word "set" that pins the concept down precisely, even if it means resorting to subjective concepts like simplicity or usefulness. On the other hand, the word "set" doesn't seem to be as well-defined as we would like it to be. I doubt that there is one unique concept that you could call "the set of all real numbers", since this concept behaves different ways in different set theories, and I see no basis on which to say one set theory or another is "incorrect".
Ideal Advisor Theories and Personal CEV

I think Sobel's fourth objection is confused about what an idealized/extrapolated agent actually would want. If it had the potential to such perfect experience that makes the human condition look worse than dead in comparison, then the obvious advice is not suicide, but rather to uplift the ordinary human to its own level. This should always be possible since we must already have achieved achieved this to create the extrapolated agent to make the decision, so we can just repeat this process at full resolution on the original human.

Godel's Completeness and Incompleteness Theorems

The reason why compactness is not provable from ZF is that you need choice for some kinds of infinite sets. You don't need choice for countable sets (if you have a way of mapping them into the integers that is). You can get a proof of compactness for any countable set of axioms by proving completeness for any countable set of axioms, which can be done by construction of a model as in Johnstone's Notes on Logic and Set Theory p. 25.

Harry Potter and the Methods of Rationality discussion thread, part 18, chapter 87

I did not say this view of it would be accurate or rational. Hermione was however very upset in this scene and already saw Draco as a rival for Harry's friendship.

Harry Potter and the Methods of Rationality discussion thread, part 18, chapter 87

Considering how Hermione reacted to the Science-with-Draco bit we can guess her reaction to might-marry-Draco-instead. Would totally look to her like Harry tried to keep his options open depending on how his orientation turned out after puberty.

3DanArmak9yI would have to support Harry here. Asking someone to declare love before they even know their sexual orientation is completely irrational.
Causal Universes

The hypothesis that should interest an AI are not necessarily limited to those it can compute but to those it could test. A hypothesis is useless if it does not tell us something about how the world looks when it's true as opposed to when it's false. So if there is a way for the AI to interact with the world such that it expects different probabilities of outcomes depending on whether the (possibly uncomputable) hypothesis holds or not then it is something worth having a symbol for, even if the exact dynamics of this universe cannot be computed.

Let's consi... (read more)

Causal Universes

Consider instead of time traveling from time T' to T, that you were given a choice at time T which of the universes you would prefer: A or B. If B was better you would clearly pick it. Now consider someone gave you the choice instead between B and "B plus A until time T' when it gets destroyed". If A is by itself a better universe than nothing, surely having A around for a short while is better than not having A around at all. So "B plus A until time T' when it gets destroyed" is better than B which in turn is better than A. So if you w... (read more)

Centre for the Study of Existential Risk (CSER) at Cambridge makes headlines.

To be fair the Guardian story only references Terminator in the header. The text body is written by Lord Martin Rees and is a short but clear description of X-risk without any sci-fi references. It also focuses more on other X-risks, perhaps a difference in opinion amongst the founders?

6Sean_o_h9yTallinn and Price are very concerned with AI-related Xrisk. Martin Rees currently considers biological risks his no.1 concern (which is not to say he's unconcerned by AI); he's famously offered bets on a major (~1 million death) bio-related catastrophe occuring in the coming years. []

("Lord Martin Rees is a British cosmologist and astrophysicist. He has been Astronomer Royal since 1995 and Master of Trinity College, Cambridge since 2004. He was President of the Royal Society between 2005 and 2010". For anyone like me who didn't know.)

Proofs, Implications, and Models

The fact that if we put any two objects into the same (previously empty) basket as any other two object we will in this basket have four objects is true before we can make any definitions. But the statement 2 + 2 = 4 does not make any sense before we have invented: (a) the numerals 2 and 4, (b) the symbol for addition + and (c) the symbol for equality =. When we have invented meanings for these symbols (symbols as things we use in formal manipulations are quite different from words and were invented quite late, much later than we started to actually solve ... (read more)

Proofs, Implications, and Models

Another data point: in Cambridge the first course in logic done by mathematics undergraduates is in third year. It covers completeness and soundness of propositional and predicate logic and is quite popular. But in third year people are already so specialised that probably way less than half of us take it.

Stuff That Makes Stuff Happen

I think the division into problems and exercises usually seen in mathematics texts would be useful: A task is considered an exercise if it's routine application of previous material, it's a problem if it requires some kind of insight or originality. So far most of the Koans have seemed more like problems than like exercises, but depending on content both may be useful. I might be slightly biased towards this as I greatly enjoy mathematics texts and am used to that style.

7Eliezer Yudkowsky9ySo... the main thing I want to convey over and above "exercise" is that rather than there being a straightforward task-to-solve, you're supposed to ponder the statement and say, "What do I think of this?" A word other than "koan" which conveys this intent-to-ponder would indeed be appreciated.
9fubarobfusco9y"Problem" suggests something different in philosophy than in math. A philosophy "problem" is a seeming dilemma, e.g. Gettier, Newcomb's, or Trolley. So I'd suggest "exercise" here. "Exercise" dominates "kōan" in that both have the sense of something to stop and think about and try to solve, but ① "exercise" avoids the misconstrual of Zen practice (the purpose of a Zen kōan is not to come up with a solution, nor to set up for an explanation), ② the Orientalism (the dubiosity of saying something in Japanese to make it sound 20% cooler), and ③ the distraction of having to explain what a kōan is to those who don't know the word. EDIT: The claim that a purpose of a Zen kōan is not to come up with a solution appears to be a matter of disagreement, so discount ①. I think ② and ③ stand, though.
Load More