All of Khaled's Comments + Replies

I can't think of another way to reason - does our brain dictate our goal, or receives a goal from somewhere and makes an effort to execute it accurately? I'd go with the first option, which to me means that whatever our brain (code) is built to do is our goal.

The complication in the case of humans might be the fact that we have more than one competing goal. It is as if this robot has a multi-tasking operating system, with one process trying to kill blue objects and another trying to build a pyramid out of plastic bottles. Normally they can co-exist somehow... (read more)

I was thinking of, y'know, biased people. Also known as me and everyone I've ever met. Telling them "don't worry, whatever you're already doing is what you really want" does not seem Friendly.

I like Kahneman's lecture here as it sums up the distinction nicely (thought it's a bit long) Edit: not sure if a post on LW exists though

You'd be taking $3 from the experimenters, but in return giving them data that represents your decision in the situation they are trying to simulate (which is a situation where only the two experimentees exist), though your point shows they didn't mange to set it up very accurately.

I realize it will be difficult to ignore the fact you mentioned once you notice it, just pointing out that not noticing it can be more advantageuos for the experimenter and yourself (not the other experimentee) - maybe another strategic ignorance

It might be of help to include elements of rationality within each course, in addition to a ToK course on it's own. For example, in physics it might be useful to teach theories that turned out to be incorrect, and to analyze how and why it seemed correct at one point of time, and by collecting more evidence etc. it turned out incorrect.

Perhaps this is too difficult to include in current curriculums, so it can be included in the ToK course as additional discussions? Kind of an application or case study of Bayes' theorem (it could be prone to hindsight bias, so this has to be taken into consideration, not to make the errors in the theory seem so obvious)

The IB courses try to integrate ToK into all the various other subjects, in the form of short in class discussions. For example, in Economics it points out that a deterioration of Terms of Trade isn't necessarily a bad thing, and then questions the use of language in restricting knowledge from passing outside the educated. Or in Physics, we talk about the various theories that attempted to explain things and failed. So we kinda do that. However, it's only really taken as seriously as the students want it to be. In Economics, we never really talk about it, but it's in the textbook. In Physics, we mention it briefly, and then maybe argue about it because we're argumentative.
IB already does a bit of this in the sciences. For example, the physics curriculum includes the evolution of the various models of the atom and the experiments that supported them (like the Geiger–Marsden experiment). The biology curriculum did this a bit as well, with its coverage of cell theory and evolution.

In relation to connectionism, wouldn't that be the expected behavior? Taking the example of Tide, wouldn't we expect "ocean" and "moon" to give a headstart to "Tide" when the "favorite detergent" fires up all detergent names in the brain. But we wouldn't expect "Tide", "favorite", and "why" to give a headstart to "ocean" and "moon"?

Perhaps the time between eliciting "Tide" and asking for the reason for choosing it would be relevant (since asking for the reason while the "ocean" and "moon" are still active in the brain can give more chance for choosing them as the reason)?

The idea of "virtual machines" mentioned in [Your Brain is (almost) Perfect] ( is tempting me to think in the direction of "reading a manual will trigger the nuerons involved in running the task and the reinforcements will be implemented on those 'virtual' runs".

How reading a manual will trigger this virtual run can be answered by the same way hearing "get me a glass of water" will trigger the neurons to do so, and if I get a "thank you" it will ... (read more)

Can we know the victory condition from just watching the game?

Just to underscore a broader point: recreational games have various characteristics which don't generalise to all situations modelled game-theoretically. Most importantly, they're designed to be fun for humans to play, to have consistent and explicit rules, to finish in a finite amount of time (RISK notwithstanding), to follow some sort of narrative and to have means of unambiguously identifying winners. Anecdotally, if you're familiar with recreational games, it's fairly straightforward to identify victory conditions in games just by watching them being played, because their conventions mean those conditions are drawn from a considerably reduced number of possibilities. There are, however, lots of edge- and corner-cases where this probably isn't possible without taking a large sample of observations.

So if the blue-minimising robot was to stop after 3 months (the stop condition is measured by a timer), can we say that the robot's goal is to stay "alive" for 3 months? I cannot see a necessry link between deducing goals and stopping conditions.

A "victory condition" is another thing, but from a decision tree, can you deduce who loses (for Connect Four, perhaps it is the one who reaches the first four that loses).

By "victory condition", I mean a condition which, when met, determines the winning, losing and drawing status of all players in the game. A stopping rule is necessary for a victory condition (it's the point at which it is finally appraised), but it doesn't create a victory condition, any more than imposing a fixed stopping time on any activity creates winners and losers in that activity.

But if whenever I eat dinner at 6I sleep better than when eating dinner at 8, can I not say that I prefer dinner at 6 over dinner at 8? Which would be one step over saying I prefer to sleep well than not.

I think we could have a better view if we consider many preferences in action. Taking your cyonics example, maybe I prefer to live (to a certain degree), prefer to conform, and prefer to procrastinate. In the burning-building situation, the living preference is playing more or less alone, while in the cryonics situation, preferences interact somewhat like oppsite forces and then motion happens in the winning side. Maybe this is what makes preferences seem like varying?

Or is it that preferences are what you get when you consider future situations, in effect removing the influence of your instincts? If I consistently applied the rationale to both situations (cryonics, burning building), and came up with the conclusion that I would prefer not to flee the burning building, that might make me a "true rationalist", but only until the point that the building was on fire. No matter what my "preferences" are, they will (rightly so) be over-ridden by my survival instincts. So, is there any practical purpose to deciding what my preferences are? I'd much rather have my instincts extrapolated and provided for.

"Yvain, don't tell tornadoes what to do"

When calculting the odds of the winning/losing/machine defect, shouldn't you add the odds of the Many Worlds hypothesis being true? Perhaps wouldn't affect the relative odds, but might change odds in relation to "moderately rich/poor and won't try Quantum Immortality"


I think one useful thing is to try and find out why some explanations are more plausible than others (which seems standard, the fact of which explanation is actually true then, won't affect the guess that much).

When asked a question by an experimenter, I imagine myself trying to give a somewhat quick answer (rather than ask the experimenter to repeat the experiment isolating some variables so that I can answer accurately). I imagine my mind going through reasons until it hits a reason that sounds like ok, i.e. would convince me if I heard it from someone ... (read more)

Given what we know about the mutable nature of memories, I'd predict subjects to actually do much worse after thinking about it for 30 minutes.

One thing that amazes me is the change over time of this desire/goal divide. Personally, with things like regular exercise, I find that in times of planning my brain will seem in complete coherence - admitting my faults for not exercising and putting forth a plan which seems agreeable to both the consious and unconcious. Once the timefor exercise comes, the tricks start playing.

Maybe the moment of coherence could be somehow captured to be used in the moment of tricks? Also, would those moments be useful to avoid unconscious signalling?

If, when planning exercise, you come up with a plan for exercising that seems like it will work, despite other plans failing in the past (even when those plans seemed likely to succeed despite even earlier plans failing), and your plan fails, that should be a sign that you're not accurately assessing the appeal of your plan to your various selves. I'd suggest eliciting the "moment of tricks" while you're planning. Maybe if you try imagining that you're going to go exercising right now, you'll perceive all the tricks your subconscious can come up with for not exercising. And then you can think of ways to counter those tricks.

Could that be of any value: trends

Maybe using social network websites could generate a better turnover in the short term

Quick ananlysis: * Main search terms are variants on empiricism and philosophy * News stories seem to use it positively in political contexts. * Interestingly there seem to be fairly consistent peaks and troughs in this, possibly correlating with academic year? * US geography shows high polarisation between states, don't know enough about US demographics to guess why, maybe universities? * Worldwide Philippines is highest, no idea why...


Where does this fit with the idea that voluntary behavior can become involuntary in time. Like driving, where you start by fully thinking (consciously) of each move, and in time it becomes unconscious (not sure if we can call it involuntary). This was discussed a bit by Schrodinger in What is Life.

Will this, now unconsious action, be susceptible to reinforcement? If you find you make lots of accidents, maybe driving will jump back to voluntary?

3Swimmer963 (Miranda Dixon-Luinenburg) 12y
I'm not sure if automatic, learned behavior such as driving is at all the same as truly 'involuntary' behavior, like bile secretion and body temperature/heart rate. To me, it seems that the voluntary behavior of driving has just been 'chunked' so that decisions occur at a higher level. Instead of deciding 'ok, now I will put on my turn signal, I will slow down and check my blind spot and look in my mirrors and if there's no cars coming I'll make a left-hand turn', the decision is 'i'll turn left on the way to Suzy's house.' If the route to Suzy's house is also memorized to the point of being automatic, the voluntary point is leaving home and the decision is 'I'll drive to Suzy's house'. Even a novice driver's behavior is chunked; you don't think of each individual muscle contraction involved in turning the steering wheel. For most people, this is already learned and chunked. It's still voluntary.

Your pleasant thoughts were about "being able to speak Swahili" rather than "learning Sawahili". Your thoughts were about the joy of the reward, which I guess are not reinfornced in total independence from actions (imagine trying to learn Swahili without the rewarded thoughts, you'd probably not make it through th first few calsses), but are certainly not identical.

What would happen if you think about the effort of actually learning? Will it get negatively reinforced the same way as actually doing the effort?

after the computer program above calculates the amplitude (the same every time we run the program), can we incorporate in the program additional steps to simulate our magical measurement tool (the detector)?

Or to be mentioned and praised by people, therefore, it is also for himself

Isn't this like saying I wont pay for grocery because al the grocer wanted was to get paid?

Anyways, my counter argument will be "I have no choice in giving credit/blame too". Of course, the reply could well be "and I have no choice in debating the idea" etc - which, I confess, can lead to some wasted time

Since you presumably don't actually believe that you have no choice in giving credit or blame, you could answer by giving your consequentialist reasons for doing so (and by implication, what it would take to get you to stop doing so), but I assume you're trying to work out what would be socially effective rather than most accurate.

I think the distinction between decisions (as an end result) and other brain processes can be useful in fields of behavioral economics and the like on the short term, as it reahes results quite fast. But the complexity of decisions makes me visit the examples of unifications in physics. Perhaps if all decisions (not only final output) are treated as the same phenomena, aspects like framing can be understood as altering sub decisions by using their constant value functions, leading to a different decision later in time (which just happens to be the output d... (read more)

now the Guardian is mentioned on LW too, that could really start an infinite loop that takes over some of cyber space's space

[This comment is no longer endorsed by its author]Reply
ok, gone

Perhaps it is sometimes rational to prefer agreeing with your friends over being rational :-) ?

Without overfitting, the robot has the goal of shooting at what it sees blue. It achieves its goal. What I get from the article is that the human intelligence mis interprets the goal. Here I see the definition of a goal to equal what the program is written to do, hence it seems inevitable that the robot wll achieve its goal (if there is a bug in the code that misses shooting a blue object every 10 days, then this should be considered part of the goal as well, since we are forced to define the goal in hindsight, if we have to define one)

Do you reason similarly for humans?

Shouldn't the human intelligence part be considered part of the source code? With its own goals/value functions? Otherwise it will be just a human watchig the robot kind of thing.

Should we then draw different conclusions from their experiments?

I assume you mean if you only saw one of them (knowing the researcher's intentions ineither case)? In that case, I would say yes. For the first, the N is random, while for the second N is the smallest N were r>=60. In the second case, the question is: what is the probability that the cure rate will ever reach 60%, while the first case answers the Q: what is the cure rate probability accoding to a sample on N=100

Yes, I would say, draw very differenct conclusions since you ar answering very different questions!

When you described your possible experiment, you raised the probability that your theory is provable/disprovable, hence you raised the probability of it being rational.

Your desciption didn't raise the probability of your theory being correct, it raised the probability of it being a theory!

To me the biggest problem with Judaism, Christianity, and Islam is that they do not seem to provide one clear answer as to why God created the universe in the first place. Given this, I have no way of changing probabilities when the world seems cruel or contradicting, since they do not claim the world as perfect. This of course doesn't depend on what my prior is.

For evolution, I find a weakness (I am not an expert on the subject) that related to being able to explain all outcomes equally. If an animal feature seems in perfection with survival, this is due... (read more)

1Jim Balter2y
You're not merely not an expert ... you lack a fundamental understanding.
Agreed that if a theory T can explain both actual and counterfactual events equally well, then that's a huge weakness in T. (In fact, it makes T pretty much useless.) This is both true and important. That said, your understanding of what kinds of explanations "evolution" provides is deeply confused.