This is a special post for quick takes by Richard_Kennaway. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Richard_Kennaway's Shortform
48Richard_Kennaway
13Cedar
45Richard_Kennaway
21Richard_Kennaway
3andrew sauer
17Richard_Kennaway
7localdeity
3Haiku
3Haiku
2Richard_Kennaway
1Haiku
4Richard_Kennaway
1Haiku
2Roman Malov
1Sting
2Richard_Kennaway
-2Szeth
3Richard_Kennaway
17Richard_Kennaway
1Victor Novikov
15Richard_Kennaway
12Richard_Kennaway
4David Hornbein
4niplav
11Richard_Kennaway
11jbash
9Richard_Kennaway
2Richard_Kennaway
4cousin_it
2Said Achmiz
2Richard_Kennaway
1ProgramCrafter
8Richard_Kennaway
10Richard_Kennaway
10Richard_Kennaway
9Richard_Kennaway
11gwern
4Owain_Evans
5Dagon
8Richard_Kennaway
4winstonBosan
3Richard_Kennaway
12winstonBosan
1Richard_Kennaway
4Dagon
2cubefox
2Dagon
2cubefox
2Dagon
2cubefox
2Dagon
2Richard_Kennaway
2cubefox
2Richard_Kennaway
8Richard_Kennaway
2Dagon
7Richard_Kennaway
1Karl Krueger
2Richard_Kennaway
1CstineSublime
3Richard_Kennaway
0CstineSublime
2Richard_Kennaway
1npostavs
2Richard_Kennaway
7Richard_Kennaway
7Richard_Kennaway
4Richard_Kennaway
1Nate Showell
2Richard_Kennaway
2Nate Showell
7Richard_Kennaway
6Richard_Kennaway
3Viliam
5Richard_Kennaway
2Viliam
6Richard_Kennaway
6Richard_Kennaway
6Richard_Kennaway
6Richard_Kennaway
5Richard_Kennaway
5Richard_Kennaway
4Richard_Kennaway
4Richard_Kennaway
4Richard_Kennaway
4Richard_Kennaway
8Gunnar_Zarncke
2Gunnar_Zarncke
2Gunnar_Zarncke
2Richard_Kennaway
2Gunnar_Zarncke
4Richard_Kennaway
3Richard_Kennaway
2Dagon
3Richard_Kennaway
3Richard_Kennaway
2Richard_Kennaway
2Richard_Kennaway
2Richard_Kennaway
3Mitchell_Porter
2Richard_Kennaway
2Richard_Kennaway
2Viliam
2Richard_Kennaway
2Richard_Kennaway
4gwern
2Richard_Kennaway
2Richard_Kennaway
1Richard_Kennaway
5quetzal_rainbow
1Richard_Kennaway
1Richard_Kennaway
0Richard_Kennaway
1Karl Krueger
2Richard_Kennaway
1Karl Krueger
"Thought For The Day" is a daily three-minute religious-flavoured sermon on current events that goes out on BBC Radio 4 six days a week around 07:45, slotted into its daily 3-hour news programme "Today". (Perennially, people question why, yet it continues.)
Some years back there was a recurring speaker in the slot -- possibly Anne Atkins, I'm not sure -- who from time to time would take as her topic some recent discovery or pronouncement from scientists. She always had one of two reactions: to expostulate either "well of COURSE!" or "what NONSENSE!"
When I noticed the pattern, my thought for the day was that you can go through life meeting each new thing with one or other of those reactions, and so avoid the indignity of ever being seen to learn something.
A true story from a couple of days ago. Chocolates were being passed round, and I took one. It had a soft filling with a weird taste that I could not identify, not entirely pleasant. The person next to me had also taken one of the same type, and reading the wrapper, she identified it as apple flavoured. And so it was. It tasted much better when I knew what it was supposed to be.
On another occasion, I took what I thought was an apple from the fruit bowl, and bit into it. It was soft. Ewww! A soft apple is a rotten one. Then I realised that it was a nectarine. Delicious!
Today in Azathoth news:
"Eurasian hoopoes raise extra chicks so they can be eaten by their siblings"
It seems that the hoopoes lay extra eggs in times of abundance — more than they would be able to see through to fledging — as a way of storing up food for the older siblings. It is rather gruesomely called the "larder" hypothesis.
“What surprised me the most was the species practicing this aggressive parenting,” says Vladimir Pravosudov, an ecologist at the University of Nevada, Reno. Hoopoes primarily eat insects, he notes, so their long, curved bills aren’t ideal for killing and eating chicks. That might be why, Soler says, mother hoopoes often grab the unlucky chick and shove it into the mouth of an older chick, which swallows it whole.
Literal baby-eaters!
3
Never, ever take anybody seriously who argues as if Nature is some sort of moral guide.
Epistemic status: fiction. A variation on a parable of Peter Singer.
A man walked by a pond, where he saw that a little girl had fallen in, and would drown. But he was wearing a very fine suit and did not want to ruin it by wading into the pond to save her. He was not callous, for he was on his way to meet some very wealthy people, to solicit donations for the charity that he ran, and they would never take him seriously if he showed up late, muddy, and bedraggled. With these funds he would save far more lives than that of one little girl.
The decision was clear, he told himself, and
7
Rescue the girl and plan to explain to the wealthy people what happened. Possibly try to bring her with him, for purposes including lending credence to his story.
3
1. Second-order effects matter. Doing X is not only doing X. Becoming the type of person who does things in the reference class of X is one of the effects of your actions. Your reputation can likewise be affected. Your behavior can be copied by others, sometimes lossily. If it would be downright catastrophic for a chunk of society to do things vaguely shaped like X on a regular basis, you probably shouldn't do X.
2. If a utilitarian calculus concludes that you should do something that strongly violates societal norms of morality (a la virtue effects and deontology), your math is probably bad. Be humble. (You probably didn't invent new physics while doing your math homework, you probably didn't discover that "letting kids drown is sometimes good actually.")
3. Be suspicious of unintuitive moral conclusions that violate norms in your favor.
3
I thought another one: 4. Beware of simplifying assumptions and the Either/Or fallacy. You do not live in a moral philosophy dilemma. Reality has high dimensionality and the trade-offs are not always obvious and clean.
2
A general response to the responses to this fable.
The story ends where it ends. It is fiction, so there is no more to it than the story itself. It does not say what the man did. Anyone continuing it or adding further details, whether convenient or inconvenient, is writing their own fanfiction of the story to express their own response to the hypothetical.
There are two obvious continuations: "The decision was clear, he told himself, and walked on", and "The decision was clear, he told himself, and waded into the pond."[1] All further elaboration would be adding detail to make the story come out the way one would like.
For a lower stakes version of this story, consider the highly-paid lawyer who donates some time to working at a soup kitchen. This is a standard example in EA of Ineffective Altruism. The lawyer should instead (it is said) work those hours at lawyer rates and donate the money to do more good, or accept that he is working in the soup kitchen for the warm fuzzies, not to do all the good that he might. Is this different from the high-flying charity fund-raiser who can be more effective if he passes by the drowning girl? If he saves her, should he take care to be mindful that he is only doing it for the fuzzies, a concession to his imperfection, a necessary evil undertaken only to preserve his effectiveness?
Regarding types of response to hypotheticals:
* "Don't fight the hypothetical!"
* The Least Convenient Possible World.
* "Do fight the hypothetical!"
* "Question the hypothetical!"
* Steelmanning.
* "Against Devil's Advocacy."
Tagging those who commented: @Haiku, @Roman Malov, @localdeity, @Szeth, @Sting.
----------------------------------------
1. The latter continuation has something in common with Huckleberry Finn's decision whether to betray an escaped slave. ↩︎ ↩︎
1
I find myself somewhat disappointed that the scenario and the response don't seem to have much to say. I actually thought the original scenario was meant as a thinly-veiled scathing critique of naive utilitarianism. For any non-psychopath who is actually in the scenario, saving the drowning girl doesn't require any moral argumentation, and I wouldn't feel safe being in community with anyone who sincerely preferred the other ending.
I think when people first discover rationalism and effective altruism, they often descend into a valley of learning to "shut up and multiply" and casting their intuitions aside. It's very dangerous to remain in that valley without gaining humility about the limits of those tools. To be one of the good guys, you do have to rejoin human society and continue to love people in normal ways, and that isn't only about one's own personal warm fuzzies.
The neutrality of your follow-up reveals the original scenario as something like a scaffold upon which to hang one's perspective and observe other perspectives being hung. The original example leaves me with a lingering itch, though, because the alternate perspective is not something that anyone should be encouraged to entertain.
4
I gave no resolution of the problem raised by this fable, because I do not have one.
Besides the decision the man is faced with, the story raises a deeper problem about morality. As far as I have seen in the literature, philosophical or theological, no-one has a solution. I shall call it the Problem of Totalisation.
Peter Singer preaches totalising morality. “The Most Good You Can Do”, to quote the title of one of his books, is the good that you must do. That good (he argues) must weigh everyone everywhere equally. The little girl in danger of drowning before your eyes is worth exactly the same as every child everywhere in danger of death. "The Life You Can Save" (another of his books) is the life you must save. To fail to do all you can to save all the people you can is morally equivalent to passing by the drowning girl. (Bought a new car? How many dead children did it cost?) You should not value your own children any more highly than a starving child on the other side of the world that you will never see and whose fate you will never know. He who would be Good must think of nothing else, do nothing else.
This conflicts with almost everyone’s intuitions. It conflicts with yours. It conflicts with mine. But who will argue against altruism? Who will argue, as inspirationally as Singer, that duty is bounded, that I do not have to do all the good that I can, that I will not sacrifice everything to the egregore Tutulma, and not merely as a grudging concession to my frailty? That no-one can live up to the demand is, for Singer, no refutation. “Yes,” he says (imagine the Chad YES meme here), “it is demanding.” Failing to live up to the standard is just that, a failure.
So there is the problem. Meanwhile, I don’t let myself get hijacked by clever words just because I don’t have cleverer words to set against them.
1
The former does not imply the latter. That's my point here, is that despite their lives having equal worth, the situations are not equivalent. Likewise, the "large man" variation of the trolley problem is not an equivalent scenario to the original. If the situations were equivalent, they would have equivalent effects. But they do not have equivalent effects. The alternative choices are not felt the same way, and they are not perceived the same way. They do not affect the actors or onlookers the same way, therefore the effect is not the same. The effects are not equivalent, meaning the outcome is not equivalent, so the choices can't be equivalent.
For a taste of this: Wouldn't walking past the drowning girl harden your heart against all manner of clear and present suffering, and cause distant onlookers to condemn you, perhaps permanently ruining your reputation? Wouldn't pushing a man off a bridge land you in prison and give you nightmares for the rest of your life, or make you more willing to take a life in the future for lesser reasons?
Any argument that these scenarios should be equivalent is analogous to a physics student proclaiming that cows "ought to be" spherical. When we do physics, we strip out a tremendous amount of detail to simplify our calculations. We turn it into math. If our assumptions are correct, the loss of detail won't significantly change the final result, and we can turn the math back into a prediction about physics.
From the standpoint of consequentialism, any moral philosophy that creates these equivalencies is incorrect, in an objective, predictive sense. Humans are not spherical cows in a moral vacuum. When a scenario is presented that adds the atmosphere back in, we do not get to continue with our simplified assumptions without consequence. Human psychology is a very important part of human morality, and ignoring its role doesn't just produce counter-intuitive results, it produces bad predictions, which if acted upon can have devastati
2
he called rescuers, thereby advancing the Pareto frontier and avoiding pesky utilitarian calculations (which are mostly incomputable for humans anyway).
1
Strong upvoted. Is it missing the ending, or
2
The story appears exactly as I intended it to.
-2
3
I'll see if any discussion develops before giving the Word of God on the subject.
1
Sufficient optimization pressure destroys all that is not aligned to the metric it is optimizing. Value is fragile.
Roko's Basilisk, as told in the oldest extant collection of jokes, the Philogelos, c. 4th century AD.
A pedant having fallen into a pit called out continually to summon help. When no one answered, he said to himself, "I am a fool if I do not give all a beating when I get out in order that in the future they shall answer me and furnish me with a ladder."
H/T Dynomight.
It seems to me I've not heard much of cryonics in the last few years. I see from Wikipedia that as of last year Alcor only has about 2000 signed up, of which 222 are suspended. Are people still signing up for suspension, as much as they ever have been? Are the near-term prospects of AGI making long-term prospects like suspension less attractive?
4
>Are the near-term prospects of AGI making long-term prospects like suspension less attractive?
No. Everyone I know who was signed up for cryonics in 2014 is still signed up now. You're hearing about it less because Yudkowsky is now doing other things with his time instead of promoting cryonics, and those discussions around here were a direct result of his efforts to constantly explain and remind people.
4
I don't know about signup numbers in general (the last comprehensive analysis was in 2020, when there was a clear trend)—but it definitely looks like people are still signing up for Alcor membership (six from January to March 2023).
However, in recent history, two cryonics organizations have been founded, Tomorrow.bio in Europe in 2019 (350 members signed up) and Southern Cryonics in Australia. People are being preserved, and Southern Cryonics recently suspended their first member.
I do not have answers to the question I raise here.
- Historical anecdotes.
Back in the stone age — I think something like the 1960's or 1970's, I read an article about the possible future of computing. Computers back then cost millions and lived in giant air-conditioned rooms, and memory was measured in megabytes. Single figures of megabytes. Someone had expressed to its writer the then-visionary idea of using computers to automate a company. They foresaw that when, for example, a factory was running low on some of its raw materials, the computer would automatically know that, and would make out a list of what was needed. A secretary would type that up into an order to post to a supplier, and a secretary there would input that into their computer, which would send the goods out. The writer's response was "what do you need all those secretaries for?"
Back in the bronze age, when spam was a recent invention (the mid-90's), there was one example I saw that was a reductio ad absurdum of fraudulent business proposals. I wish I'd kept it, because it was so perfect of its type. It offered the mark a supposed business where they would accept orders for goods, which the business staff that ...
The vision is of everything desirable happening effortlessly and everything undesirable going away.
Citation needed. Particularly for that first part.
Hack your brain to make eating healthily effortless. Hack your body to make exercise effortless.
You're thinking pretty small there, if you're in a position to hack your body that way.
If you're a software developer, just talk to the computer to give it a general idea of what you want and it will develop the software for you, and even add features you never knew you wanted. But then, what was your role in the process? Who needed you?
Why would I want to even be involved in creating software that somebody else wanted? Let them ask the computer themselves, if they need to ask. Why would I want to be in a world where I had to make or listen to a PowerPoint presentation of all things? Or a summary either?
Why do I care who needs me to do any of that?
Why climb Kilimanjaro if a robot can carry you up?
Because if the robot carries me, I haven't climbed it. It's not like the value comes from just being on the top.
Helicopters can fly that high right now, but people still walk to get there.
...Why paint, if Midjourney will do it better t
9
Yet these are actual ideas someone suggested in a recent comment. In fact, that was what inspired this rant, but it grew beyond what would be appropriate to dump on the individual.
Perhaps the voice I wrote that in was unclear, but I no more desire the things I wrote of than you do. Yet that is what I see people wishing for, time and again, right up to wanting actual wireheading.
Scott Alexander wrote a cautionary tale of a device that someone would wear in their ear, that would always tell them the best thing for them to do, and was always right. The first thing it tells them is "don't listen to me", but (spoiler) if they do, it doesn't end well for them.
2
There are authors I would like to read, if only they hadn't written so much! Whole fandoms that I must pass by, activities I would like to be proficient at but will never start on, because the years are short and remain so, however far an active life is prolonged.
4
I think something like the Culture, with aligned superintelligent "ships" keeping humans as basically pets, wouldn't be too bad. The ships would try to have thriving human societies, but that doesn't mean granting all wishes - you don't grant all wishes of your cat after all. Also it would be nice if there was an option to increase intelligence, conditioned on increasing alignment at the same time, so you'd be able to move up the spectrum from human to ship.
2
See also Stanislaw Lem on this subject:
2
See also.
1
Upvoted as a good re-explanation of CEV complexity in simpler terms! (I believe LW will benefit from recalling the long understood things so that it has a chance on predicting future in greater detail.)
In essence, you prove the claim "Coherent Extrapolated Volition would not literally include everything desirable happening effortlessly and everything undesirable going away". Would I be wrong to guess it argues against position in https://www.lesswrong.com/posts/AfAp8mEAbuavuHZMc/for-the-sake-of-pleasure-alone?
That said, current wishes of many people include things they want being done faster and easier; it's just the more you extrapolate the less fraction wants that level of automation - just more divergence as you consider higher scale.
8
I suppose it does. That article was not in my mind at the time, but, well, let's just say that I am not a total hedonistic utilitarian, or a utilitarian of any other stripe. "Pleasure" is not among my goals, and the poster's vision of a universe of hedonium is to me one type of dead universe.
This is a story from the long-long-ago, from the Golden Age of Usenet.
On the science fiction newsgroups, there was someone—this is so long ago that I forget his name—who had an encyclopedic knowledge of fannish history, and especially con history, backed up by an extensive documentary archive. Now and then he would have occasion to correct someone on a point of fact, for example, pointing out that no, events at SuchAndSuchCon couldn't have influenced the committee of SoAndSoCon, because SoAndSoCon actually happened several years before.
The greater the irrefutability of the correction, the greater people's fury at being corrected. He would be scornfully accused of being well-informed.
"ChatGPT is Bullshit"
Thus the title of a recent paper. It appeared three weeks ago, but I haven't seen it mentioned on LW yet.
The abstract: "Recently, there has been considerable interest in large language models: machine learning systems which produce human-like text and dialogue. Applications of these systems have been plagued by persistent inaccuracies in their output; these are often called “AI hallucinations”. We argue that these falsehoods, and the overall activity of large language models, is better understood as bullshit in the sense explored by Fr...
I'd say that they are wrong when they say a LLM may engage in 'soft bullshit': a LLM is simulating agents, who are definitely trying to track truth and the external world, because the truth is that which doesn't go away, and so it may say false things, but it still cares very much about falsity because it needs to know that for versimilitude. If you simply say true or false things at random, you get ensnared in your own web and incur prediction error. Any given LLM may be good or bad at doing so - the success of story-based jailbreaks suggests they are still far from ideal - but it's clear that the prediction loss on large real-world texts written by agents like you or I, who are writing things to persuade each other, like I am writing this comment to manipulate you and everyone reading it, require tracking latents corresponding to truth, beliefs, errors, etc. You can no more accurately predict the text of this comment without tracking what I believe and what is true than you could accurately predict it while not tracking whether I am writing in English or French. (Like in that sentence right there. You see what I did there? Maybe you didn't because you're just skimming and tldr, b...
4
The "Still no lie detector for language model" paper is here: https://arxiv.org/pdf/2307.00175
The paper in the OP seems somewhat relate to my post from earlier this year.
5
I think that's true, but not very important (in the short term). On Bullshit - Wikipedia was first published in 1986, and was a humorous, but useful, categorizaton of a whole lot of human communication output. ChatGPT is truth-agnostic (except for fine-tuning and output tuning), but still pretty good on a whole lot of general topics. Human choice of what GPT outputs to highlight or use in further communication can be bullshit or truth-seeking, depending on the human intent.
In the long-term, of course, the idea is absolutely core to all the alignment fears and to the expectation that AI will steamroller human civilization because it doesn't care.
If, on making a decision, your next thought is “Was that the right decision?” then you did not make a decision.
If, on making a decision, your next thought is to suppress the thought “Was that the right decision?” then you still did not make a decision.
If you are swayed by someone else asking “Was that the right decision?” then you did not make a decision.
If you are swayed by someone repeating arguments you already heard from them, you did not make a decision.
Not making that decision may be the right thing to do. Wavering suggests that you still have some d...
1
14
I'm very confused. Because it seems like for you decision should not only clarify matters and narrow possibilities, but also eliminate all doubt entirely and prune off all possible worlds where the counterfactual can even be contemplated.
Perhaps that's indeed how you define the word. But using such a stringent definition, I'd have to say I've never decided anything in my life. This doesn't seem like the most useful way to understand "decision" - it diverges enough from common usage and mismatches with the hyperdimensional-cloud of word meaning for decision sufficiently to be useless in conversation with most people.
3
A decision is not a belief. You can make a decision and still be uncertain about the outcome. You can make a decision while still being uncertain about whether it is the right decision. Decision neither requires certainty nor produces certainty. It produces action. When the decision is made, consideration ends. The action must be wholehearted in spite of uncertainty. You can steer according to how events unfold, but you can't carry one third of an umbrella when the forecast is a one third chance of rain.
In about a month's time, I will take a flight from A to B, and then a train from B to C. The flight is booked already, and I have just have booked a ticket for a specific train that will only be valid on that train. Will I catch that train? Not if my flight is delayed too much. But I have considered the possibilities, chosen the train to aim for, and bought the ticket. There are no second thoughts, no dwelling on "but suppose" and "what if". Events on the day, and not before, will decide whether my hopes[1] for the journey will be realised. And if I miss the train, I already know what I will do about that.
----------------------------------------
1. hope: (1) Desire for an outcome which one has only limited power to steer events towards. (2) A good breakfast, but a poor supper. ↩︎
If “you can make a decision while still being uncertain about whether it is the right decision”. Then why can’t you think about “was that the right decision”? (Lit. Quote above vs original wording)
It seems like what you want to say is - be doubtful or not, but follow through with full vigour regardless. If that is the case, I find it to be reasonable. Just that the words you use are somewhat irreconcilable.
1
Because it is wasted motion. Only when new and relevant information comes to light does any further consideration accomplish useful work.
One day I might write an article on rationality in the art of change ringing, a recreation I took up a few years ago. Besides the formidable technicalities of the activity, it teaches such lessons as letting the past go, carrying on in the face of uncertainty, and acting (by which I mean doing, not seeming) assuredly however unsure you are. I have also heard (purely anecdotally) that change ringers seem to never get Alzheimers.
4
This seems like hyperbolic exhortation rather than simple description. This is not how many decisions feel to me - many decisions are exactly a belief (complete with bayesean uncertainty). A belief in future action, to be sure, but it's distinct in time from the action itself.
I do agree with this as advice, in fact - many decisions one faces should be treated as a commitment rather than an ongoing reconsideration. It's not actually true in most cases, and the ability to change one's plan when circumstances or knowledge changes is sometimes quite valuable. Knowing when to commit and when to be flexible is left as an excercise...
2
But if you only have a belief that you will do something in the future, you still have to decide, when the time comes, whether to carry out the action or not. So your previous belief doesn't seem to be an actual decision, but rather just a belief about a future decision -- about which action you will pick in the future.
See Spohn's example about believing ("deciding") you won't wear shorts next winter:
2
Correct. There are different levels of abstraction of predictions and intent, and observation/memory of past actions which all get labeled "decision". I decide to attend a play in London next month. This is an intent and a belief. It's not guaranteed. I buy tickets for the train and for the show. The sub-decisions to click "buy" on the websites are in the past, and therefore committed. The overall decision has more evidence, and gets more confident. The cancelation window passes. Again, a bit more evidence. I board the train - that sub-decision is in the past, so is committed, but there's STILL some chance I won't see the play.
Anything you call a "decision" that hasn't actually already happened is really a prediction or an intent. Even DURING an action, you only have intent and prediction. While the impulse is traveling down my arm to click the mouse, the power could still go out and I don't buy the ticket. There is past, which is pretty immutable, and future, which cannot be known precisely.
I think this is compatible with Spohn's example (at least the part you pasted), and contradicts OP's claim that "you did not make a decision" for all the cases where the future is uncertain. ALL decisions are actually predictions, until they are in the past tense. One can argue whether that's a p(1) prediction or a different thing entirely, but that doesn't matter to this point.
"If, on making a decision, your next thought is “Was that the right decision?” then you did not make a decision." is actually good directional advice in many cases, but it's factually simply incorrect.
2
That's an interesting perspective. Only it doesn't seem fit into the simplified but neat picture of decision theory. There everything is sharply divided between being either a statement we can make true at will (an action we can currently decide to perform) and to which we therefore do not need to assign any probability (have a belief about it happening), or an outcome, which we can't make true directly, that is at most a consequence of our action. We can assign probabilities to outcomes, conditional on our available actions, and a value, which lets us compute the "expected" value of each action currently available to us. A decision is then simply picking the currently available action with the highest computed value.
Though as you say, such a discretization for the sake of mathematical modelling does fit poorly with the continuity of time.
2
Decision theory is fine, as long as we don't think it applies to most things we colloquially call "decisions". In terms of instantaneous discrete choose-an-action-and-complete-it-before-the-next-processing-cycle, it's quite a reasonable topic of study.
2
A more ambitious task would be to come up with a model that is more sophisticated than decision theory, one which tries to formalize your previous comment about intent and prediction/belief.
2
I think it's a different level of abstraction. Decision theory works just fine if you separate the action of predicting a future action from the action itself. Whether your prior-prediction influences your action when the time comes will vary by decision theory.
I think, for most problems we use to compare decision theories, it doesn't matter much whether considering, planning, preparing, replanning, and acting are correlated time-separated decisions or whether it all collapses into a sum of "how to act at point-in-time". I haven't seen much detailed exploration of decision theory X embedded agents or capacity/memory-limited ongoing decisions, but it would be interesting and important, I think.
2
It is exhortation, certainly. It does not seem hyperbolic to me. It is making the same point that is illustrated by the multi-armed bandit problem: once you have determined which lever gives the maximum expected payout, the optimum strategy is to always pull that lever, and not to pull levers in proportion to how much they pay. Dithering never helps.
Yes. But only as such changes come to be. Certainly not immediately on making the decision. "Commitment" is not quite the concept I'm getting at here. It's just that if I decided yesterday to do something today, then if nothing has changed I do that thing today. I don't redo the calculation, because I already know how it came out.
2
Yes, but that arguably means we only make decisions about which things to do now. Because we can't force our future selves to follow through, to inexorably carry out something. See here:
2
My left hand cannot force my right hand to do anything either. Instead, they work harmoniously together. Likewise my present, past, and future. Not only is the sage one with causation, he is one with himself.
That is an example of dysfunctional decision-making. It is possible to do better.
I always do the dishes today.
2
The open question is whether this includes the universe itself.
1
Ordinary lying is a pretty big one too.
2
The whole practice of dark arts is the search for and exploitation of security vulnerabilities.
1
What use of the term hypnosis do you mean? The practice associated with figures like Émile Coué that is represented in cartoons by magicians with spiral disks, or the idea of a state of focused attention?
3
Well now, that's a dark arts question right there. A straw man in one hand and an attempt to lead the witness in the other, like a stage magician leading someone to "choose" the card the magician has already chosen.
No. I select a card that is not in either of your hands.
I am talking about hypnosis as it has been presented in this discussion forum, most recently by @jimmy.
0
And what sense did Jimmy present it as? I am unfamiliar with it.
2
I did give you a link to his profile. One of his recent postings even has “hypnosis” in the title.
As I observe it (but he is the one to speak—or be read—in detail) the act involves such things as a confident manner, establishing a frame for the conversation and breaking any conflicting frame coming from the subject, internally staying always at the meta-level and regarding the subject as a thing to be operated, not an equal in a conversation. There are any number of detailed technical tricks that may be employed, many of them coming from NLP. The purpose is to get the subject to do whatever it is that one wants them to do, that they may not be able or inclined to do just on someone’s say-so.
I have described this in a way that says nothing about using this art for good or for evil. (On the evil side, see the Zizians and Leverage, of which there are detailed accounts on LW.)
They say in computer security, “a trusted system is one that has the ability to break your security.” One trusts that the system will only use its elevated access to do the things you gave it that access for.
Beware who you give root access to your self.
1
https://www.lesswrong.com/posts/StoioB9Tv9uyMbkLW/the-spectrum-of-attention-from-empathy-to-hypnosis says
This sounds like it corresponds to "the idea of a state of focused attention", so I don't understand why you rejected it. Just because he talks about it as a spectrum (vs a state)? Or something else?
2
"Focussed attention" is far more wide-ranging than what people talk about, when they talk about hypnosis, including the article that you linked, and even the quote you excerpted from it.
Right now my attention is focussed on composing this comment. Is that "hypnosis"? A couple of days ago my attention was focussed for several hours on a long bicycle ride. (Or to put that another way, I did a long bicycle ride.) Is that "hypnosis"? This evening, my attention will be focussed, for a time, on bell-ringing. Are any of these things "hypnosis"?
I say, no, they are not. The only reason to describe hypnosis as just "focussed attention" is to conceal everything else that is going on.
To know how the magician does the trick, watch his other hand.
Today, Academia.edu (with which I have a free account) offered me an AI-generated podcast about one of my papers. The voice was very lifelike: it took about a minute before its robotic nature became clear. But the text! It was ridiculously over the top for a very minor paper more than 35 years old constructing an algorithm of no practical importance, that I doubt anyone has looked at much after its original publication. Truly, the AI has me beat hands down at turning a molehill of content into a mountain of puffery.
I declined to let Academia.edu display it on my profile.
A long time ago, the use of calculators in schools was frowned upon, or even forbidden. Eventually they became permitted, and then required. How long will it be before AI assistants, currently frowned upon or even forbidden, become permitted, and then required?
The recent Gemini incident, apparently a debacle, was also a demonstration of how easy it is to deliberately mould an AI to force its output to hew to a required line, independent of the corpus on which it was trained, and the reality which gave rise to that corpus. Such moulding could be used by an ...
4
Here's an AI disaster story that came to me on thinking about the above.
1. Schools start requiring students to use the education system's official AI to criticise their own essays, and rewrite them until the AI finds them acceptable. This also removes the labour of marking from teachers.
2. All official teaching materials would be generated by a similar process. At about the same time, the teaching profession as we know it today ceases to exist. "Teachers" become merely administrators of the teaching system. No original documents from before AI are permitted for children to access in school.
3. Or out of school.
4. Social media platforms are required to silently limit distribution of materials that the AI scores poorly.
5. AIs used in such important capacities would have to agree with each other, or public confusion might result. There would therefore have to be a single AI, a central institution for managing the whole state. For public safety, no other AIs of similar capabilities would be permitted.
6. Prompt engineering becomes a criminal offence, akin to computer hacking.
7. Access to public archives of old books, online or offline, is limited to adults able to demonstrate to the AI's satisfaction that they have an approved reason for consulting them, and will not be corrupted by the wrong thoughts they contain. Physical books are discouraged in favour of online AI rewrites. New books must be vetted by AI. Freedom of speech is the freedom to speak truth. Truth is what it is good to think. Good is what the AI approves. The AI approves it because it is good.
8. Social credit scores are instituted, based on AI assessment of all of an individual's available speech and behaviour. Social media platforms are required to silently limit distribution of anything written by a low scorer (including one to one messaging).
9. Changes in the official standard of proper thought, speech, and action would occur from time to time in accordance with socia
1
This sequence of steps looks implausible to me. Teachers would have a vested interest in preventing it, since their jobs would be on the line. A requirement for all teaching materials to be AI-generated would also be trivially easy to circumvent, either by teachers or by the students themselves. Any administrator who tried to do these things would simply have their orders ignored, and the Streisand Effect would lead to a surge of interest in pre-AI documents among both teachers and students.
2
That will only put a brake on how fast the frog is boiled. Artists have a vested interest against the use of AI art, but today, hardly anyone else thinks twice about putting Midjourney images all through their postings, including on LessWrong. I'll be interested to see how that plays out in the commercial art industry.
2
You're underestimating how hard it is to fire people from government jobs, especially when those jobs are unionized. And even if there are strong economic incentives to replace teachers with AI, that still doesn't address the ease of circumvention. There's no surer way to make teenagers interested in a topic than to tell them that learning about it is forbidden.
There's an article in the current New Scientist, taking the form of a review of a new TV series called "The Peripheral", which is based on William Gibson's novel of the same name. The URL may not be useful, as it's paywalled, and if you can read it you're probably a subscriber and already have.
The article touches on MIRI and Leverage. A relevant part:
...But over the past couple of years, singulatarian [sic] organisations [i.e. the Singularity Institute/MIRI and Leverage/Leverage Research] have been losing their mindshare. A number of former staffers at Leve
Why has this heavily downvoted posting suddenly surfaced as posted today, when according to the dates of the comments, it is two years old?
3
The date of the post is today, so maybe it was edited today?
5
Does editing an old post change its date to the present? Editing an old comment doesn't, it just adds a * to the timestamp, and adds a "Last edited" time to the timestamp's mouseover text.
2
I don't know for sure actually, that was just my best guess.
The blind have seeing-eye dogs. Terry Pratchett gave Foul Ole Ron a thinking-brain dog. At last, a serious use-case for LLMs! Thinking-brain dogs for the hard of thinking!
2The safer it is made, the faster it will be developed, until the desired level of danger has been restored.
The physicality of decision.
A month ago I went out for a 100 mile bicycle ride. I'm no stranger to riding that distance, having participated in organised rides of anywhere from 50 to 150 miles for more than twelve years, but this was the first time I attempted that distance without the support of an organised event. Events provide both the psychological support of hundreds, sometimes thousands, of other cyclists riding the same route, and the practical support of rest stops with water and snacks.
I designed the route so that after 60 miles, I would be just ...
1The game of Elephant begins when someone drags an elephant into the room.
Epistemic status: a jeu d'esprit confabulated upon a tweet I once saw. Long enough ago that I don't think I lose that game of Elephant by posting this now.
Everyone knows there's an elephant there. It's so obvious! We all know the elephant's there, we all know that we all know, and so on. It's common knowledge to everyone here, even though no-one's said anything about it. It's too obvious to need saying.
But maybe there are some people who are so oblivious that they don't realise it's t...
I have a dragon in my garage. I mentioned it to my friend Jim, and of course he was sceptical. "Let's see this dragon!" he said. So I had him come round, and knocked on the garage door. The door opened and the dragon stepped out right there in front of us.
"That can't really be a dragon!" he says. It's a well-trained dragon, so I had it walk about and spread its wings, showing off its iridescent scaly hide.
"Yes, it looks like a dragon," he goes on, "but it can't really be a dragon. Dragons belch fire!"
The dragon raised an eyebrow, and discreetly belched som...
I read this.
Then I had this in my email from Academia.edu:
Dear Dr. Kennaway,
Based on the papers you’ve autocfp, we think you might be interested in this recently published article from
"autocfp". Right. There is not the slightest chance I will be interested in whatever follows.
I have at last had a use for ChatGPT (that was not about ChatGPT).
I was looking (as one does) at Aberdare St Elvan Place Triples. (It's a bellringing thing, I won't try to explain it.) I understood everything in that diagram except for the annotations "-M", "-I", etc., but those don't lend themselves to Google searching.
So I asked ChatGPT as if I was asking a bell ringer:
...When bellringing methods are drawn on a page, there are often annotations near each lead end consisting of a hyphen and a single letter, such as "-M", "-I", etc. What do these annotation
8
For what it's worth, I use ChatGPT all the time, multiple times every day, even most hours. I usually don't hit the request limits, but have three times now. Once with a sequence of Dall-E, I guess because evaluating an image is faster than evaluating a long text and because the generated pictures require a lot of tuning (so far).
I use it
* as a Google replacement.
* to figure out key words for Google queries like you did and to find original sources.
* to summarize websites. Though for "generally known" stuff "no browse" is often better and faster.
* as a generator for code fragments.
* for debugging - just paste code and error message into it.
* for understanding topics deeper in an interactive discussion style.
* as a language coach. "Explain the difference between zaidi and kuliko in Swahili with examples."
* for improving texts of all kinds - I often ask: "criticise this text I wrote".
and more I guess. I also let my kids have dialogs with it.
2
...and I use a custom GPT that a colleague created to write nice Scrum tickets based on technical information provided. Just paste in what you have, such a mail about a needed service update, a error log entry, or a terse protocol from a meeting and PO GPT will create a ticket readable for non-technies out of it.
https://chat.openai.com/g/g-FmWRcwG0i-po-gpt
2
...and I use ChatGPT to
* generate illustrations for posts and inspirational images
* read text from screenshots
2
MacOS has for a while had the capability to copy text out of images. Is ChatGPT just invoking a text-recognition library?
2
Yeah, there are also tools for Windows that can do that. But ChatGPT can format nicely, convert to CVS, bullet points etc.
No, it can do a lot of things with images, describe what is in there, style, etc. I think it is another image model.
[ETA: alfredmacdonald's post referred to here has been deleted.]
Well, well, alfredmacdonald has banned me from his posts, which of course he has every right to do. Just for the record, I'll paste the brief thread that led to this here.
Richard_Kennaway (in reply to this comment)
I also notice that alfredmacdonald last posted or commented here 10 years ago, and the content of the current post is a sharp break from his earlier (and brief) participation. What brings you back, Alfred? (If the answer is in the video, I won't see it. The table of contents was enou...
When I watch a subtitled film, it is not long before I no longer notice that I am reading subtitles, and when I recall scenes from it afterwards, the actors’ voices in my head are speaking the words that I read.
2
Me too! it's a very specific form of synesthesia. For languages I know a little bit, but not well enough to do without subtitles, it can trick me into thinking I'm far more good at understanding native speakers than I am at all.
I can't wait until LLMs are good, fast, and cheap enough, and AR or related video technology exists, such that I can get automatic subtitles for real-life conversations, in English as well as other languages.
Epistemic status: crafted primarily for rhetorical parallelism.
All theories are right, but some are useless.
[Wittgenstein] once greeted me with the question: 'Why do people say that it was natural to think that the sun went round the earth rather than that the earth turned on its axis?' I replied: 'I suppose, because it looked as if the sun went round the earth.' 'Well,' he asked, 'what would it have looked like if it had looked as if the earth turned on its axis ?" (Source)
Something on the lighter side: the Mastermind box design.
Think less “family night,” and more “I will find the hidden structure of your choices, and then I will destroy you.”
Here's something that may be of interest here. It's more EA than rationality, but I'm not on the EA forum. The Reith Lectures 2025, given by Rutger Bregman. Currently being broadcast on BBC Radio 4 and the BBC World Service, and available on BBC Sounds.
"Titled Moral Revolution, the lectures will delve into the current 'age of immorality', explore a growing trend for unseriousness among elites, and ask how we can follow history’s example and assemble small, committed groups to spark positive change."
What is valence, and is there such a thing?
I'm currently studying a paper that talks about the concept of "valence", but I have not been able to discern the reasons for thinking that there is such a thing. I recalled that there was a sequence on Valence on LessWrong a few years ago. I passed it by at the time, but on reading it now, I have not found in it a reason to think that the thing is a thing. Nor am I satisfied by the references cited in the paper I am studying, which include these: 1, 2, 3, 4, 5. (Valence is not the subject of the paper, just somet...
3
How deep is your skepticism? In the context of consciousness, valence basically means the qualia of value. Are we denying a particular theory of valence, or proposing that valence is a wrong way to think about the phenomenology of value, or denying that there is any phenomenology of value at all?
2
At the moment, "valence" seems to me, in the material I've read, no more than a name reifying a noun phrase of the form "the thing that..." It is like saying that salt tastes salty due to its saltiness, or to use a well-known example, that opium produces sleep through its dormitive principle.
Uttering a noun phrase does not conjure into existence a thing that it refers to. It can conjure an idea of such a thing into your head, and then you can look at various phenomena and "see" it there, just like you can "see" the phlogiston coming out of a burning log, or "see" the demon possessing an epileptic. That is what the whole Valence sequence looks like to me, and the valence literature that I cited.
My scepticism then extends to the word "value", as applied in this context. Also, I had not got the impression from the material that valences were defined to be conscious experiences, but rather, that they pervade all decision-making in the brain, that comparison of valence is "the thing that" makes decisions. For example, the fourth reference I cited is a purely speculative article asserting the existence of "micro-valences" in low-importance rapid choices like which coffee mug to select from a cupboard. But perhaps the authors would say that these micro-valences are micro-consciously perceived.
Today, for the first time ever, I used an LLM. That is, for something other than answering the question, "I wonder what an LLM will do with this?" It was for some questions on the theory of elasticity of orthotropic composite materials, which I'll skip, but its answers passed the sanity checks I was able to make.
12
I think the trick is asking the kinds of questions where it would be a lot of work to figure it out for yourself, but relatively easy to verify the received answer.
Interesting application of a blockchain. What catches my attention is this (my emphasis):
The deepest thinkers about Dark Forest seem to agree that while its use of cryptography is genuinely innovative, an even more compelling proof of concept in the game is its “autonomous” game world—an online environment that no one controls, and which cannot be taken down.
So much for "we can always turn the AI off." This thing is designed to be impossible to turn off.
"Parasite gives wolves what it takes to be pack leaders", Nature, 24 November 2022.
Toxoplasma gondii, the parasite well-known for making rodents lose their fear of cats, and possibly making humans more reckless, also affects wolves in an interesting way.
"infected wolves were 11 times more likely than uninfected ones to leave their birth family to start a new pack, and 46 times more likely to become pack leaders — often the only wolves in the pack that breed."
4
The gesturing towards the infected wolves being more reproductively fit in general is probably wrong, however. Of course wolves can be more aggressive if it's actually a good idea, there's no need for a parasite to force them to be more aggressive; the suggestion about American lions going extinct is absurd - 11,000 years is more than enough time for wolves to recalibrate such a very heritable trait if it's so fitness-linked! So the question there is merely what is going on? Some sort of bias or very localized fitness benefit?
Is there a selection bias whereas ex ante going for pack leader is a terrible idea, but ex post conditional on victory (rather than death/expulsion) it looks good? Well, this claims to be longitudinal and not find the sorts of correlations you'd expect from a survivorship. What else?
Looking it over, the sampling frame 1995-2020 itself is suspect: starting in 1995. Why did it start then? Well, that's when the wolves came back (very briefly mentioned in the article). The wolf population expanded rapidly 5-fold, and continues to oscillate a lot as packs rise and fold (ranging 8-14) and because of overall mortality/randomness on a small base (a pack is only like 10-20 wolves of all ages, so you can see why there would be a lot of volatility and problems with hard constraints like lower bounds):
So, we have at least two good possible explanations there: (a) it was genuinely reproductively-fit to take more risks than the basal wolf, but only because they were expanding into a completely-wolf-empty park and surrounding environs, and the pack-leader GLM they use doesn't include any variables for time period, so on reanalysis, we would find that the leader-effect has been fading out since 1995; and (b) this effect still exists, and risk-seeking individuals do form new packs and are more fit... but only temporarily because they occupied a low-quality pack niche and it goes extinct or does badly enough that they would've done better to stay in the or
What is happiness?
This is an extract from an interview with the guitarist Nilo Nuñez, broadcast yesterday on the BBC World Service. Nuñez was born and brought up in Cuba, and formed a rock band, but he and his group came more and more into conflict with the authorities. He finally decided that he had to leave. When the group received an invitation to tour in the Canary Islands, and the Cuban authorities gave them permission to go, they decided to take the opportunity to leave Cuba and not return. They only had temporary visas, so they stayed on in the Cana...
5
I need a bot for writing assistance because writing from scratch for me is very tiring, while editing is not.
From New Scientist, 14 Nov 2022, on a 50% fall in honeybee life expectancy since the 1970s:
“For the most part, honeybees are livestock, so beekeepers and breeders often selectively breed from colonies with desirable traits like disease resistance,” says Nearman.
...“In this case, it may be possible that selecting for the outcome of disease resistance was an inadvertent selection for reduced lifespan among individual bees,” he says. “Shorter-lived bees would reduce the probability of spreading disease, so colonies with shorter lived bees would appear healt
Yesterday I received this email:
————
Driver & Vehicle Licensing Agency
Dear Vehicle Owner,
Our system has identified that your vehicle tax will be due on Saturday, 06 December, 2025.
To ensure that your vehicle remains legally paid and complies with applicable traffic regulations, you are required to complete the renewal process before the due date
[Update Now]
Thank you for keeping your vehicle tax up to date and for helping us maintain safe and legal roads across the UK.
————
I own a car, and the DVLA and vehicle tax are real things, so my immediate reacti...
1
Another contributing factor might be a person's level of anxiety about the DVLA, or government, or email, or "the system". When people are anxious about what "the system" might do to them, and prepared for it to make novel demands upon them, that primes them to be scammable.
A different example: If you want to phish an office-worker, one way to do it is to pretend to be their boss and make sudden urgent demands of them. If the office-worker fears that they will be fired if they don't comply with novel demands from their boss, then they are primed to be scammable. Workers who feel unsafe questioning "their boss's" orders will be more scammable than workers who feel safe calling bullshit on their actual boss once in a while.
2
The office worker has gone wrong already by panicking and failing to ask, "Is this actually from the boss?"
Always be asking "What am I looking at?"
1
Sure, but the boss can go wrong by creating an incentive structure in which questioning a message "from the boss" is unsafe.
Successful anti-phishing campaigns instill not only doubt ("Is this actually from the boss?") but also permission to act on that doubt ("I've got the boss's cell phone number already; when I'm not sure if the message is from the boss, I'm supposed to call the boss and check, with no chance of bad consequences for pestering her.")
Curated and popular this week