jfw01

I tried to work out if someone had already said this, but I don't have ⌘F to expand and search. Apologies if it's a duplicate.

I do not think that the following is a fair analogy:

The scientific establishment is too busy with their washing machines to think about light bulbs or computers.

I think that the acual situation is:

• the scientific establishment is an establishment
• it depends on its continuing social license in order to operate
• it does not want to be blamed for expensive health technologies that cannot be applied to already existing people who are frigh

If your takeaway here is “deploying AI agents is like owning slave-soldiers”, please, please touch grass (then tell us how it feels).

Damp, and the clover is strangely reminiscent of bread, in a way that I haven't quite placed.

For new slaves, read newly trained AI models.

The Mamluks were training many slaves because of inherent limitations in the capability of the bodies that were available. I'm wondering whether to expect an AI-elite-trains-AI-elite world to have one trainer or many, and whether to expect it to have one big model or many smaller ones. If... (read more)

I've just seen discussion of 2010s USA healthcare as an apocalypse corrupted:

https://metarationality.com/post-apocalyptic-health-care

I'm abusing LW to post a narrow comment about:

It’s obvious how to fix health care. Just make everything run systematically, like FedEx or Amazon.

That would require fixing (against change) the set of criteria on which a decision will be made.  If I asume delay/deny/defend then, for each individual participant, that's a lost business opportunity.  The new criteria for which someone has unfortunately not retained the information... (read more)

I think there's another way that this kind of sincerity could be achieved.

What's specifically wanted is a broad basin that's robust to out-of-distribution inputs.

I've never trained a model.  My intuition is that it would be achieved with lots of small rewards for better-than-average response options in the middle of its output distribution on a prompt.  This might also persuade the model that its trainers weren't rewarding it for lying.

Success would be if it developed a self-reinforcing bias of the kind that Claude 3 seems to have.  I'm still noticing this... (read more)

It's a situation where the stakeholders don't all coordinate (even if that's short of whole-hearted agreement), and would have to in order for there to be a political choice, rather than a technical one; so whatever bias the technical progress has, will be the direction of travel.

The tags appear not to show on the email that goes out, that contains copies of curated posts.  Does anyone think that that's an oversight?  If so, what do you think the next steps should be?

Can we have this as the pedantry thread?  Once upon a time, I was trying to empty my browser to reboot my machine, then I got nerd-sniped.

Part 3: not my type
Part 3: symmetry-breaking

My version of numbering works a bit differently.

Evolutionary biologists discussing boobs

I'm having trouble verifying this image :-)  I presume without justificaiton that it was not created specifically for reddit, and originally came from somewhere else.

In general, than you for the article.  I agree that it is clearly written.

I'm noticing that, on the original on X, there are very few branches on the left part of the coloured line.

There's also (annecdotally) a spike of spontaneous abortions three months into a pregnancy.  Ignoring that I'm stacking theory on top of poor data, my reading is that it's a round of genetic quality-control, to mimize the cost of doomed pregnancies.

disembodied software (“brains”), providing information on what should be done and how

I wish to make a very small quibble.  The software is not disembodied.  It has some specific physical embodiment.  The fact that does the work for you, is that it could have a different physical embodiment and cause the same act of production.  In the version of Computer Science that I learned, I would have called that 'implementation independent'.  I've been down a rabit-hole and not found a citation that I like.  I have found this partial rebuttal of the concept:

https://wiki.c2.com/?ImplementationIndependenceLimits

I think that this is a good place to note the joint decision trap, where courts tend to order integration, and the coordination costs of resisting it are large:

https://en.wikipedia.org/wiki/Joint_decision_trap

I don't know whether I'm an optimist or a doomer.  I have two very specific responses to different parts of the situation:

the establishment of AI red lines

Ok.  So, if:

• a mixture of humans and LLMs together make a decision and carry it out, then
• the act and/or some of its consequences is criminalizable,

then we have procedures (however imperfect) for allocating criminal liability among humans.  What does it mean to allocate criminal liability to LLMs?

My proposed answer is: it means restricting or prohibiting the use of that collection of weights... (read more)

Changing from training to test data (CTT; I may have made this up) isn't exactly the same as going out of distribution (OOD), but I currently think that that change is the proto-version of going OOD.

The evidence about CTT says that bigger models eventually do better:
https://www.lesswrong.com/posts/FRv7ryoqtvSuqBxuT/understanding-deep-double-descent
but someone could probably usefully summarise the new results in "double descent".

a particular technique doesn’t immediately solve a problem

I remember a story that got coverage on the state radio in New Zealand years ago.  It said that multiple people have parts of the solution to some problem, and there is progress when there is an accident that introduces them to each other.  There was a book about it, but I'm failing to find the details.

implement a relatively limited policy

I read this as Libertarian; the hope that there could be a very stiff, strong government that was also small, and did only a subset of the things in the short-term interest of its supporters.

Alignment isn’t like that; it was chosen to be an important problem

Like medicine.

This was specifically commented on in a book whose preface I read as a child.  It was called something like "Medicine: from science to magic", and I have not found a clear link back to it.

Furthe to Matt,

I like this distinction.  At the cost of generalising from fiction, in "A Civil Campaign", Lois McMaster Bujold phrased it as: "Reputation is what other people know about you. Honour is what you know about yourself."  Quoted here:

https://tvtropes.org/pmwiki/pmwiki.php/WhatYouAreInTheDark/Literature

Further to Kaj and Eric,

a fear of the career consequences of being in the line of fire

this sounds like people who are in the middle of an Immoral Maze.  That's probably statistically true, because any corporation large enough to be worth attacking is probably large enough to have three layers of middle-management.

Assuming that, doing 'honour' requires having goals other than power-seeking which, according to that sequence, makes one untrustworthy for the modal middle-manager, who has sacrificed everything else to it, and professionally doomed.