All of JenniferRM's Comments + Replies

I think the utility function and probability framework from VNM rationality is a very important kernel of math that constrains "any possible agent that can act coherently (as a limiting case)".

((I don't think of the VNM stuff as the end of the story at all, but it is an onramp to a larger theory that you can motivate and teach in a lecture or three to a classroom. There's no time in the VNM framework. Kelly doesn't show up, and the tensions and pragmatic complexities of trying to apply either VNM or Kelly to the same human behavioral choices in real life a... (read more)

The intellectually hard part of Kant is coming up with deontic proofs for universalizable maxims in novel circumstances where the total list of relevant factors is large. Proof generation is NP-hard in the general case!

The relatively easy part is just making a list of all the persons and making sure there is an intent to never treat any of them purely as a means, but always also as an end in themselves. Its just a checklist basically. To verify that it applies to N people in a fully connected social graph is basically merely O(N^2) checks of directional bi... (read more)

I laughed out loud on this line...

Perhaps my experience in the famously kindly and generous finance industry has not prepared me for the cutthroat reality of nonprofit altruist organizations.

...and then I wondered if you've seen Margin Call? It is truly a work of art.

My experiences are mostly in startups, but rarely on the actual founding team, so I have seen more stuff that was unbuffered by kind, diligent, "clueless" bosses.

My general impression is that "systems and processes" go a long way into creating smooth rides for the people at the bottom, but tho... (read more)

With apologies for the long response... I suspect the board DID have governance power, but simply not decisive power.

Also it was probably declining, and this might have been a net positive way to spend what remained of it... or not?

It is hard to say, and I don't personally have the data I'd need to be very confident. "Being able to maintain a standard of morality for yourself even when you don't have all the data and can't properly even access all the data" is basically the core REASON for deontic morality, after all <3

Naive consequentialism has a huge ... (read more)

That's part of the real situation though. Sam would never quit to "spend more time with his family".

When we predict good outcomes for startups, the qualities that come up in the supporting arguments are toughness, adaptability, determination. Which means to the extent we're correct, those are the qualities you need to win.

Investors know this, at least unconsciously. The reason they like it when you don't need them is not simply that they like what they can't have, but because that quality is what makes founders succeed.

Sam Altman has it. You could parachut

I wrote a LOT of words in response to this, talking about personal professional experiences that are not something I coherently understand myself as having a duty (or timeless permission?) to share, so I have reduced my response to something shorter and more general. (Applying my own logic to my own words, in realtime!)

There are many cases (arguably stupid cases or counter-producive cases, but cases) that come up more and more when deals and laws and contracts become highly entangling.

Its illegal to "simply" ask people for money in exchange for giving them... (read more)

When I read this part of the letter, the authors seem to be throwing it in the face of the board like it is a damning accusation, but actually, as I read it, it seems very prudent and speaks well for the board.

You also informed the leadership team that allowing the company to be destroyed “would be consistent with the mission.”

Maybe I'm missing some context, but wouldn't it be better for Open AI as an organized entity to be destroyed than for it to exist right up to the point where all humans are destroyed by an AGI that is neither benevolent nor "aligned ... (read more)

This is a diagram explaining what is, in some sense, the fundamental energetic numerical model that explains "how life is possible at all" despite the 2nd law:

The key idea is, of course, activation energy (and the wiki article on the idea is the source of the image).

If you take "the focus on enzymes" and also the "background of AI" seriously, then the thing that you might predict would happen is a transition on Earth from a regime where "DNA programs coordinate protein enzymes in a way that was haphazardly 'designed' by naturalistic evolution" to a regime ... (read more)

Your summary did not contain the keyword "unlearning" which suggested that maybe he people involved didn't know about how Hopfield Networks form spurious memories by default that need to be unlearned. However, article you linked mentions "unlearn" 10 times so my assumption is that they are aware of this background and re-used the jargon on purpose.

So the way humans solve that problem is (1) intellectual humility plus (2) balance of power.

For that first one, you aim for intellectual humility by applying engineering tolerances (and the extended agentic form of engineering tolerances: security mindset) to systems and to the reasoner's actions themselves. 

Extra metal in the bridge. Extra evidence in the court trial. Extra jurors in the jury. More keys in the multisig sign-in. Etc.

(All human institutions are dumpster fires by default, but if they weren't then we would be optimizing the value of info... (read more)

Assuming we have a real uh... real "agent agent" (like a thing which has beliefs for sane reasons and plans and acts in coherently explicable ways and so on) then I think it might just be Correct Behavior for some extreme versions of "The Shutdown Problem" to be mathematically impossible to "always get right".

Fundamentally: because sometimes the person trying to turn the machine off WILL BE WRONG.

...

Like on Petrov Day, we celebrate a guy whose job was to press a button, and then he didn't press the button... and THAT WAS GOOD.

Petrov had Official Evidence t... (read more)

Neat!

The above is figure 1 from the 2011 paper "Assessment of synchrony in multiple neural spike trains using loglinear point process models".

The caption for the figure is:

Neural spike train raster plots for repeated presentations of a drifting sine wave grating stimulus. (A) Single cell responses to 120 repeats of a 10 second movie. At the top is a raster corresponding to the spike times, and below is a peri-stimulus time histogram (PSTH) for the same data. Portions of the stimulus eliciting firing are apparent. (B) The same plots as in (A), for a d

Concretely: I wish either or both of us could get some formal responses instead of just the "voting to disagree".

In Terms Of Sociological Abstractions: Logically, I understand some good reasons for having "position voting" separated from "epistemic voting" but I almost never bother with the later since all I would do with it is downvote long interesting things and upvote short things full of math.

But I LIKE LONG INTERESTING THINGS because those are where the real action (learning, teaching, improving one's ontologies, vibing, motivational stuff, fact... (read more)

So this caught my eye:

If you believe that the only path to compute governance is a surveillance state, and you are accelerating AI and thus when we will need and when we will think we need such governance, what are the possibilities?

I'm somewhat sympathetic to "simply ban computers, period" where you don't even need a "total surveillance state", just the ability to notice fabs and datacenters and send cease and desist orders (with democratically elected lawful violence backing such orders).

Like if you think aligning AI to humanistic omnibenevolence is basi... (read more)

A bold move! I admire it the epistemology of it, and your willingness to back it with money! <3

Importing some very early comments from YouTube, which I do not endorse (I'd have to think longer), but which are perhaps interesting for documenting history, and tracking influence campaigns and (/me shrugs) who knows what else?? (Sorted to list upvotes and then recency higher.)

@Fiolsthu95 3 hours ago +2

I didn't ever think I'd say this but.. based Trump?!?

@henrysleight7768 1 hour ago +1

"What Everyone in Technical Alignment is Doing and Why" could literally never 

@scottbanana1 3 hours ago +1

The best content on YouTube

@anishupadhayay3917 14 minutes ago... (read more)

Here I'm going to restrict myself to defending my charitable misinterpretation of trevor's claim and ignore the FDA stuff and focus on the way that the Internet Of Things (IoT) is insecure.

I. Bluetooth Headsets (And Phones In General) Are Also Problematic

I do NOT have "a pair of Bluetooth headphones, which I use constantly".

I rarely put speakers in my ears, and try to consciously monitor sound levels when I do, because I don't expect it to have been subject to long term side effect studies or be safe by default, and I'd prefer to keep my hearing and avoid ... (read more)

I was curious about the hypothetical mechanism of action here!

I hunted until I found a wiki page, and then I hunted until I found a citation, and the place I landed as "probably the best way to learn about this" was a podcast!

SelfHacked Radio, Dec 19, 2019, "Microdosing with Dr. David Rabin" (53 minutes)

[Intro:] Today, I’m here with Dr. David Rabin, who is a psychiatrist and neuroscientist. 

We discuss PTSD, psychedelics and their mechanisms, and the different drugs being used for microdosing.

I have not listened to the podcast, but this wiki article cites... (read more)

If I was going to try to charitably misinterpret trevor, I'd suggest that maybe he is remembering that "the S in 'IoT' stands for Security". 

(The reader stops and notices: I-O-T doesn't contain an S... yes! ...just like such devices are almost never secure.) So this particular website may have people who are centrally relevant to AI strategy, and getting them all to wear the same insecure piece of hardware lowers the cost to get a high quality attack? 

So for anyone on this site who considers themselves to be an independent source of world-saving ... (read more)

I see there as being (at least) two potential drivers in your characterization, that seem to me like they would suggest very different plans for a time traveling intervention. 

Here's a thought experiment: you're going to travel back in time and land near Gnaeus Pompeius Magnus, who you know will (along with Marcus Licinius Crassus) repeal the constitutional reforms of Sulla (which occurred in roughly 82-80 BC and were repealed by roughly 70BC).

Your experimental manipulation is to visit the same timeline twice and either (1) hang out nearby and help dr... (read more)

🕯️

I apologize! Is there anything (1) I can afford that (2) might make up for my share of the causality in the harm you experienced (less my net causal share of benefits)?

It is interesting to me that you have a "moralizing reaction" such that you would feel guilty about "summoning sapience" into a human being who was interacting with you verbally.

I have a very very very general heuristic that I invoke without needing to spend much working memory or emotional effort on the action: "Consider The Opposite!" (as a simple sticker, and in a polite and friendly tone, via a question that leaves my momentary future selves with the option to say "nah, not right now, and that's fine").

So a seemingly natural thing that occurs to me is ... (read more)

I am struck by the juxtaposition between: calling the thing "sapience" (which I currently use to denote the capacity for reason and moral sentiment, and which I think of as fundamentally connected to the ability to negotiate in words) and the story about how you were sleep walking through a conversation (and then woke up during the conversation when asked "Can you speak more plainly?").

Naively, I'd think that "sapience" is always on during communication, and yet, introspecting, I do see that some exchanges of words have more mental aliveness to them than o... (read more)

Apologies for TWO comments (here's the other), but there are TWO posts here! I'm justified I think <3

I slip a lot, but when I'm being "careful and good in my speech" I distinguish between persons, and conscious processes, and human beings.

A zygote, in my careful language, is a technical non-central human being, but certainly not a person, and (unless cellular metabolism turns out to have "extended inclusive sentience") probably not "conscious".

...

I. Something I think you didn't bring up, that feels important to me, is that the concept of "all those able... (read more)

I tried to attribute each theory to some "philosophy hero", then I used Critch's N counts and Huffman Encoded thusly:

"0" = "Buddha" 
"10" = "Metzinger" 
"1100" = "Descartes" 
"1101" = "Heidegger" 
"11110" = "Pollock" 
"111110" = "Nagel" 
"111111" = "Hume"

This is NOT a unique Huffman Encoding (the 2s can be hot-swapped, which would recluster things):

Buddha(14)-----------------------------------------------------------------0|--"All!"
Metzinger(7.5)-----------------------------------------0|--"Science"(10.5)-1| 
Descartes(4)-0|--"Cont

I beg the tolerance of anyone who sees these two very long comments.

I personally found it useful to learn "yet another of my interlocutors who seems to be opposed to AI regulations has just turned out to just be basically an anarchist at heart".

Also, Shankar and I have started DMing a bunch, to look for cruxes, because I really want to figure out how Anarchist Souls work, and he's willing to seek common epistemic ground, and so hopefully I'll be able to learn something in private, and me and Shankar can do some "adversarial collaboration" (or whatever), an... (read more)

When you say "I do not blame a slave for his submission" regarding Daylight Savings Time, that totally works in the second frame where "l'état ce ne sont que des bureaucrates humains". 

You're identifying me, and you, and all the other "slaves" as "mere transistors in society".

I dunno about you, but I grew up in a small (unincorporated) town, that was run by the Chamber of Commerce and Rotary Club. My first check as pay for labor was as a soccer referee when I was 12, reffing seven year olds. There was a Deputy of the County Sheriff, but he was not the... (read more)

"L'état c'est nous" though? (The state, it is us.)

I'm pretty sure I am not an eldritch horror and I suspect you aren't either, Shankar! Does the "eldritch horror part" arises from our composition? Is so, why and how? Maybe it is an aspect of humans that emerges somehow from a large number of humans?

"L'état ce ne sont que des bureaucrates humains" is another possibility (the state, it is merely some human bureaucrats) who I fully admit might not be saints, and might not be geniuses, but at least they are humans, operating at human speeds, with human common ... (read more)

Or a pre-school or kindergym or (if a building design is opulent enough to offer room-specific temperature control) a two-year-old's bedroom?

Small bodies have much higher surface area to volume ratios, and a 10 month old can barely even explain the problem they face!

In grocery stores when I was really little, I'd stay "just outside" the cold aisle, and then run to the other end to try to avoid the chill, when along with parents on a shopping trip who wanted to loiter in the middle of it. It was only much later that I understood the physics of why they weren't bothered, and the psycho-politics of why no one optimized that stuff "for me".

Oooh! High agreement on something this downvoted is curiosity catnip!

(Currently I see -18 for position, and +7 for agreement... I haven't touched either button, but I'll definitely upvote a response to my questions here <3)

I thought "this is nice" would be a common human reaction, but apparently I'm miscalibrated?

The "agreement votes" suggest that even people who think you're being mean kinda grudgingly admit that you're saying something accurate...

...but like... What? 

Don't "normal people" also like in a basic public space (that isn't a museum or ... (read more)

I think offices are kept at a "low" temperature because there is actually wide variation in temperature preferences and tolerances among normal humans, and maybe also because it is considered easier for women and skinny people to add a sweater than for others to change gender, lose weight, or wear ice packs.

I think I approve of this for spaces that aren't going to have kids, but I think that for kid-centric spaces a higher temperature than is maximally comfy for large men is still correct? Maybe? 

(Or you could try to maintain gradients and zones? I've... (read more)

This is a great guide! I hit ^f[music] and don't see any hits, so I'll add that when I visited the Lightcone office, I was talking about something, and there was nice music in the background, and then I just had to interrupt myself, point at the speakers, and say "are we in the tropical village from breathe of the wild?... I think I love it!" and then we went back to chatting about <topic> after the nod and smile in response.

I'm not sure how standard this is, or what tools were used, but just adding this soundtrack (and stuff like it?) to the overall ambiance of the visuals and so on was quite nice :-)

"Early in the Reticulum[Internet] -thousands of years ago— it became almost useless because it was cluttered with faulty, obsolete, or downright misleading information," Sammann said. 

"Crap, you once called it," I reminded him.

"Yes-a technical term..."

...

"As a tactic for planting misinformation in the enemy’s reticules[webpages/webservers], you mean," Osa said. "This I know about. You are referring to the Artificial Inanity programs of the mid–First Millennium A.R." 

Source: Anathem (2009) transcription via Redditors fanning about it.

See also: bog... (read more)

I cannot quickly find a clean "smoking gun" source nor well summarized defense of exactly my thesis by someone else.

(Neither Google nor the Internet seem to be as good as they used to be, so I no longer take "can't find it on the Internet with Google" as particularly strong evidence that no one else has had the idea and tested and explored it in a high quality way that I can find and rely on if it exists.)

...in place of a link, I wrote 2377 more words than this, talking about the quality of the evidence I could find and remember, and how I process it, and ... (read more)

I've been having various conversations in private, where I'm quite doomist and my interlocutor is less doomist, and I think one of the key cruxes that has come up several times is that I've applied security mindset to the operation of human governance, and I am not impressed.

I looked at things like the federal reserve (and how you'd implement that in a smart contract) and the congress/president/court deal (and how you'd implement that in a smart contract) and various other systems, and the thing I found was that existing governance systems are very poorly ... (read more)

I don't know about you, but I'm actually OK dithering a bit, and going in circles, and doing things that mere entropy can "make me notice regret based on syntactically detectable behavioral signs" (like not even active adversarial optimization pressure like that which is somewhat inevitably generated in predator prey contexts).

For example, in my twenties I formed an intent, and managed to adhere to the habit somewhat often, where I'd flip a coin any time I noticed decisions where the cost to think about it in an explicit way was probably larger than the di... (read more)

I don't know if you're still working on this, but if don't already know of the literature on choice supportive bias and similar processes that occur in humans, they look to me a lot like heuristics that probably harden a human agent into being "more coherent" over time (especially in proximity to other ways of updating value estimation processes), and likely have an adaptive role in improving (regularizing?) instrumental value estimates.

Your essay seemed consistent with the claim that "in the past, as verifiable by substantial scholarship, no one ever prov... (read more)

Voting is, of necessity, pleiotropically optimized. It loops into reward structures for author motivation, but it also regulates position within default reading suggestion hierarchies for readers seeking educational material, and it also potentially connects to a sense that the content is "agreed to" in some sort of tribal sense.

If someone says something very "important if true and maybe true" that's one possible reason to push the content "UP into attention" rather than DOWN.

Another "attentional" reason might be if some content says "the first wrong idea ... (read more)

I'd like to say up front that I respect you both, but I think shminux is right that bhauth's article (1) doesn't make the point it needs to make to change the "belief about the whether a set of 'mazes' exist whose collective solution gives nano" for many people working on nano and (2) this is logically connected to issue of "motivational stuff".

A key question is the "amount of work" necessary to make intellectual progress on nano (which is probably inherently cross-disciplinary), and thus it is implicitly connected to motivating the amount of work a human ... (read more)