LESSWRONG
LW

383
Wei Dai
43775Ω3180148522818
Message
Dialogue
Subscribe

If anyone wants to have a voice chat with me about a topic that I'm interested in (see my recent post/comment history to get a sense), please contact me via PM.

My main "claims to fame":

  • Created the first general purpose open source cryptography programming library (Crypto++, 1995), motivated by AI risk and what's now called "defensive acceleration".
  • Published one of the first descriptions of a cryptocurrency based on a distributed public ledger (b-money, 1998), predating Bitcoin.
  • Proposed UDT, combining the ideas of updatelessness, policy selection, and evaluating consequences using logical conditionals.
  • First to argue for pausing AI development based on the technical difficulty of ensuring AI x-safety (SL4 2004, LW 2011).
  • Identified current and future philosophical difficulties as core AI x-safety bottlenecks, potentially insurmountable by human researchers, and advocated for research into metaphilosophy and AI philosophical competence as possible solutions.

My Home Page

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
10Wei Dai's Shortform
Ω
2y
Ω
385
Wei Dai's Shortform
Wei Dai31m20

It looks like I agreed with you too quickly. Just double-checked with Gemini Pro 3.0, and its answer looks correct to me:

This is a fascinating turn of events. Oliver is quoting from Section 3.01 of the bylaws, but he appears to be missing the critical conditional clause that precedes the text he quoted.

If you look at the bottom of Page 11 leading into Page 12 of the PDF, the sentence structure reveals that the "Directors = Members" rule is a fail-safe mechanism that only triggers if the initial member (Oliver) dies or becomes incapacitated without naming a successor.

Here is the text from the document:

[Page 11, bottom] ...Upon the death, resignation, or incapacity of all successor Members where no successor [Page 12, top] Member is named, (1) the directors of this corporation shall serve as the Members of this corporation...

By omitting the "Upon the death, resignation, or incapacity..." part, he is interpreting the emergency succession plan as the current operating rule.

Reply
Wei Dai's Shortform
Wei Dai41m20

Ah, I had indeed missed that part. A couple of AIs I asked also missed it, and together with the quoted statement from you, made me pretty sure my interpretation was correct. Sorry, and thanks for the quick correction. I've edited my post, and hope it didn't mislead too many people.

[This comment is no longer endorsed by its author]Reply
Problems I've Tried to Legibilize
Wei Dai1h20

Yeah, a book as successful as Superintelligence could do a lot. Once the LW team implements the ability to download my LW post/comment history, I'll check how far a modern LLM can get with turning it into a book. (@habryka) Unfortunately the thought of writing a whole book by hand does not fill me with motivation, so it would have to be a shortcut like this, or someone else writing it.

Reply
Wei Dai's Shortform
Wei Dai1h28

Ah thanks, I remember a bit more now. Looking back at the voting announcement posted by Vaniver, it didn't mention how important your role would be on LW 2.0:

In case you’re just tuning in now, some basic details: I’ve been posting on LW for a long time, and about two years ago thought I was the person who cared most about making sure LW stayed alive, so decided to put effort into making sure that happened. But while I have some skills as a writer and a programmer, I’m not a webdev and not great at project management, and so things have been rather slow. My current role is mostly in being something like the ‘senior rationalist’ on the team, and supporting the team with my models of what should happen and why. The actual work is being done by a combination of Oliver Habryka, Raymond Arnold, and Ben Pace, and their contributions are why we finally have a site that’s ready to come out of beta.

And I didn't pay much attention to the LW 2.0 / Lightcone organizational structure in the following years, so it came as kind of a surprise when you said "This is (approximately) my forum."

Reply
Wei Dai's Shortform
Wei Dai1h20

According to the bylaws I linked, you (as the sole member of Lightcone) have "the exclusive right to remove a director, with or without cause". Since the bylaws also allow Lightcone to have as few as 1 director, my understanding is that at any time, you could choose to invoke the option of removing the other directors and become the sole director. (I'm not familiar with the nonprofit world, and don't know how common or standard this is, but it seems fair to describe this as an organization controlled by one individual.)

Reply
Wei Dai's Shortform
Wei Dai2h*17-2

This is (approximately) my forum.

I was curious what Habryka meant when he said this. Don't non-profits usually have some kind of board oversight? It turns out (from documents filed with the State of California), that Lightcone Infrastructure, which operates LW, is what's known as a sole-member nonprofit, with a 1-3 person board of directors determined by a single person (member), namely Oliver Habryka. (Edit: It looks like this is correct after all, but was unintentional. See Habryka's clarification.)

However, it also looks like the LW domain is owned by MIRI, and MIRI holds the content license (legally the copyright is owned by each contributor and licensed to MIRI for use on LW). So if there was a big enough dispute, MIRI could conceivably find another team to run LW.

I'm not sure who owns the current code for LW, but I would guess it's Lightcone, so MIRI would have to also recreate a codebase for it (or license GreaterWrong's, I guess).

I was initially confused why Lightcone was set up that way (i.e., why was LW handed over to an organization controlled by a single person), but the structure probably makes it more nimble and the risk of Lightcone "going rogue" is mitigated to a large extent by MIRI retaining the option to swap out the team.

Anyway it took me a while to figure all this out, and I thought I'd share it so others would be informed while participating on LW.

Reply
Wei Dai's Shortform
Wei Dai20h40

Thanks, to clarify some more in case it's helpful, I think I've only complained about 2 things, the Said banning and the author moderation policy, and the word count was just from a lot of back and forth, not the number issues I've had with the mod team? A lot of what you do is just invisible to me, like the user pre-filtering that habryka mentioned and the routine moderation work, but I assume you're doing a good job on them, as I'm pretty happy with the general LW environment as far as lack of spam, generally good user behavior, and not seeing many complaints about being unfairly moderated by the mod team, etc.

Found my quote about not leaving:

My response to this is that I don't trust people to garden their own space, along with other reasons to dislike the ban system. I'm not going to leave LW over it though, but just be annoyed and disappointed at humanity whenever I'm reminded of it.

Yeah I think you misinterpreted it. I was just trying to say that unlike those who got what they wanted (the author mod policy) by leaving or threatening to leave, I'm explicitly not using this threat as a way to get what I want. It was a way to claim the moral high ground I guess. Too bad the message misfired.

Reply1
Wei Dai's Shortform
Wei Dai22h20

@Ben Pace I'm surprised that you're surprised. Where did your impression that I generally disapprove of the job site moderators are doing on LW come from, if you can recall?

Reply
Wei Dai's Shortform
Wei Dai22h1310

There is actually a significant difference between "Nowhere on the whole wide internet works like that!" and "few places work like that". It's not just a nitpick, because to support my point that it will be hard for Eliezer to get social legitimacy for freely exercising author mod power, I just need that there is a not too tiny group of people on the Internet who still prefers to have no author moderation (it can be small in absolute numbers, as long as it's not near zero, since they're likely to congregate at a place like LW that values rationality and epistemics). The fact that there are still even a few places on the Internet that works like this makes a big difference to how plausible my claim is.

Reply1
Wei Dai's Shortform
Wei Dai1d144

Not all moderation is censorship.

I'm not saying this, nor are the hypothetical people in my prediction saying this.

Claiming this is about "rationality" feels like mostly a weird rhetorical move.

We are saying that there is an obvious conflict of interest when an author removes a highly upvoted piece of criticism. Humans being biased when presented with COIs is common sense, so connecting such author moderation with rationality is natural, not a weird rhetorical move.

The rest of your comment seems to be forgetting that I'm only complaining about authors having COI when it comes to moderation, not about all moderation in general. E.g. I have occasional complaints like about banning Said, but generally approve of the job site moderators are doing on LW. Or if you're not forgetting this, then I'm not getting your point. E.g.

I don't think it's rational to pretend that unmoderated discussion spaces somehow outperform moderated ones.

I have no idea how this related to my actual complaint.

Reply11
Load More
150Please, Don't Roll Your Own Metaethics
Ω
6d
Ω
59
126Problems I've Tried to Legibilize
Ω
10d
Ω
24
348Legible vs. Illegible AI Safety Problems
Ω
9d
Ω
93
71Trying to understand my own cognitive edge
16d
17
10Wei Dai's Shortform
Ω
2y
Ω
385
66Managing risks while trying to do good
2y
28
49AI doing philosophy = AI generating hands?
Ω
2y
Ω
24
228UDT shows that decision theory is more puzzling than ever
Ω
2y
Ω
56
163Meta Questions about Metaphilosophy
Ω
2y
Ω
80
34Why doesn't China (or didn't anyone) encourage/mandate elastomeric respirators to control COVID?
Q
3y
Q
15
Load More
Carl Shulman
2 years ago
Carl Shulman
2 years ago
(-35)
Human-AI Safety
2 years ago
Roko's Basilisk
7 years ago
(+3/-3)
Carl Shulman
8 years ago
(+2/-2)
Updateless Decision Theory
12 years ago
(+62)
The Hanson-Yudkowsky AI-Foom Debate
13 years ago
(+23/-12)
Updateless Decision Theory
13 years ago
(+172)
Signaling
13 years ago
(+35)
Updateless Decision Theory
15 years ago
(+22)
Load More