LESSWRONG
LW

1412
Ben Pace
36616Ω10912784758201
Message
Dialogue
Subscribe

I'm an admin of LessWrong. Here are a few things about me.

  • I generally feel more hopeful about a situation when I understand it better.
  • I have signed no contracts nor made any agreements whose existence I cannot mention.
  • I believe it is good to take responsibility for accurately and honestly informing people of what you believe in all conversations; and also good to cultivate an active recklessness for the social consequences of doing so.
  • It is wrong to directly cause the end of the world. Even if you are fatalistic about what is going to happen.

Randomly: If you ever want to talk to me about anything you like for an hour, I am happy to be paid $1k for an hour of doing that.

(Longer bio.)

Sequences

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
23Benito's Shortform Feed
Ω
7y
Ω
333
Legible vs. Illegible AI Safety Problems
Ben Pace1dΩ564

I think Eliezer has oft-made the meta observation you are making now, that simple logical inferences take shockingly long to find in the space of possible inferences. I am reminded of him talking about how long backprop took.

In 1969, Marvin Minsky and Seymour Papert pointed out that Perceptrons couldn't learn the XOR function because it wasn't linearly separable.  This killed off research in neural networks for the next ten years.

[...]

Then along came this brilliant idea, called "backpropagation":

You handed the network a training input.  The network classified it incorrectly.  So you took the partial derivative of the output error (in layer N) with respect to each of the individual nodes in the preceding layer (N - 1).  Then you could calculate the partial derivative of the output error with respect to any single weight or bias in the layer N - 1.  And you could also go ahead and calculate the partial derivative of the output error with respect to each node in the layer N - 2.  So you did layer N - 2, and then N - 3, and so on back to the input layer.  (Though backprop nets usually had a grand total of 3 layers.)  Then you just nudged the whole network a delta - that is, nudged each weight or bias by delta times its partial derivative with respect to the output error.

It says a lot about the nonobvious difficulty of doing math that it took years to come up with this algorithm.

I find it difficult to put into words just how obvious this is in retrospect.  You're just taking a system whose behavior is a differentiable function of continuous paramaters, and sliding the whole thing down the slope of the error function.  There are much more clever ways to train neural nets, taking into account more than the first derivative, e.g. conjugate gradient optimization, and these take some effort to understand even if you know calculus.  But backpropagation is ridiculously simple.  Take the network, take the partial derivative of the error function with respect to each weight in the network, slide it down the slope.

If I didn't know the history of connectionism, and I didn't know scientific history in general - if I had needed to guess without benefit of hindsight how long it ought to take to go from Perceptrons to backpropagation - then I would probably say something like:  "Maybe a couple of hours?  Lower bound, five minutes - upper bound, three days."

"Seventeen years" would have floored me.

Reply
People Seem Funny In The Head About Subtle Signals
Ben Pace2d140

I am starting to get something from these posts.

Reply
Reasons against donating to Lightcone Infrastructure
Ben Pace4d20

I feel confused about the notion that people only want to donate to a thing if they will be on the hook for needing to donate every year forevermore to keep it afloat, as opposed to donating to cause it to get its business in order and then it can sustain itself.

Reply
Reasons against donating to Lightcone Infrastructure
Ben Pace6d180

He just learned that keeping secrets is bad in general, and so he doesn’t by default, unless explicitly agrees to.

This is not true! My policy is simply that you should not assume that I will promise to keep your secrets after you tell me, if you didn't check with me first.

I can confirm; Oliver keeps many secrets from me, that he has agreed to others, and often keeps information secret based on implicit communication (i.e. nobody explicitly said that it was secret, but his confident read of the situation is that it was communicated with that assumption). I sometimes find this frustrating because I want to know things that Oliver knows :P 

Reply1
Reasons against donating to Lightcone Infrastructure
Ben Pace6d50

And he sent the message in a way that somehow implied that I was already supposed to have signed up for that policy, as if it's the most normal thing in the world, and with no sense that this is a costly request to make (or that it was even worth making a request at all, and that it would be fine to prosecute someone for violating this even if it had never been clarified at all as an expectation from the other side).

Speaking generally, many parties get involved in zero-sum resource conflicts, and sometimes form political alliances to fight for their group to win zero-sum resource conflicts. For instance, if Alice and Bob are competing to get the same job, or Alice is trying to buy a car for a low price and Bob is trying to sell it to her for a high price, then if Charlie is Alice's ally, she might hope that Charlie all will take actions that help her get more/all of the resources in these conflicts.

Allies of this sort also expect that they can share information that is easy to use adversarially against them between each other, with the expectation it will be consistently used either neutrally or in their favor by the allies.

Now, figuring out who your allies are is not a simple process. There are no forms involved, there are no written agreements, it can be fluid, and picked up in political contexts by implicit signals. Sometimes you can misread it. You can think someone is allied, tell them something sensitive, then realize you tricked yourself and just gave sensitive information to someone. (The opposite error also occurs, where you don't realize someone is your ally and don't share info and don't pick up all the value on the table.)

My read here is that Mikhail told Habryka some sensitive information about some third party "Jackson", assuming that Habryka and Jackson were allied. Habryka, who was not allied with Jackson in this way, was simply given a scoop, and felt free to share/use that info in ways that would cause problems for Jackson. Mikhail said that Habryka should treat it as though they were allies, whereas Habryka felt that he didn't deserve it and that Mikhail was saying "If I thought you would only use information in Jackson's favor when telling you the info, then you are obligated to only use information in Jackson's favor when using the info." Habryka's response is "Uh, no, you just screwed up."

(Also, after finding out who "Jackson" is from private comms with Mikhail, I am pretty confused why Mikhail thought this, as I think Habryka has a pretty negative view of Jackson. Seems to me simply like a screw-up on Mikhail's part.)

Reply
Mikhail Samin's Shortform
Ben Pace8d20

Second one seems reasonable. 

Clarifying in the first case: If Bob signs up and DMs 20 users, and one reports spam, are you saying that we can only check his DM, or that at this time we can then check a few others (if we wish to)?

Reply1
Vale's Shortform
Ben Pace11d40

Not sure if you're intending to disagree, but I do sometimes have like a post-list or the quick-takes fail to load, with a red error message instead, and then if I refresh it goes away.

(I can't recall it happening very often, mostly I see it when I run a dev instance.)

Reply1
1a3orn's Shortform
Ben Pace11d75

I have a hard time imagining someone writing this without subtweeting. Feels like classic subtweeting to me, especially "I think this is pretty obvious". Like, it's a trivially true point, all the debate is in the applicability/relevance to the situation. I don't see any point in it except the classic subterfuge of lowering the status of something in a way that's hard for the thing to defend itself against.

My standard refrain is that open aggression is better than passive aggression. The latter makes it hard to trust things / intentions, and makes people more paranoid and think that people are semi-covertly coordinating to lower their status around them all the time. For instance, and to be clear this is not the current state, but it would not be good for the health of LW for people to regularly see people discussing "obvious" points in shortform and ranting about people not getting them, and later find out it was a criticism of them about a post that they didn't think would be subject to that criticism!

Reply11
kave's Shortform
Ben Pace12d111

There is a strong force in web forums to slide toward news and inside-baseball; the primary goal here is to fight against that. It is a bad filter for new users if a lot of that they see on first visiting the LessWrong homepage is discussions of news, recent politics, and the epistemic standards of LessWrong. Many good users are not attracted by these, and for those not put off it's bad culture to set this as the default topic of discussion.

(Forgive me if I'm explaining what is already known, I'm posting in case people hadn't heard this explanation before; we talked about it a lot when designing the frontpage distinction in 2017/8.)

Reply1
Load More
AI Alignment Writing Day 2019
Transcript of Eric Weinstein / Peter Thiel Conversation
AI Alignment Writing Day 2018
Share Models, Not Beliefs
134The Inkhaven Residency
3mo
35
37LessOnline 2025: Early Bird Tickets On Sale
8mo
5
20Open Thread Spring 2025
8mo
50
281Arbital has been imported to LessWrong
9mo
30
141The Failed Strategy of Artificial Intelligence Doomers
9mo
77
109Thread for Sense-Making on Recent Murders and How to Sanely Respond
9mo
146
83What are the good rationality films?
Q
1y
Q
54
942024 Petrov Day Retrospective
1y
25
136[Completed] The 2024 Petrov Day Scenario
1y
114
55Thiel on AI & Racing with China
1y
10
Load More
LessWrong Reacts
a month ago
LessWrong Reacts
a month ago
LessWrong Reacts
a month ago
LessWrong Reacts
a month ago
(+3354/-3236)
LessWrong Reacts
a month ago
LessWrong Reacts
a month ago
(+638/-6)
LessWrong Reacts
a month ago
(+92)
LessWrong Reacts
a month ago
(+248)
Adversarial Collaboration (Dispute Protocol)
10 months ago