What's up with Arbital?

[-]WhySpace_duplicate0.92616921290755279y200

This is really sad. I'm sorry to hear things didn't work out, but I'm still left wondering why not.

I guess I was really hoping for a couple thousand+ word post-mortem, describing the history of the project, and which hypotheses you tested, with a thorough explanation of the results.

If you weren't getting enough math input, why do you think that throwing more people at the problem wouldn't generate better content? Just having a bunch of links to the most intuitive and elegant explanations, gathered in one place, would be a huge help to both readers and writers. Students trying to learn are already doing this through blind googling, so the marginal work to drop the links is low.

Pulling all the info together into a good explanation still requires one dedicated person, but perhaps that task can be broken down into chunks too. Like, once one version is written, translating it for non-mathy people should be relatively easy. Same for condensing things for mathy people.

But, why wouldn't adding more mathematicians mean a few would be good at and interested in writing new articles? Where did you do outreach? What did you do? There are entire communities, scattered across the web, who exist ... (read more)

7Alexei9y

Yes, many students would benefit from a math explanation platform. But it was hard for us to find writers, and we weren't getting as much traction with them as we wanted. We reached out to some forums and to many individuals. That version of Arbital was also promoted by Eliezer on FB. When we switched away from math, it wasn't because we thought it was hopeless. We had a lot of ideas left to try out. But when it's not going well, you have to call it quits at some point, and so we did. There was also the consideration that if we built a platform for (math) explanations, it would be hard to eventually transition to a platform that solved debates (which always seemed like the more important part). I think if someone wanted to give it a shot with another explanation platform and had a good strategy for getting writers, I'd feel pretty optimistic about their chance of success.

[-]Viliam9y180

I feel sad that the project is gone before I even understood how it was supposed to work.

I was like: "I have absolutely no idea what this is supposed to be or to do, but smart people seem enthusiastic about it, so it's probably a smart thing, and maybe later when I have more time, I will examine it more closely."

Now, my question is... how much should I use this as an outside view for other activities of MIRI?

5pcm9y

I'm unsure whether you should think of it as a MIRI activity, but to the extent you should, then it seems like moderate evidence that MIRI will try many uncertain approaches, and be somewhat sensible about abandoning the ones that reach a dead end.

4Alexei9y

Don't use the outside view. Use your brain. If Arbital was confusing, but you didn't look closer at it, then you didn't use your brain. If MIRI seems confusing and you don't look closer at it, then you aren't using your brain. The whole concept of "smart people" whom you can trust with anything is just wrong. There are only niche experts. My two cents: Arbital had very little to do with MIRI, aside from Arbital being Eliezer's idea. But this was definitely out of his realm of expertise. MIRI/AI stuff is not.

6gjm9y

One of the things people (rightly) use their brains for is deciding what things merit closer attention. I don't think it's fair to say that Viliam didn't use his brain, merely because the conclusion from using his brain was "I don't care enough about this to examine it more closely right now". And his question about the outside view isn't about how he should have evaluated Arbital before but about how he should evaluate MIRI activities in the future. (Where, again, "not worth looking closer right now" is a perfectly legitimate evaluation.) Are you suggesting that as a matter of general policy one shouldn't use the outside view to evaluate how likely something is to succeed, to have large impact, etc.? Or does that only apply to certain projects, and if so which ones?

3ChristianKl9y

The fact that a startup pivots doesn't automatically mean that it's founders are bad at what they are doing. Most startups fail or pivot from the outside view.

3gjm9y

I don't think Viliam was claiming it means the founders are bad at what they're doing (in any sense stronger than that they aren't able to do it in such a way as to make a commercial success). Only that "smart people seem enthusiastic about it" is perfectly consistent with "nothing will come of it in the end".

[-]michaelkeenan9y180

Debates didn't work because... well, it's a very complicated problem.

I'd love to hear about this in more detail. What have you learned about the problem? Do you know what good solutions would look like, but they're too hard or expensive to implement? Or have you learned that it isn't feasible?

[-]Alexei9y210

Here is my person take on why it's complicated:

When you ask someone if they would like a debate platform and describe all the features and content it'll have, they go: "Hell yeah I'd love that!" And it took me a while to realize that what they are imagining is someone else writing all the content and doing all the heavy lifting. Then they would come along, read some of it, and may be leave a comment or two. And basically everyone is like that: they want it, but they are not willing to put in the work. And I don't blame them, because I'm not willing to put in the work (of writing) either. There are just a handful of people who are.

So the problem is definitely not on the technical side. It's a problem with the community / society in general. Except I'm hesitant to even call it a "problem," because that feels like calling gravity a "problem." This is just the way humans are. They want to do things they want to do.

[-]ChristianKl9y390

It seems to me like you never really tried to seriously invite people to participate and write content. The fact that people had to follow Arbitals process through "whispers, rumors, and clairvoyant divination" gave the impression that it was more in a closed state than that it was inviting people to participate.

8Alexei9y

That's a very good point. When we were doing math explanations, we did reach out to a lot of people (just not via LW). When we were doing debates, we reached out to a few people, because we didn't quite know what shape we wanted to the debate to take. So we didn't need that many people. (It would be a bit silly to move a community from one platform to another that's basically the same.) So, yes, there were multiple times where we thought that we should invite more people / throw open the doors. Some of those times we postponed it because we weren't ready; one of the other times we probably should have done it. You can think of this post as an invitation to use the platform.

[-]ChristianKl9y160

How about writing a "top 10 posts on Arbital" post for LW discussion? That way it's easier for people to see discussions to which they might want to contribute.

When we were doing debates, we reached out to a few people, because we didn't quite know what shape we wanted to the debate to take. So we didn't need that many people.

That's a valid strategy but if that's what you did, why do you think your experience proves that it's hard to get people to contribute to the discussion?

5Alexei9y

I think that's better done by people who are actively writing and want to invite commenters. Because we reached out to people who were pretty excited about the platform and who were already spending a lot of their time blogging / doing discussions. I imagine if we reached out to people who are less excited / who write less, we would have gotten even less of a response.

2[anonymous]9y

Given that you've been so involved, I don't think anyone would think it out of place for you to make a post. I, for one, would be happy to have a general "Introduction to the top 10 Arbital posts". Aside from the haziness and uncertainty, I also often felt off-put because of a lack of a prominent wiki / introduction to the vast space of things Arbital offers.

2math_viking9y

If you're still looking for content, I could be persuaded to start (slowly) working on some set theory/analysis concepts.

1Alexei9y

I'm not into persuading people. :) If you want to write, go for it. I still think Arbital is a really good platform for writing up math explanations.

1math_viking8y

I rather forgot about this due to being accidentally logged out of this account and not realizing so until later. If I were to try my hand at this, what would be the best place to see what's already been done (without having to click through arbital) and submit my contributions?

1Alexei8y

This will take a long time to load, but it's comprehensive: https://arbital.com/explore/math/

1math_viking8y

I'll take a look, thanks!

[-]VAuroch9y180

I agree with Christian. Did Arbital ever even come out of closed beta? My impression was that it did not, and you still needed to be whitelisted to have the chance to contribute.

6Viliam9y

Yeah, I can imagine myself doing a similar mistake. For websites, both the engine and the content are important. A bad engine can ruin otherwise good content (just look at the damage done to LW but not being able to effectively moderate sockpuppets and karma assasinations), but if you don't have enough content, it does not matter what engine you have, people will not come there. And the more options you want to provide for the reader, the more text needs to be written.

3ChristianKl9y

The start page also direct me to the latest discussions the way the reddit start page does. It's hard to actually find the discussions. If I go to a discussion like https://arbital.com/p/requisites_for_personal_growth/, there's no text field in which I can leave my comment, like there's on reddit (and here) or blogs that have a comment sections. I can click the button "Propose comment" but that design suggests to me that commenting isn't supposed to be my default reaction.

1Alexei9y

Correct. We (somewhat prematurely) worried about trolls, so by default people can only propose comments. And it would be up to Toon to approve them. (If there is sufficient demand, I can add a feature to let users have open commenting. But in general adding features to old Arbital is not high on my priority list.)

[-]Lumifer9y160

So, if I understand this correctly, you had a non-promoted invite-only platform which you think failed because not enough people contributed content?

I am confused. Surely it crossed someone's mind at some point...

3Alexei9y

See this comment: http://lesswrong.com/lw/otq/whats_up_with_arbital/dq9h I think we likely made a mistake with respect to openness, but it's not obvious when/how. Probably the biggest problem is that we couldn't settle on what we wanted the users to do once they were on the platform.

[-]Lumifer9y100

but it's not obvious when/how

Any. Fucking. Time.

I notice that you tell people to come to Arbital, but it is still invite-only.

we couldn't settle on what we wanted the users to do once they were on the platform

"...the street finds its own uses for things" -- William Gibson

Have you considered letting users play freely and then learning from them instead of trying to construct an optimal-by-some-criteria maze that mice surely will joyfully choose to run through?

2Alexei9y

How is it invite only? Are you talking about the comment section? Originally the plan was to do exactly that if we couldn't figure how to build a "joyful maze": just throw open the doors and see what people do with it. Unfortunately there is still a significant amount of work left to do that well, and right now I'm more optimistic about the new platform than I am about scavenging the current version.

2gilch9y

The last time I tried making an Arbital account, it failed. Does it require human approval? Then it's still invite-only. Is it broken? Then that's why no-one signed up.

1Alexei9y

I'm not sure when you tried. It works right now.

2Lumifer9y

Didn't you say which is an explicit whitelisting system?

1Alexei9y

Yes, but that's not "invite-only".

2Lumifer9y

You can knock on the door. But you have to be invited (=whitelisted) by someone with sufficient authority in order to enter.

1Alexei9y

Not someone with sufficient authority, just the blog owner. That seems fair though. You can create you own blog and then you would be in charge of which comments to approve.

9Lumifer9y

I am sure you are well aware of how default-approve (=blacklisting) and default-deny (=whitelisting) policies affect the popularity and usage of publishing platforms.

2ChristianKl9y

That doesn't seem the default way most blogs work. Most blogs simply allow you to leave a comment (or they don't have comment sections at all).

7ChristianKl9y

Why do you think it's a better decision to switch strategy to a microblogging platform instead of switching strategy to being as open and as inviting of contribution as possible?

1Alexei9y

1. Currently it's not clear to anyone what Arbital is, what it can do, who it's for, etc.. It needs to solve a real problem and present itself as solving that clear problem. 2. The tech we used is now somewhat obsolete. The codebase has accumulated a lot of unnecessary features. Also Google Material UI turned out to be too heavyweight and not as pleasant to design with as I thought initially. (These are all arguments for remaking the platform.) 3. The blogging platform will be "as open and as inviting of contribution as possible."

4Yosarian29y

Ok, but when you do create "Arbital 2.0", the best thing you can do to help it succeed is to have a pre-existing base of users who are interested in using it, especially the smaller number but extremly important "power users" who will be the ones who generate most of your content, and a lot of good content you can move over from your old site. Anything you can do to encourage people to be more active now at creating content (including comments) and interacting with the site in general probably included your odds of success long-term.

2plethora9y

This is what incentive structures are for. There are quite a few people who have strong incentives to publish high-quality writing, you know... An open-access journal for debates seems like it ought to be possible, although it'd have to actively solicit contributions (an encyclopedia for debates?) and reward them with academic status, which means you'd need solid academic backing.

[-]cousin_it9y150

That sure was an expensive way to test demand for Arbital 1.0. Have you thought about cheaper ways to test demand for Arbital 2.0?

2Alexei9y

Yes. There is already a pretty large demand for blogging platforms, and Arbital 2.0 will have features which will make it a much better option for some users. I'll also be personally reaching out to a lot of bloggers to interview them about their experience / wishes. I'll also be testing the key value propositions with a graphic that's being created right now. But also sometimes you just have to go ahead and build the thing to really test it.

[-]VAuroch9y140

It seems disingenuous to call this new project Arbital.

2Alexei9y

Why?

9gjm9y

Well, what does it have in common with the already-existing thing called Arbital?

0Alexei9y

It's a blogging platform, it's done by me with some support from Eliezer, and I'm doing it because it will help with x-risk. This is essentially identical to what we had in 2015.

[-]gjm9y260

because it will help with x-risk

A new microblogging platform will help with x-risk?

I mean, I know Tumblr is bad, but it's hardly an existential threat.

2mako yass9y

Any platform that contributes to poisoning public discourse antagonizes coordination efforts. I think entering that space is a valid EA pursuit. Anyone who's read meditations on moloch will do a better job of promoting the spread of truth and preventing spurious conflicts and than, EG, twitter.

0Alexei9y

That's step 1. Steps 2 and after involve slowly converging towards the original Arbital vision. I just don't think you can get there without mass adoption.

[-]gjm9y140

Is there actually any sort of obvious path from a microblogging platform to "the original Arbital vision"? I confess I don't really see one, but maybe I haven't understood what the original Arbital vision was.

-2Alexei9y

Here are like 5 pages explaining all the visions: https://arbital.com/p/more_about_arbital/ Basically what we tried is: "let's figure out how people are supposed to have truth-seeking conversations, build a platform that facilities that, and then grow it." Step 1 is very hard. Step 3 is made harder because your platform only attracts truth-seeking people. New approach: "build a platform that facilities communication, grow it, then shape the ongoing discussion to be more truth-seeking." Step 1 is still hard, but not made harder. Step 3 sounds a lot more doable.

3ChristianKl9y

Why do you believe that your new microblogging service will get mass adoption?

-5Alexei9y

8VAuroch9y

Arbital has vague positive affect from being an attempt to solve a big problem in a potentially really impactful way. Yet Another Blogging Platform, without the special features envisioned originally, is not solving a big problem (or actually any problem), and has a maximum plausible impact of "makes you a bunch of money and you donate that somewhere". Re-using the name is a self-serving attempt to redirect the positive affect from the ambitious, failed, altruistic project to the mundane, new, purely-capitalistic project. Why aren't you just admitting defeat and going on to build something different?

3Alexei9y

See my reply to gjm: http://lesswrong.com/lw/otq/whats_up_with_arbital/dqa0?context=3

[-]VAuroch9y100

If you can't succeed without first getting mass adoption, then you can't succeed. See the 'success' of Medium, and how it required losing everything they set out to do.

If Arbital has failed, Arbital has failed. Building neoTumblr and hoping to turn it into Arbital later won't make it fail any less, it will just produce neoTumblr.

1Douglas_Knight9y

Could you detail what you mean about Medium? This doesn't sound like a claim that is very easy to look up.

9[anonymous]9y

Somewhat recently Medium laid off about a third of their team as they tried to re-orient to figure out what happened when their original vision went away.

3Lumifer9y

Isn't this a pretty well-worn path by now? Start with lofty visions, discover that your incentive system rewards for eyeballs and clicks, start to optimize for eyeballs and clicks, become shit soon afterwards.

4[anonymous]9y

I do think we see this pattern across a lot of content platforms, like YouTube, as well. My impression is that earnest creators who try to make quality things often get out-competed by attention-hijackers / things that optimize for the action rather than the intent (ala Goodhart's Law).

-5Alexei9y

[-]Brendan Long9y130

As a software engineer, it seems strange to me that Arbital is trying to be an encyclopedia, debate system, and blogging site at the same time. What made you decide to put those features together in one piece of software?

[-]Alexei9y100

Eliezer said he wanted all of those features. (And he is using basically all of them.) But also we worked on it for 2 years, so a lot of features accumulated as we were trying different approaches.

1mako yass9y

Resources required for each application would tend to be useful to the others * The debate system settles disputes over encyclopedia maintenance * Monitoring activity in the encyclopedia can potentially feed the debate system with user qualification metrics for use in filtering comments * A sufficiently well structured debate effectively is an outline of an explanation * Systems for presenting articles and displaying structured debates are blog-complete: Blogs and articles are basically the same kind of information, and the debating stuff could presumably be simplified into a comment section.

[-]MugaSofer9y80

Will this "Arbital 2.0" be an entirely unrelated microblogging platform, or are you simply re-branding Arbital 1.0 to focus on the microblogging features?

5Alexei9y

I'm thinking of it as a completely new, unrelated platform. Whether or not it will live at arbital.com and be called Arbital is not yet decided, though currently I'm leaning towards yes. (But if, for example, some people are using the old Arbital, then it would probably be easier for me to put the new platform under a different domain & name.)

[-]hoghoghoghoghog9y70

Is there a description anywhere of the history of Arbital development?

Specifically I once heard a 4th-hand description of Arbital as "the Stacks Project, but for everything." It seemed to me at the time that a good step would be to 'simply' extend the Stacks project until it encompasses all of math[*]. I'd like to know which of Arbital's difficulties would still apply there.

5Alexei9y

We don't have Arbital history anywhere, although I guess there is the blog, which captures a fraction of it. "The Stacks Project, but for everything" was a decent description for part of our first approach. However, it's relatively easy for people to answer questions: doesn't take much time and you get instant credit. It's much harder to get people to write wiki pages / explanations: it takes a long time and you don't get that much credit.

[-]Martin-29y50

What were some specific ideas you had for "solving debates"? I was hoping Arbital would take the debate around a given topic and organize it into a tree. You start with an assertion that branches into supporting and opposing arguments, then those branch into rebuttals, then those branch into counter-rebuttals, etc.

7Alexei9y

That's one approach we ruled out pretty much from the start because that kind of structure is hard to read and laborious to create and maintain. However that mechanic on the blog level makes sense and that's basically how debates work right now in the wild. Our main approach was creating "claims". Blogs would reuse claims and the discussion around each claim. I'd say that part was actually moderately successful. One idea we played around with but didn't get to implement was allowing comments to easily leverage double-crux structure.

[-]seez9y50

This post seems like an overly brief and vague description compared to what I was hoping for and would guess the community would be interested in.

6Alexei9y

Hmm, I had that feeling too, but wasn't sure what else to add. I'm happy to answer specific/vague questions.

[-]Riothamus9y40

I see finding high-quality content producers was a problem; you reference math explanations specifically.

I notice that people are usually good at providing thorough and comprehensible explanations in only their chosen domains. That being said, people are interested in subjects beyond those they have mastered.

I wonder if it is possible to approach quality content producers with the question of what content they would like to passively consume, and then try and approach networks of content producers at once. For example: find a game theory explainer who want... (read more)

1Alexei9y

Hmm, I'm skeptical a barter system would work. I don't think I've seen a successful implementation of it anywhere, though I do hear about people trying. Yes, we've considered paying people, but that's not scalable. (A good 3-5 page explanation might take 10 hours to write.)

3Riothamus9y

I had not imagined a strict barter system or scaling of paid content; the objective in both cases is only to make up the difference between the value content producers want versus the value they expect for the first wave. The point of diminishing returns would be hard to judge for paid content, but perhaps the two strategies could work together: survey prospective content producers for the content they want to see, and then pay for the most popular subjects to draw the rest. Once you have enough content established to draw the first wave of voluntary content producers, everything else can build off of that for no/minimal further investment. That being said, it would probably be a good idea to keep surveying and perhaps paying for content on a case by case, say to alleviate a dry spell of contributions or if there is some particular thing which is in high demand but no one is volunteering to produce. What about a contest with a cash award of some kind? This could drive a lot of content for a fixed upfront investment, and then you would also have the ability to select among the entries for the appropriate style and nuance, which reduces the risk of getting unsatisfactory work.

[-]hamnox9y40

I expect that micro-blogging will be an excellent combination with the arbital-style of voting on things. I especially think that you could get very good results from voting on per-post 'related links' submitted by users. Tumblr has reblogs for responding to things, but those naturally become mediated by viralness instead of internal coherence.

I can imagine a neural-activation-like effect coming out of that, where frequently co-active posts naturally rise to the top of each other's links and become threads or topics.

Are you planning anything like that?

1Alexei9y

Not sure what you mean by this.

1hamnox9y

If a lot of people vote that two articles are linked, they become linked to each other. This will naturally form topic threading somewhere between between Wikipedia link diving and Tumblr reblogs.

[-]efenj9y40

Thank you for the summary of the state of Arbital!

It seems that while you haven't achieved your full goals, you have created a system that Eliezer is happy with, which is of non-zero value in itself (or, depending on what you think of MIRI, the AI alignment problem etc., of very large value).

It'd be interesting to work out why projects like Wikipedia and StackOveflow succeeded, while Arbital didn't, to such an extent. Unfortunately, I don't really have much of an idea how to answer my own question, so I'll be among those who want all the answers, but don't... (read more)

6ChristianKl9y

I had some experience being involved in Citizendium which failed. To me it feels like the key difference is openness to contribution. Both Wikipedia and StackOverflow make it easy for anybody to get involved. Eric recently shared a link to a post he wrote on Arbital on Facebook. I could either answer on Facebook or an Arbital. Given that Arbital didn't gave me a text field in which I could write my comment it was easier to answer on facebook and that's what I did. Arbital also might be too slow. It's a general pattern that page loading time matters a great deal and I'm not aware of any successful website that is that slow in 2017.

5Alexei9y

My guess for Wikipedia's success is that they were one of the first; and there was more of a sense of an online community back then. Also it's easier to create Wikipedia content than, say, a good explanation. StackOverflow succeeded because asking and answering questions is pretty easy, you get instant feedback, and they got community management right. (They solved exactly one problem well!) The founders were also really well known so it was easy for them to seed the platform. I can't open-source the platform as long as I'm doing the for-profit venture, since the platforms are too similar. However, if at some point I have to stop, then I'll be happy to open source everything at that point.

0efenj9y

Thanks for the fast reply! OTOH Eliezer is also quite well-known, at least in the relevant circles. For example, at my non-American university, almost everyone doing a technical subject, that I know, has heard of and usually read HPMoR (I didn't introduce them to it). Most don't agree with the MIRI view on AI risk (or don't care about it...), but are broadly on board with rationalist principles and definitely do agree that science needs fixing, which is all that you need to think that something like Arbital is a Good Idea. It's a bit of a shame that HPMoR was finished before Arbital was ready. I'm also not entirely sure about the comparison with Wikipedia, regarding ease of creating entries vs. writing explanations — in some cases, writing a logical explanation, deriving things from first (relevant) principles is easier than writing an encyclopaedic entry, having the appropriate citations (with Wikipedia policy encouraging secondary over primary sources). Writing things well is another challenge, but that's the case for both. The remaining arguments are probably sufficient, in themselves, though. That makes sense!

[-]ChristianKl9y30

Why do you not continue on "solving online debates" but try to shift to micro-blogging?

3Alexei9y

See my reply to michaelkeenan: http://lesswrong.com/r/discussion/lw/otq/whats_up_with_arbital/dq91

[-]JoshuaFox9y30

Eliezer is still writing AI Alignment content on it, ... MIRI ... adopt Arbital ...

How does Eliezer's work on Arbital relate to MIRI? Little is publicly visible of what is is doing in MIRI. Is he focusing on Arbital? What is the strategic purpose?

9Alexei9y

See MIRI's recent post under "Going Forward": https://intelligence.org/2017/03/28/2016-in-review/ Basically explaining why people still don't get AI safety is a very important task and Eliezer is particularly well suited for it.

[-]PhilGoetz9y20

This sounds great! There is no FAQ on the linked-to website, though. Is Arbital open-source? What are the key licensing terms? How's it implemented? How does voting work?

If we're all supposed to use the same website, there are advantages to that, but I would be less excited about that.

Also, the home page links to https://arbital.com/explore/math, but that page is blank. Er... https://arbital.com/explore/ai_alignment is also blank for me. Perhaps Arbital doesn't work for Chrome on Windows 7 without flash installed.

1Alexei9y

It's not open sourced. The pages might take a while to load (up to 30 seconds).

2gjm9y

I get: * First, what looks like a blank page with a navigation/search bar at the top. * This stays there for ~10s with absolutely no indication that more might be coming. * Then an extra rectangular thingy gets overlaid and my browser window stops responding to me. * This stays for maybe another 10s. * Finally, some actual content appears. (This is with Firefox on Windows, on a reasonably beefy machine.)

[-]ThoughtSpeed9y10

I had asked someone how I could contribute, and they said there was a waitlist or whatever. Like others have mentioned, I would recommend prioritizing maximal user involvement. Try to iterate quickly and get as many eyeballs on it as you can so you can see what works and what breaks. You can't control people.

LESSWRONG
LW

LESSWRONG
LW

41

What's up with Arbital?

41

41

So what now?