I believe that you would see a sizable bump in your effectiveness if you adopted customer development and lean startup practices.
These practices were developed precisely to combat the problem of investing too much time or effort in projects which are not wanted or needed.
In high level principle form, that looks something like this:
Thanks. This is definitely something I think we've struggled with, owing -- I think -- to lack of group cohesion, due to intentional decentralization. But we'll definitely have to keep this in mind to avoid wasting our time.
"A stronger community for the effective altruist movement should better encourage existing EAs to contribute more and better attract new people to consider becoming EA. By building the EA Community, we hope to indirectly improve recruitment and retention in the effective altruist movement, which in turn indirectly results in more total altruistic effort, in turn resulting in more reduced suffering and increased happiness."
I'm going to predict that .impact struggles to meet this objective.
I think you're taking a naive view of how movement building works.
I think you need to see the distinction between retaining and recruiting members as analogous to the tension between a core and casual fan base. In order to recruit new EAs, your pitch will almost definitely have to downplay certain areas that many core EAs spend lots of time thinking about. That way, you'll bring in a lot of new people that, for example, buy the argument that you should donate to the charity that provides the most bang for your buck and yet still, for example, have zero interest in AI or animals. If you refuse to alienate core EA member values in order to get more casual EAs (e.g. people that donate to GiveWell's top charities and give a bit more than average) then, well, that's admirable, I guess, but you're movement building won't go anywhere. There's a reason why for-profit organizations do this - it actually works.
The amount of people that share most EA values is going to remain low for a very long time. Increasing that number wouldn't involve "recruitment" as much as it would involve full-on conversion. As long as your goal is to increase that number, you're going to see very low recruitment rates. Most people aren't on the market shopping for new worldviews - but individual new beliefs or values, maybe. And if you won't agree with a worldview, you aren't going to join the community just because it's active.
If you want more "total altruistic effort," go convince people to show more altruistic effort. Trying to movement build a group as complex and alienating as EA by strengthening its internal ties will dissuade most outsiders from wanting to join you. Pre-existing communities can be scary things to self-identify with.
You know how some parents make their kids try cigarettes at a young age so that they'll hate it and then not want to smoke when they're older? Well, a website like Brian Tomasik's is like that for most potential EAs. Way too much, too soon.
In order to recruit new EAs, your pitch will almost definitely have to downplay certain areas that many core EAs spend lots of time thinking about.
I think "core" EAs understand and are comfortable with that, so they won't feel alienated.
-
As long as your goal is to increase that number [of core EAs], you're going to see very low recruitment rates.
Some of this comes down to what counts as an "EA". What kind of conversion do we need to do, and how much? I also think I'll be pretty unsuccessful at getting new core EAs, but what can I get? How hard is it? These are things I'd like to know, and things I believe would be valuable to know.
-
If you want more "total altruistic effort," go convince people to show more altruistic effort.
So you expect movement building / outreach to be a lot less successful than community building ("inreach", if you will)?
So you expect movement building / outreach to be a lot less successful than community building ("inreach", if you will)?
Yes, especially if the same strategies are expected to accomplish both. They're two very different tasks.
Some of this comes down to what counts as an "EA". What kind of conversion do we need to do, and how much? I also think I'll be pretty unsuccessful at getting new core EAs, but what can I get? How hard is it? These are things I'd like to know, and things I believe would be valuable to know.
I think you can convince people to give more of their money away, you can convince people to take the effectiveness of the charity into account, you can convince people to care more about animals or to stop eating meat, and possibly that there are technological risks that are greater than climate change and nuclear war. I don't think you'll convince the same person of all of these things. Rather they'll be individuals that are on board with specific parts and that may or may not identify with EA.
So you expect movement building / outreach to be a lot less successful than community building ("inreach", if you will)?
Yes, especially if the same strategies are expected to accomplish both. They're two very different tasks.
I wouldn't say the same tasks will work equally well for both. But I do think either would have spillover effects for the other. Right now, it seems we're focused on community building, though.
~
I think you can convince people to give more of their money away, you can convince people to take the effectiveness of the charity into account, you can convince people to care more about animals or to stop eating meat, and possibly that there are technological risks that are greater than climate change and nuclear war. I don't think you'll convince the same person of all of these things. Rather they'll be individuals that are on board with specific parts and that may or may not identify with EA.
I'd be interested in how much overlap there are between these groups. It never was my intention to try and convince people of the entire meme set at once, but I wouldn't rule it out as implausible. I think better understanding these channels (how people come to these beliefs) is most important.
If you're talking about recruiting new EAs, it sounds like you mean people that agree enough with the entire meme set that they identify as EAs. Have there been any polls on what percentage of self-identifying EAs hold which beliefs? It seems like the type of low-hanging fruit .impact could pick off. That poll would give you an idea of how common it is for EAs to believe only small portions of the meme set. I expect that people agree with the majority of the meme set before identifying as EA. I believe a lot more than most people and I only borderline identify as EA.
If you formulate a bunch of question we could add the poll question to the next LW census. In general formulatting an EA census might also be a worthwhile project.
We've just launched one, with Peter's help: http://lesswrong.com/lw/k60/2014_survey_of_effective_altruists/
There's a reason why for-profit organizations do this - it actually works.
It costs them huge advertising budgets and gets less effective as time goes by.
For-profit organization actually do this because they don't have a cause to rally around. Making more money for shareholders isn't giving anyone a feeling of community.
Steve Jobs got rid of focus groups telling him what the people want and build products to fit his own standards. As a result Apple has managed to develop a strong brand.
And if you won't agree with a worldview, you aren't going to join the community just because it's active.
But you might switch to self identifying yourself as EA because there are people on Skillshare doing nice things for you without asking for something in return.
That self identification will then improve the chances that you are doing other things to advance EA. It helps with retention.
I'm saying it helps with retention but barely at all with recruitment - and that it may even get in the way of recruitment of casual EAs. I don't think Skillshare favours will make people want to self-identify as EA. Only a minority of people even require the sorts of favours being offered.
The best way to recruit is to have people who are passionate enough about a subject that they tell their friends.
Yup - although in the case of EA, that's still likely to be a very slow process. This isn't the sort of thing that can go viral. It takes months or years of cultivating before someone transfers from complete outsider to core member.
Yup - although in the case of EA, that's still likely to be a very slow process.
I don't think that's a problem. We don't need to think in terms of corporate quarter results but are free to think more long-term.
Focusing on quick fixes isn't what successful movement building is about.
To clarify, I also don't think EA has much potential as a social movement even if marketed properly. Specific EA beliefs are much more spreadable memes down the line IMO.
As far as Skillshare goes, if you want to spend additional time I would suggest automatic posting to facebook.
Someone who puts up a new offer, might want that the offer also gets shown on his facebook wall. Someone who puts in a want might want to know that his facebook friends know about it.
While there are probably 3-6 people that seem promising possibilities for future regular contributors, they have not yet given their time. As a result, .impact has pivoted away from spending time on active recruiting efforts.
You could use the Skillshare wanted lift to highlite individual projects for whom you seek contributiors.
Thank you for putting so much time into spelling out your work and thought process !
Question: Did you try to assess whether converting existing software/platforms or joining/taking over existing online communities would be better (along the various metrics you care about) ? If so, what were your conclusions ?
Question: Did you try to assess whether converting existing software/platforms or joining/taking over existing online communities would be better (along the various metrics you care about)?
That's something we've never thought about. Taking over existing online communities sounds hard, though. This comment seems relevant.
Jointly written by Peter Hurford, Ozzie Gooen, Patrick Brinich-Langlois, and Ciarán Phillips
-
.impact is a group of people who want to volunteer on really useful projects. This means, however, we need some idea of whether or not our projects are actually useful. Thus, the purpose of this essay is to analyze whether or not our projects have been useful, whether putting more time and money into .impact is worth the cost, where .impact has stumbled, and what our plans for the future are.
.impact was launched with the goal "to provide infrastructure to get people and useful projects together" and "to help volunteers learn useful skills, meet great people, and create something substantial." Since .impact's inception in mid-October and as of this writing, 25 projects entered Active Development, of which 7 later left to inactive development, and 4 were completed. Additionally, we've had 9 projects enter the planning stage, of which 2 were later obsoleted. We've also had 25 more projects brainstormed but not yet pushed forward.
.impact is not one organization, but rather a collection of projects. Therefore, it makes sense to assess our impact based on the goals of each individual project.
Summary
The expected impact of this project is to connect and create sharing between Effective Altruists. So far the application has 55 users and 59 offers. The codebase has 498 commits, is well tested (97% coverage) and cleanly written (4.0 on code climate). Patrick found extending the app to allow users to request things to be much more difficult than he expected, which may mean that the code isn't well structured, or that he doesn't have much experience and is bad at predicting software schedules. Ozzie spent approximately 30 hours on the project, Patrick spent about 250 hours.
It's not obvious yet how much actual sharing has happened because of Skillshare. There were approximately 4-6 people who said that they had used the site successfully.
Size of Potential Impact
There are many potential ways that the project could become provide value. The main two are that (a) it could benefit the effective altruist community and (b) it could grow into a full company. We think that (b) is very unlikely. In considering (a), the project is fundamentally limited by the size of the effective altruist community. Right now that may be a few thousand individuals, but later it may grow.
It seems possible (assuming a fantastic application) that on average, each person in the community (if defined in a binary way) uses it once per year, on average, to successfully request one thing. Many of the requests involve meeting people, for mentorship or education. If the average meeting is worth $30-$100, the benefit of the project would be about $30-$100 times the number of effective altruists per year. Right now, if there are 1,000, then this could possibly create $30,000 - $100,000 of value per year to the community according to this estimate. So far we are quite far away from this (51 users).
As of 6 March 2014, 101 messages have been sent through the site. Each message sent through the site corresponds to one user requesting another user's offer (follow-up e-mails are sent directly between users and can't be tracked). A large proportion of the e-mails that were sent were test e-mails. Patrick Brinich-Langlois's 90% confidence interval is 18 to 70 real e-mails sent. (The metadata for these e-mails are deleted after thirty days, which is why we don't know the exact number.) Adding the ability to track use of the site is a priority, though it's been a priority for about three months, and nothing has been done about it.
The official launch of the site is about four months behind schedule. This isn't for any technical or pragmatic reason, but rather because it's really hard for us to get motivated to advertise promote our stuff.
Assessment
Patrick Brinich-Langlois probably put too much effort into this application given the current number of users, especially since use seems to have declined over the past two months (though use has steadily climbed back up after publicity). He also spent too much time working on relatively unimportant features, such as allowing users to log in without Facebook. It's unclear whether further, judicious investment in this project is worthwhile. Other projects may have more low-hanging fruit. Owing to inertia, Patrick will probably continue to work on the site unless somebody tells him to do something else.
The failure to have launched for more than a month (i.e., advertise the existence of this app on websites frequented by effective altruists) despite the application being useful and operational is strange, since launching seems pretty easy. Patrick probably should have taken responsibility for this part, but he will do anything to get out of doing anything else.
One big benefit of the project was that it motivated Patrick to finish his first Rails application. Before he started working on this, he was twiddling his thumbs, wondering if he'd have to make the world's elevnty gazillionth Twitter clone to learn Rails. But this project provided him a useful, fulfilling learning opportunity.
Summary
The expected impact of this project is to learn about the effectiveness of donating to Facebook ads for vegetarianism, to inform current donation efforts in the space and inform future earning to give efforts. If positive results are found, the superior methodology of the study and the independence from pro-veg groups should reassure donors and bring a lot more money toward a high-impact activity. If negative results are found, we also learn to avoid donating here and instead donate somewhere else, which is also important. So far, the project has taken 32 hours and no money.
Size of Potential Impact
Peter suspects there is a decent-sized risk that few people will be influenced by the study. Peter has been able to find four people not currently donating to vegetarian advocacy who say they would begin to donate if this study has positive results, suggesting around $80K annual donations could be moved to veg ads from GiveWell top charities. However, Peter has not yet found anyone who say they will move away from donating to vegetarian advocacy if the study has negative results.
Risk of Project Collapse
Right now progress is punctuated, but steady. The project seems to have survived Jason Ketola resigning from the project, with Peter instead collaborating with some researchers from the University of Texas. The next trying time for the study will be in the initial pilot test. There is a decent size risk that the project will be found unfeasible or much less useful than initially anticipated due to insufficient response rates on the survey. The need to pilot the study potentially several times combined with the need to wait 2-3 months for results also suggests that the timeline for the survey might be long, taking about a year to complete.
Summary
.meta is an individual project that involves the work done to keep infrastructure up to make sure all the other projects are well planned and coordinated. .meta is basically the project that manages the ".impact" brand as a whole. So far .meta has taken X hours (60 from Peter and Y from Ozzie) and $5.95/year for the dotimpact.im domain name. The potential impact from .meta is discussed throughout this analysis.
Summary
By making important effective altruist websites look better, we hope to make them more engaging and improve readership, thereby potentially influencing people more than they otherwise would have been.
Giving What We Can Redesign
Ciaran Phillips created a custom drupal module for part of the new Giving What We Can website (currently hosted here), and did some other work on implementing the new design across some pages. He estimates 35-40 hours for creation of the custom module. The other work Ciaran did on the site took an estimated 20 hours.
The project as a whole isn't really part of .impact, as work was already ongoing, but Ciaran's involvement came about through .impact (and he was originally contacted about this through Skillshare.im). The site is still not live, but this work has helped move forward the project quite a bit.
Centre for Effective Altruism Redesign
Ciaran Phillips also helped redesign the Centre for Effective Altruism home page, which recently finished. This project invovled identification of suitable wordpress themes, voting to choose one, installation and customization of the theme and other tasks - paypal, slider, image choice/optimization etc. Overall the project likely took between 40-50 hours of Ciaran's time (as well as some time from staff within CEA).
The general response suggests this redesign presents a more professional impression of CEA, though the actual impact of this probably can't be measured. One metric available is donation tracking, but given the lack of 'impulse donations' among the EA community, it's unlikely this would provide any useful information. Google analytics was not enabled on the old website.
Utilitarian Essays Redesign
Brian Tomasik asked .impact to help with redesigning his Utilitarian Essays page, though the redesign ended up being very simple, with minimal changes, and used suggestions mostly from people who were not involved with .impact.
Brian estimates that about 90 hours were put into creating and implementing the redesign -- 10 hours for discussions, 20 hours for implementation, and 50 hours for pictures. Implementation required a lot of grunt work involving fixing header tags and adding table of contents and summaries. Pictures involved a lot of learning about how-tos and legal aspects.
While web traffic does not show any clear boosts as a result of the redesign, Brian suspects that the improved readibility and credibility has a decent chance of affecting people over the future years, creating a high payoff, though this will be forever difficult to assess.
.impact helped retool Jeff Kaufman's r/smartgiving subreddit into an EA general subreddit, providing an alternate and potentially higher-quality venue for EA discussion (relative to the EA FB group). So far, we've added 43 new subscribers (780 -> 823) since the re-launch on 5 Feb 2014. Posting activity has increased by a factor of about five, though this represents the addition of only two more active posters. Commenting activity has moved from non-existent to present. Generally speaking, the subreddit has still not taken off. Marginal time spent on relaunching the subreddit from where it was has been estimated at five hours among all people. No money was spent.
.impact has several other projects in the works, but many of them have not yet developed enough for their impact to be clear.
Meetup Map
The goal of the meetup map is to inform people about meetups and therefore increase meetup attendance and EA networking. This project still is in development and not yet launched.
Job Board
The goal of the job board is to inform people about opportunities for employment in the effective altruist movement, increasing people's knowledge about careers. This project has not followed a minimal viable product model, however, instead opting for a more complex app in order to allow Peter to learn programming in Ruby/Rails. So far, Peter has spent 37.5 hours on this, Rob K. has spent about 10 hours on this, and Ozzie has spent $5.95/year on the domain name. The project is still in development and has not yet been launched, but it is very near completion. It's actual impact is still very unclear.
EA Research Base
The goal of the research base was to have a center to collaborate on and publicly host in-progress research. So far it has several documents. Most of these are only partially completed.
Effective Altruist Community Blog using the LW Codebase
The goal of the community blog is to adapt the LW codebase to create a high-quality community blog for effective altruists. Currently, this project seems to have steady progress. The project is still in development and has not yet been launched, so it's actual impact is still very unclear. Ryan Carey estimates the site will be ready by September 2014 (definitely later than May 2014, definitely earlier than March 2015).
EA Blog Compilation / Introduction
The goal of the EA Blog Compilation is to create a book resource that can bring people from "I've heard someone talk about effective altruism a few times" to "I am now able to participate in strategic career choice and cause selection". Currently, this project seems to have steady progress, with authors lined up and the table of contents written. The project is still in development and has not yet been launched, so it's actual impact is still very unclear.
EA Group Development
The goal of EA Group Development is to help people build up groups of EAs in their local communities, such as EA Netherlands, EA Ottawa, EA Vancouver, and EA Melbourne. Individual groups seem to have had steady progress, but the opportunities for collaboration still remain untapped.
Yearly EA Decision Survey
The goal of the Yearly EA Decision Survey is to gather potentially useful statistics on the effective altruist movement. Currently, this project seems to have steady progress. The project is still in development and has not yet been launched, so it's actual impact is still very unclear.
Expanding Google Grants
Google Grants offers many registered charities up to $120K of Google Ads per year for free. Yet many effective non-profits are not signed up. This makes it potentially important to (a) identify the Google Grant status of a lot of non-profits and (b) assist non-profits in setting up Google Grants. The goal of this project is to make that happen. So far, this is just it's initial stages, so it's actual impact is still very unclear.
Improving EA Presence on Wikipedia
Wikipedia is the third most visited website, behind Facebook and Google. EA content here could potentially reach a lot of people in an authoritative manner. Right now, several articles have been written on EA organizations, ideas, and people. Vigilance and upkeep have also prevented some of these articles from being deleted.
EA Interview Series
By interviewing EAs, we hope to provide a relatable way of explaining EA to those people who are less familiar, as well as providing stories to inspire existing EAs to do more than they otherwise would. Anecdotal evidence right now points to some inspirational value. Right now with only about 30 hours invested in this project, even influencing one person to switch from donating 10% to 20% would be an amazing return on investment, and this seems like a plausible outcome based on existing anecdotes. However, more rigorous evidence seems like it would be really hard to come by.
While .impact is a collection of projects, we often are more than the sum of our parts. Therefore, it also makes sense to assess ourselves holistically. While the bottom line for .impact is ultimately reducing suffering and maximizing happiness as much as possible, we see ourselves as providing that in five key ways:
Why is this important?
A stronger community for the effective altruist movement should better encourage existing EAs to contribute more and better attract new people to consider becoming EA. By building the EA Community, we hope to indirectly improve recruitment and retention in the effective altruist movement, which in turn indirectly results in more total altruistic effort, in turn resulting in more reduced suffering and increased happiness. Additionally, by improving the community, we hope to improve networking, which could allow people to more easily find collaborators on important projects. Overall, movement building could potentially be a powerful and potentially underfunded path to impact.
What have we done to accomplish this goal?
We've obviously built a small community around .impact that did not previously exist, so we have at least minimal impact here. Through r/smartgiving and Skillshare, we've begun to make some progress on giving EAs more outlets to gather than just the Facebook group, though we have not yet had much payoff for any of this yet. We look forward to reassessing where we stand on this in the future, after more projects are launched.
Why is this important?
Simply put, a community that can do more is a more powerful community. By building infrastructure, we aim to make it easier for people to do things in our community, thereby improving their existing altruistic effort, in turn resulting in more reduced suffering and increased happiness.
What have we done to accomplish this goal?
Much of the “infrastructure” so far has been infrastructure of .impact itself. This included much of the work done by .meta, the project groups focused on enhancing .impact. .impact has also encouraged transparency, which is both good as a terminal value and for improving information available to the EA community about what is being done and by who, hopefully lowering costs to engaging in activities and lowering the risk of duplicate work. We also expect Skillshare to provide useful infrastructure when more widely promoted, which we have just begun to do so.
I'm not sure if this is the correct section to mention this, but the thing I personally find most valuable about .impact is having a single place that I can go to be updated about what is going on in terms of infrastructure building in the wider EA community. I think .impact is the best place in the world right now for tracking the success of diverse EA projects, and I find that valuable. — Niel Bowerman
Why is this important?
Many people in the EA movement want to do more, but don't know how to get involved. This is a massively untapped resource. It would be good to learn more about how to tap into volunteerism, so we can better point people's spare time toward altruistic effort, in turn resulting in more reduced suffering and increased happiness.
What have we done to accomplish this goal?
.impact has learned a fair amount from talking to different people about the projects they are doing and by trying to interest people in getting more involved. In particular, we've come to believe that it is much more difficult to make good use of time commitments under a certain threshold. The vast majority of the work so far has come from the top 4-6 members. Many of the other contributions have been fairly minor. Most of the signups on our web form indicate that people are willing to put less than 6 hours of work per week into projects; our existing experience shows that in these cases, very little work gets done. We discuss what precisely we learned in "What setbacks has .impact run into?" below.
Why is this important?
People with more skills and more credibility are able to go further. If all goes to plan, a person who works on a .impact project won't just be creating the project, but will also be learning from the project and will be able to leverage the project as a demonstration of their skills. This could lead to better careers and more talent in the EA movement, thereby making existing altruistic effort go further, in turn resulting in more reduced suffering and increased happiness.
What have we done to accomplish this goal?
Several people have used .impact as a platform to improve their own web development skills. Much of the recent work on Github from Peter Hurford, Patrick Brinich-Langlois, Ciaran Phillips, and Ozzie Gooen were from .impact projects. The four of us have also networked together, sharing our knowledge and resources to hopefully mutually improve our chances of landing jobs. Many of us intend to use these projects as part of our portfolio for applying to web development jobs, but we have not sent in applications yet.
Why is this important?
Even if people don't coordinate through .impact, it's important to send a message to the effective altruist movement that making projects is possible, failure is not something that should be scary, and we all stand to gain significantly if only we were to try more things and see how they worked. A hopeful side-effect of .impact is to spread this message, encouraging people to consider more projects, thereby increasing altruistic effort, in turn resulting in more reduced suffering and increased happiness.
What have we done to accomplish this goal?
Since the creation of .impact, we anecdotally have observed an uptick in the amount of people thinking through and doing projects, even if not running those projects through .impact itself. While it's very hard to say what would have been different without .impact, it seems possible that the recent increase in activity in things like infographics, Niel suggesting a wedding donation registry project, or some collaborations with FHI, all would have not happened to the same degree if not for .impact creating an atmosphere where projects are seen as possible. There's a lot of reason to be skeptical of this tenuous claim, however, and we hope to continue to monitor this. There are no clear victories on this goal yet.
The counterfactual on my case was that I first offered it to Giving What We Can, and they declined, and so I then offered it to .impact and it was analysed in enough detail to see that it wasn't as high priority as other useful things we could have been doing. That was useful analysis. I would not have offered it to .impact if .impact hadn't existed, but I would have still offered it to GWWC. — Niel Bowerman
No organization progresses perfectly without making some mistakes. Here are some of our stumbling blocks and lessons learned.
When .impact began, there was an implicit expectation that the amount of free time people had available for EA projects was moderately plentiful, and we could put together a good group of maybe a dozen people or so. Instead, we've learned that volunteering is not as plentiful as we had hoped and .impact has failed to add any regular contributors (>3 hours each week) from the initial four at the beginning (Peter, Ozzie, Patrick, Ciarán) despite several attempts at recruiting, though perhaps Ryan Carey or Pablo Stafforini could also be seen as regular contributors, despite operating more independently.
While there are probably 3-6 people that seem promising possibilities for future regular contributors, they have not yet given their time. As a result, .impact has pivoted away from spending time on active recruiting efforts.
We do have some people who could contribute very small amounts of time, but we have not really had small tasks available for them to do, though perhaps a greater build-up and publicity of the EA Wiki could help this problem. Generally speaking, .impact's to-do list is larger than we have people available to do things, and the demand for volunteering exceeds the supply of volunteerism.
Interestingly, so far it seems like it has been much easier to find people to work on challenging technical tasks than more basic ones. We have had significant website development work, but relatively little promotion, data gathering, and writing (outside of .meta).
So far, it has taken a substantial amount of time to make much progress on anything. Approximately 400 hours total have gone into the web applications so far, with relatively little user activity to show for it, though this is probably because significantly less time has been put into launching and promoting the applications. Maintenance has been a significant hurdle; not only do applications have to be created, but they also require substantial work to keep from breaking.
One problem with knowing if we are effectively using our time is it is very difficult to estimate the value of our projects.
Needs Analysis
Right now, .impact is a collection of projects that people have wanted to work on. While we did encourage some prioritization through having people vote up projects they thought most beneficial, there has been no formal needs analysis done to prioritize projects. Some of this is a good thing -- people work better on projects they enjoy. Furthermore, .impact should have a lot of elements of being a collection of what individual people find most important rather than imposing a prioritization scheme upon people. However, it would be beneficial if our projects could be informed by more prioritization work.
Metric Measurement
Assessing the impact of our projects is difficult, but we can work through some things to come up with good metrics for our projects. We just haven't put the time into doing this yet.
Throughout the first couple months of .impact, we spent a significant amount of time in meetings. For many people in .impact, more time was spent in meetings than outside of meetings. Therefore, we started out by slimming down meeting time considerably separating out into separate meetings by function (project updates, brainstorming, recruitment), and then later switched to bi-weekly meetings. There's a risk this could lead to less motivation on projects (because you're no longer focused on making weekly progress for weekly meetings), but hopefully it will give people more time for .impact.
We originally intended that volunteers produce useful software while at the same time developing useful skills. In practice these two goals have been quite separate. Often the most practical solutions involve relatively simple technology. However, volunteers seem most inclined to work on projects that would be technologically interesting. It seems very difficult to optimize both for learning programming and making useful projects at the same time, and it would be better for us to try to do both separately.
Case: Skillshare.im
A very substantial codebase was written for Skillshare.im. The code has 97% test coverage and a 4.0 on code climate for code quality. Several interesting features were added, including a custom Twitter feed for new requests. Skillshare.im also received a somewhat novel design. However, it had relatively low growth, and it's technical infrastructure and design made it substantially more difficult to modify. Whenever new features were added or changed, tests broke and had to be repaired, and the design needed special maintenance. After some discussion, it seems like the project was well designed to show to future prospective employers, but not optimized well for users. Later the website was ready for a public “launch” but this delayed several weeks for a lack of time (this was Ozzie's responsibility).
Case: Job Board
As stated above, development on the Job Board has mostly been geared to learning how to use Ruby on Rails. Features included automated scripting from 80,000 Hours' job listings and user registration. An alternative would have been to make a much more basic application without users with a simple table and do the inputing of jobs manually. An even simpler and easier alternative would have been to create a Hackpad document or similar with the listings of jobs. Both of these alternatives would have been much better uses of time for the purpose of giving the effective altruist community a workable listing of all of the relevant jobs. However, they weren't as technically interesting, and thus were not pursued.
Case: Javascript
A few members wanted to do work in JavaScript frameworks (Backbone.js, Ember.js, Angular.js). These are now very popular among employers. We didn't have any obvious projects that could use JavaScript frameworks at the time, so may have lost additional work. We may consider doing projects using JavaScript in the future, but it's not obvious what these will be. It's currently assumed that JavaScript front-ends are not essential to many of the ideas we currently have. Therefore they may be poor for prototyping and long-term value.
.impact has a lot of people with very similar skillsets and experiences -- young, white, male beginning coders. We have a lot of room to grow in diversifying on gender, race, skills, and experiences. This problem isn't confined to .impact (the same is true of both programming and effective altruism), but .impact seems affected to a larger extent. This is something for us to be wary of, though we're not sure how to make progress on this issue given past failures to recruit people.
Neither Ozzie and Peter (the main “project managers”, for lack of a better term) have too much experience in volunteer management and we're both still learning how to run an organization. Overall, we don't really know what to expect in running .impact and we strongly suspect that more experience would make us much more effective. We try a lot to seek the advice of more experienced people, but it can only go so far. The rest needs to be navigated via trial and error.
A key question for ourselves is whether we should keep putting more effort into .impact, or whether we should do something else instead.
How much money does it take to run .impact?
Monetary costs in .impact are very low. Current total spending on all .impact projects is less than $50.
How much time does it take to run .impact?
However, .impact involves more than just money -- it also involves the time of the people involved with the project. Time spent here is time not spent elsewhere, and this presents a clear opportunity cost. We estimate our time spent breaks down roughly as follows:
...on the following projects:
In total, we spent about 860 person hours on .impact to date. If you value our time at $40 an hour, this works out to a total cost of running .impact as $34.4K to date.
What else could we have done with this time and money?
Peter: "The time I spent working on .impact would probably be instead spent on direct volunteering for other EA organizations (adding work for the Centre for Effective Altruism and The Life You Can Save, in addition to the work for Animal Charity Evaluators I already do) and more focus learning computer programming to boost my employability. It seems plausible but unlikely the work I'd do for CEA / TLYCS would be higher impact (given considerations of skill and replaceability) than trying out .impact. It's also quite plausible that it would be better for me to focus more on learning computer programming than more marginal .impact work (and I have shifted some in that direction), but the learning value from .impact has been immense for me."
Ozzie: "The most obvious thing is that I could have spent more time improving working for 80,000 Hours, where I was employed at the time. This could have meant an extra 10-30% speed or functionality, or less bugs. I could have also attempted to get some extra consulting work, or worked a lot more on my blog. I could definitely have learned more about computer programming if I would have done other projects which would have been more specialized. I think this was marginally beneficial to me in terms of career capital, though I did enjoy making friends and meeting people. I would thus expect most of the benefit of my work to be things that occurred to other people or the community. While at the time the benefit seemed expected to be greater than earning to give, I'm not sure about this at this point."
Patrick: "The amount of money spent on the Skillshare app is not worth remarking on. I spent about 50–100 hours on the app during my work hours, and the opportunity cost of this time was low. Although the project was a good learning experience, I probably would have learned at least as much if I had spent some of the time working on other .impact apps, and my impact would probably have been greater."
Ciarán: "The time/effort I've spent on .impact, would most likely have been otherwise spent on personal projects for career development. I did gain some career capital from these projects (how much may depend on the companies I apply for in my current job search), though I would have learned more directly useful skills had I been putting the time into my own projects. This was also my first opportunity to engage with other EAs beyond facebook discussions, and I've gained a better understanding of the community as well a stronger commitment to EA ideas; I feel that due to this I'll be contributing more in the future than I would have otherwise."
Obviously a lot could be done with 860 hours of highly motivated EA time. If all our efforts were channeled into earning to give and donated to GiveWell top charities (a typical comparison for EA effort), this effort could have saved 13 lives in expectation. Though, notably, a lot of .impact time was free time that could not have been easily converted into earning-to-give work, and the testimonials above show that the time may not have been spent otherwise on high-impact activities.
The impact of .impact is very hard to calculate at the moment. Admittedly, much of our self-analysis is not very rigorous, and a large majority of our impact remains in our yet-to-be-realized potential. Our core argument mainly rests in taking on causes that seem very high impact in expectation (growing the EA movement), and doing our best to progress on them in a transparent way, with a lot of learning value.
It does seems decently possible that our impact to date is not equal to or greater than an equivalent donation to GiveWell. However, it's also decently possible that our impact is many times greater than an equivalent donation to GiveWell. Overall, we remain decently confident (~70% sure) that the impact we've stated above is better than or equal to having donating to GiveWell instead.