I think yes.
My understanding from the startup world is generally that different startups can have vastly different outcomes trying to solve the same problem at the same time with competent team members. Sometimes it can be a highly tactical implementation detail or marketing strategy that creates success where others saw failure.
In a hypothetical universe where the outcomes of startups are completely random, wouldn't we observe similar outcomes? I mean "two startups trying to solve the same problem, only a tiny difference in marketing strategy, one of them succeeded, the other failed".
There are other reasons that I think we're not in that world, among them serial entrepreneurs being much more successful than usual with such tactical successes.
I say mostly that you have competent people working on the right cause. You do also need to look at exactly what you are doing, but the reason having competent people working on the cause matters more is because finding the right thing to do is the hardest part. If it were obvious it would already be done, and mediocre execution on the right thing beats superlative execution on the wrong thing, it seems to me.
As an intuition pump, consider that successful startups usually pivot at some point and this is why investors prefer evaluating the team to the idea. A little more consideration of the same point reveals that the people are where the investment goes and how the capital is built in both the literal and gearsy senses.
How much of the impact of an organization is covered by it being ‘a group of size M working on X’, relative to the specifics of how and what they do in working on X? What if we also include a single scale of how functional they are?
For instance, does it mostly matter that my research group, AI Impacts, is a certain size force for AI risk related thinking (with the size determined by the number and competence of people and the functionality of the organization, say), or does it matter whether we write journal articles or news stories or blog posts or research pages, or whether we choose our projects collectively vs. individually, or whether we get most of our feedback internally vs. externally? Maybe most of these things can mostly be translated into ‘functionality’. But the ‘type of thing we are producing’ one doesn’t seem to as easily.
How much does what exactly you are producing matter? It could matter almost entirely or not at all, to my knowledge. For instance, I have some intuition that ‘there are about three small orgs in that space’ is a reasonable description of how much effort is going into a goal, but I also have intuitions that, say, minor differences in the responsiveness or navigability or style of a website can make the difference between it seeming great or annoying or crankish, and being read or not, or liked by different people. Which seems like it should just have a fairly different effect. These seem vaguely in conflict.
I originally meant for AI Impacts to be an unusual kind of entity, and the form of the output (website of ongoingly updated research pages on modular questions, hierarchically supporting one another) was a key part of the experiment. Which doesn’t imply a strong view on the importance of format—experimentation might make sense if format is so important it is worth searching alternative ones, but it might also make sense if it is so unimportant that it won’t mess anything up.
But there are costs to doing unusual things (e.g. people are confused about what you are doing, other entities aren’t organized to interface with you), so if format is totally unimportant, maybe we should move to more normal things. (Or the same goes if format is important, and this one is clearly worse.)
This question is also relevant in how you pick organizations to donate to, so maybe people who regularly do that know the answers. Does it mostly matter that you have competent people working on the right cause, or having checked that, do you generally also need to look closely at exactly what they are doing?