I listened to the 80,000 Hours podcast with Tom Kalil, who spent 16 years as Deputy Director of the Office of Science and Technology Policy at the White House. Kalil seems skilled at evaluating concrete scientific plans to offer the president and finding the path of least resistance through government to effect those plans, though he is not himself someone with deep technical understanding of any single domain.

One key idea I took from the podcast was that his main use of the executive branch of government is as a coordination mechanism. I moved away from thinking of the President as an expert who makes decisions like a CEO, and much more as an individual with immense coordination power trying his best to take any concrete plans given to him and coordinate the country around executing on them. That is, not someone who comes up with plans, not someone who executes on the plans, but someone who coordinates people to execute the concrete plans that are waiting to be picked up and run with.

Below are relevant and very interesting quotes, followed by a few more updates I made listening to the podcast.

Key Quotes

Robert Wiblin: So, do you think people under-appreciate how much the executive branch can just do autonomously?
Tom Kalil: Yes. Yeah. Not only what it can do, but the President’s ability to convene.
[...]
Tom Kalil: One thing that I used to ask people is to imagine that you have a 15 minute meeting with the President in the Oval Office and he says, “Rob, if you give me a good idea for”, pick your cause, “reducing existential risk, then I will call anyone on the planet. It can be a conference call so there can be more than one person on the line. If it’s someone from inside the government, that I can direct them to something because I’m their boss, and if it’s someone outside the government then I can challenge them to do something. So, you not only have to tell me, what is your idea, but in order to make your idea happen, who would I call and what would I ask them to do?”
Tom Kalil: There are several reasons for this thought experiment. One is that if you work for the President you have the ability to send the President a decision memo and have him check the box that says yes. Over time that give you a sense of what psychologists call agency, a sense that many things that you see in the world around you are the result of human action or inaction, as opposed to the laws of physics. That’s one thing, a more expansive view of what do you think is potentially changeable. The second is, it’s sort of a version of the Hamming question, presumably if you really did have a meeting with the President you’d use it to describe an issue that you thought was really important as opposed to a secondary or third tier issue. The third is that many complex problems cannot be solved by a single individual organization, they require coalitions.
Tom Kalil: You can’t build a coalition if you can’t articulate, number one, who are the members of the coalition, and number two, what are the mutually reinforcing steps that you would want them to take. That’s one thing that I talk about in the Policy Entrepreneurship, then I also talk about something that people don’t ever really appreciate, which is that policy makers do things with words. What do I mean by that? Well, think about when the priest says, “I now pronounce you man and wife”, he has changed the state of affairs by virtue of, A, him being a priest, and B, him saying, “I now pronounce you man and wife”. Similarly, the way that a policy maker both frames and makes a decision and implements that decision is through documents. When the President does it we call it an executive order or presidential memorandum, when a regulatory agency does it we call it a rule, when the Congress does it we call it legislation. But in all instances it is a document that you are creating or editing, so part of the policy process is that you are able to figure out what’s the document or documents that you need to create or edit and who is allowed to take that something from being a Word document that is on your screen to something that has some force in the world?
Tom Kalil: I would see this all the time, something would go from being a Word document on my computer to being a presidential executive order, it always seemed like this slightly magical transformation from a Word Doc to something that is instructing relevant members of the Cabinet to take some action.
Robert Wiblin: I guess this makes you more ambitious, then you’re like, “What is the best thing, what is the best memo that I could write.
Tom Kalil: Yeah, exactly, yeah. But also you have to be able to articulate some coherent relationship between ends and means. I would … a lot of times someone would come visit me and they would say, “my issue is important”.
Tom Kalil: I’d say, “great, let’s say that I’m prepared to stipulate that, what is that you want me to do?”, then they would look at me and they would say, “Well, you should make this a priority”.
Tom Kalil: I’d say, “What would that look like?” People were not able … they were able to tell you that their issue was important and that they thought the President should devote more time and energy to it, but then when you said “Alright, what is it, literally … let’s say we got the President super interested in this issue, what would they do?” They weren’t able to articulate that part.
Robert Wiblin: Yeah, I’m sympathetic to that ’cause there’s a lot of things that I think are very important, but I’m also not sure what should be done. I suppose maybe it makes sense for people to think more about that once they’ve gotten people to care about it, but at the same time, maybe it’s hard to get people to care about something if you have no actual concrete steps that they can take, they’re like, “Well, I don’t know what to do”.
Tom Kalil: Yes. Yeah, because that’s assuming that … because what you’re saying is, I’ve really thought about this issue a lot and I think it’s really important, but I don’t know what to do.”
Robert Wiblin: That’s a bad sign. So you should think about it.

Kalil also talks about in his role as Deputy Director of the Office of Science and Technology Policy, he helped raise the staff count from 40 to over 100 during the Obama administration. He just gets to hire people who are excited about an idea and want to make it happen, and then they make it happen using the coordination power of the executive office. Here's a prominent example:

Tom Kalil: Let me give you one example. A young woman emailed me and the subject line of your email was, “Cass Sunstein says I should work for you.”
Robert Wiblin: That’s a strong subject line.
Tom Kalil: Good subject line. So I did a little research on her. It turned out that she had been a child violin prodigy with Itzhak Perlman, had won the major Yale undergraduate awards, was a Rhodes scholar, and was wrapping up a post-doc at Stanford in Decision Neuroscience. I went out on a limb and I decided to take a chance on her. Her name was Maya Shankar. I asked Maya, “What do you want to do?”
Tom Kalil: She said, “The UK has created this organization called the Behavioral Insights Team, which is taking these insights from people like Kahneman and Tversky and Sunstein and Thaler and using them to inform policies and programs. These are all US researchers. Why don’t we have something like this?” She said, “I would like to create that.”
Tom Kalil: Sure enough, in her late twenties, she arrived with no money, created this new organization called the Social and Behavioral Sciences Team, recruited 20 behavioral scientists to the federal government, got them to launch 60 collaborations with federal departments and agencies and got President Obama to sign an executive order institutionalizing this new entity.
Tom Kalil: I think that’s pretty consequential for someone in their late twenties to be able to accomplish. That’s one thing I did, was to recruit people of that caliber and teach them how to get things done in the federal government because the government doesn’t come with an operating manual.

The next quote is about how the core goal of the office of science and technology policy is to take the necessary steps to get the private sector to build new tech:

Tom Kalil: One of the things I learned is that if the United States is behind in a technology, it’s very difficult to try to re-establish a leadership position… We tried to do that in the area of technologies like flat panel displays and we invested some money, but I don’t think a whole lot came out of it.
Tom Kalil: Once Korea and Japan dominated the market for things like active matrix, liquid crystal displays, then trying to get the United States back into that market is really, really hard, and might require more money than the US is willing to put into it. Because obviously we believe that the primary role of government is to create the right environment for the private sector. It’s not to engage in this sort of heavy handed-top down industrial policy that you see a China engaging in, for example… we invested in this idea of flexible electronics where the idea is – maybe you have a display that’s a piece of paper that you can roll up and put into your pocket. And, if that’s an area where no one has established a clear leadership position, that’s more likely to be effective than saying, okay, we’re going to duke it out in some market that we’ve kind of already lost.

This final quote is an example of the coordination power of the President.

Robert Wiblin: Having worked in White House for 16 years, you must have some interesting or funny stories from your experiences there, different perhaps than what people expect? Can you share one of them?
Tom Kalil: Sure. This is a story that happened in 1995 and 1996, so as I mentioned, Vice President Gore was really interested in this idea of the information superhighway and one of his goals was, what if we could connect every classroom to the internet? So I would tell people about the vice president’s interest in this issue and someone who I’d gotten to know, John Gage, who was at a company by the name of Sun Microsystems said, “Oh, I’ve got this idea called Net Day. The idea is, what if on a single day, tens of thousands of engineers showed up in schools all across California and started the process of wiring California classrooms to the internet?” I said, “Well great.” He said, “You know, I’ve got a web page of what this would look like if it actually happened.” So he emailed it to me and I gave it to the vice president and the vice president thought it was a done deal.
Tom Kalil: So at this point, it was just in the fevered imagination of John Gage. So the vice president has a weekly lunch with the president and so he said, “Mr. President, we have Sun, we have Apple, we have HP, we have IBM, we have Pacific Gas and Electric and they have all agreed that they’re going to wire thousands of classrooms and schools all across California.” The president was like, “Great. Let’s announce it.” So it turned out that they were going to be in the Bay Area anyway so they decided, we’re going to announce Net Day. So I called up John Gage and I said, “They’re going to come out and announce this.” So he and I spent the next week calling in every favor that we had to get these CEOs to show up and announce that they were for us. They were a little sketchy on the details of what it was.
Tom Kalil: What John did was he developed a website, which was a clickable map of California, that allowed you to zoom all the way down to the street level, all 12,000 public and private K through 12 schools had their own homepage. You could indicate your level of expertise from, “I am an experienced network engineer.” To, “I will bring coffee and donuts.” All the schools were color-coded red, yellow and green depending on how many volunteers had signed up. So we could look at the map and figure out which communities were getting onboard and which needed some positive reinforcement.
Tom Kalil: So they announced that not only were they supporting it but they were going to come back and personally participate in it. So by the time they did, we actually had tens of thousands of people who had volunteered, so it was this positive, self-fulfilling prophecy because they said, “Oh, there’s going to be a Net Day.” In fact, there was a Net Day and tens of thousands of engineers showed up to wire the schools and many parents showed up to wire the schools, but they discovered the windows were broken and the bathrooms didn’t work, so a lot of them got more engaged in the schools as a result.
Tom Kalil: Many states decided they were going to do this and entire countries decided that they were going to have a Net Day as well. So it was this experience, a couple of things that I took away from it, one is that you could create this positive self-fulfilling prophecy, even though that was a very nerve-wracking period of time for me personally because I’d committed to the president and vice president to do something-
Robert Wiblin: To announce the thing.
Tom Kalil: … and announce something that didn’t really exist yet. Right?
Robert Wiblin: Yeah, sounds a little bit like an episode of Veep.
Tom Kalil: Then it was sort of applying massive parallels to this problem. So as opposed to saying, “How are we going to wire 10,000 schools?” The question was, how could you get every community to take responsibility for one school? So it was just very interesting of the experience that I had of going from something being a complete fantasy to actually seeing it happen.
Robert Wiblin: So government can get things done.
Tom Kalil: Yes.
Robert Wiblin: Just in sometimes a peculiar manner.
Tom Kalil: Yes, exactly.

There is a lot more fascinating discussion in the interview, especially Kalil's comments on using financial prizes to incentivise science+tech in areas like education and poverty.

Updates

My new model is that the President's interaction with science is largely to take concrete ideas floating around in the environment that are ready for their time, and push them over the edge into actually being built by the US private sector, or into actually substantially informing government policy. This is similar to the notion that scientific ideas come about when the environment is ready for them (Newton and Leibniz both discovering calculus at the same time). There are executable plans floating around in the ether, and the President keeps getting handed them and sets them off. His department is not an originator of new ideas, it coordinates the execution of existing ones. (And there's a natural frame from which is this is the correct marginal use of attention from the President: compare 15 minutes per project versus spending a week becoming an expert in one and then executing it himself.)

I’ve updated positively on the tractability of gaining influence within the government and being able to use it on timescales of 4-8 years. (I expect I will likely make a further update when I read the blogposts of Dominic Cummings regarding UK politics, though not sure how strongly.) Overall I think influence in government, if you’re ambitious and well-connected and have a very concrete vision, is likely quite a real action one can take. I expect that from the perspective of government there is a lot of low hanging fruit to be picked.

I updated negatively on the usefulness of interacting with this part of government in the short-to-medium term. My sense is that the state of understanding of how transformative AI will be built and what impact it will have on the world is sufficiently low resolution and confused that we have no project or policy recommendations for the government, and will not be able to offer anything until we see further work that helps conceptualise this space. Listening to the podcast tells me that if you get 15 minutes to talk to the President about x-risk today, you are wasting his time, because we have no concrete plan that needs executing if only could coordinate major AI tech companies. We have no R&D projects that need funding. We have no nuanced AI-development policies for global powers to agree to. I’m pretty sure that there are people in this community who can coordinate Elon Musk and Demis Hassabis or whomever else, should we have an actionable plan, but the current state is that we have no plan to offer.

New Comment
13 comments, sorted by Click to highlight new comments since: Today at 10:53 AM
My new model is that the President's interaction with science is largely to take concrete ideas floating around in the environment that are ready for their time, and push them over the edge into actually being built by the US private sector, or into actually substantially informing government policy. This is similar to the notion that scientific ideas come about when the environment is ready for them (Newton and Leibniz both discovering calculus at the same time).

Reminded me of this:

“Only a crisis - actual or perceived - produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around. That, I believe, is our basic function: to develop alternatives to existing policies, to keep them alive and available until the politically impossible becomes the politically inevitable." (Milton Friedman)

Yeah this is a really great point, and I should've incorporated it into the OP. Thx :)

I'd like to coin a new term for that thing which the US President has a lot of: coordination capital.

This seems to require some combination of:

  • trust
  • long-term stability
  • Schelling point-y-ness
  • personal connections

____________________________________________________________________________________________

Some properties

  • Coordination capital is depreciated as it is used

Consider the priest Kalil mentions. He's able to declare people married because people think he is. It's the equilibrium, and everyone benefits from maintaining it. But if he tests his powers and start declaring strange marriages not endorsed by the local social norm, the equilibrium might shift. Similarly, if the president tries to rally companies around a stag hunt, but does so poorly and some choose rabbit, they're all more likley to choose rabbit in future.

  • There are returns-to-scale to coordination capital

The more plan executions you successfully coordinate, the more willing future projects will be to approach you with their plans.

  • There is an upper bound to the amount of coordination capital

If you have a Schelling coordination point, and someone finds it bad and declares they will build a new, better coordination point, there is risk that you'll end up not with two but with zero coordination points. Similarly, coordination capital is scarce and it can result in lock-in scenarios if held by the wrong entities.

____________________________________________________________________________________________

Background and implications

Part of the reason I want a term for this thing is that I've been experiencing a lack of this thing when working on coordination infrastructure for the EA and x-risk communities. I'm trying to build a forecasting platform and community to (among other things) build common knowledge of some timelines considerations, to coordinate around them.

However, to get people to use it, I can't just call up Holden Karnofsky, Nick Bostrom, and Nate Soares in order to kickstart the thing and make it a de facto Schelling point. Rather, I have to do some amount of "hustling", and things that don't scale -- finding people in the community with natural interest in stuff, reaching out to them personally, putting in legwork here and there to keep discussions going and add a missing piece to a quantitative model... and try to do this enough to hit some kind of escape velocity.

I don't have enough coordination capital, so I try to compensate by other means. Another example is Uber -- they're trying to move riders and drivers to a new equilibrium, they didn't have much coordination capital initially, and this requires them to burn a lot cash/free energy.

Writing this I'm a bit worried that all the leaders of the EA /x-risk communities are leaders of particular organizations with an object-level mission. They're primarily incentivised to achieve the organisation's mission, and there is no one who, like the president, simply serves to coordinate the community around the execution of plans. This suggests this function might be underutilised on the margin.

My sense is that the state of understanding of how transformative AI will be built and what impact it will have on the world is sufficiently low resolution and confused that we have no project or policy recommendations for the government, and will not be able to do so until we see further work that helps conceptualise this space.

To point to the type of thing I'm thinking about, here's a bunch of work that seems centrally useful to me (list is not exhaustive):

  • Bostrom (1, 2, 3, 4, 5, 6)
  • Yudkowsky (1, 2, 3, 4, 5, 6, 7, 8, 9)
  • Christiano (1, 2, 3, 4)
  • Cotton-Barratt (1)
  • Hanson (1)

I'm interested to see if CSET, GovAI, OpenPhil, AI Impacts, or OpenAI are able to produce anything that helps conceptualise the strategy space (of the above teams that have produced public output, I've mostly not read it yet).

I highly recommend this piece by Dominic Cummings on how government works in practice. He is certainly optimistic that it could accomplish a lot, but my interpretation of the Kalil quotes is pretty much the opposite of yours. I'll listen to the podcast to get more context.

Thx! And what’s your impression - that the White House mostly focuses on innovation and execution of plans rather than coordinating other actors?

I largely agree with Kalil's assertion that the White House is mostly about coordinating other actors.

I think it is mainly a terrible failure in this regard, chiefly because it fails to account for the fact that coordinating other actors is a plan that requires execution. The 15-minutes-of-attention standard only works for cases where everyone doing what they would normally do except under the same banner this time is the right answer.

So what do we do when we need something that is currently wrong to change? What about cases where it's a difficult challenge that has an exacting standard for success? These are cases where the normal things other actors do is either specifically wrong, or not good enough; how is the White House supposed to coordinate them against themselves with a press conference and an executive order?

So while the coordinating-actors and dividing-attention frames are useful, both for us and from within the White House (or other country's leadership), I also feel like they could easily do a much better job approaching coordination as a strategy that requires execution, and chunking their attention to that end.

Before asking how something happened, we should ask if it happened.

nods FWIW I think it’s plausible I exaggerated the levels of competence in government and could be persuaded to edit wording; my main intention with the post was to discuss what type signature the top of government in fact has.

The White House spends the vast majority of its resources putting out false press releases. My impression is that that's what Kalil did, too. Probably he shifted things in a positive direction, but the shape of the marginal effort doesn't have much to do with the shape of the total effort. That is, how much time he spent shaping the CDC actions vs NIH funding vs conferences of outsiders doesn't tells us much about how much of his useful actions fell in those categories. He had practically no direct power, so in a sense the CDC and NIH were outsiders to be coordinated, too.

Cummings burnt a lot of bridges by saying important negative things. I'm suspicious of Kalil sounding so positive. The first hour of the podcast gave me an extremely negative view of him, but then he mentioned a lot of trade-offs and strategies that seemed valuable regardless of the average level of government function. Still, I worry that he sold his soul to function in this environment and lost the ability to tell good projects from bad.

This matches the pattern for at least a few high-profile American technology successes, e.g. Apollo and the Manhattan Project.

I note that Kalil did not speak to results per se, but rather considered the mark of success being a lot of energy directed towards whatever the goal was. It is useful to think about all the things that are considered successes from the government perspective while having lots of operational failures, e.g. recent wars or the ACA.

The argument for the difference in these cases is largely that exceptional leaders were chosen to lead them; for Europe also had a version of the Apollo program which failed, and the Nazi bomb program failed. Not came in second, mind you - but failed completely in their aims. So who would be the Mueller or Groves for the AI safety program?

I find it strange to say that we don't have any plan. Surely the government could set up scholarships or research institute or some kind of committee to look into this?

There are reason why "create some kind of committee to look into this" is often jokingly referred to as a way to kill a proposal. You can say that about every topic.