While a bit dated, the series Yes, Minister provides a good view of the British civil service and the role of Permanent Secretaries.
I would espect the average Permanent Secretary nowadays to be less intelligent then Humprey was, given that private industry now pays much more compared to the burocratic positions thirty years ago, but the series was well-researched and does a good job of illustrating the system.
As usual, the descriptive part is good, the prescriptive part is suspect. Extraordinary progress is only achieved when extraordinary people end up in charge and have the power to control things. Basically "You cannot bullshit Steve Jobs". Sometimes "Steve Jobs" can be a group of people, like in the Manhattan project, or the Apollo program. Where the feedback on one's performance comes from achieving the stated goals, not (just) from how good you are with people.
Extraordinary progress is only achieved when extraordinary people end up in charge and have the power to control things.
Let's call this the "Steve Jobs theory".
That seemed like it was almost the thesis (of DC as described by the "OP"). Did the post seem off to you relative to the 'Steve jobs theory'?
One of the linked articles states that to make extraordinary progress, one needs to hire "Great connected people" and give them the freedom and finances to work on their passion, while removing the bureaucratic obstacles. It's not quite the Steve Jobs theory, but close. The same article also describes, time and again, how the results were achieved by either subverting or bypassing the existing bureaucracy. There is no way to reform it, there is no point in trying. The elected government is no better: these are people who optimize electability, not progress. The best one can do in absence of a severe crisis, when existing bureaucratic rules can be suspended, is to reduce the barriers for the great people wanting to do great things to work outside the existing bureaucracy. Elon Musk could not have achieve what he has in land- and space-transportation in, say, medicine, if his goal was to "cure cancer" or something, instead of going to Mars. Mostly because he would be thwarted at every step by the well-meaning yet stifling regulations.
Avoid the bureaucracy, don't try to reform it, if you want to get anything done.
I think the problem is that sometimes Steve Jobs happens by accident, but it is difficult to make him happen on purpose. Yes, the old method of hiring people optimizes for some other purpose than hiring the most competent. But the priors are high that the new method will also optimize for some other purpose -- maybe we can't immediately tell how exactly, if we never tried the new method, but after it is used for a few months, people will find their ways.
There are many ideas that sound nice and are supposed to work in theory. Then it turns out that democracy somehow does not produce the optimal outcomes for citizens, dictatorship of the proletariat does not produce the optimal outcomes for workers, corporations do not produce the optimal outcomes for the shareholders... Moloch always finds its way.
What could possibly go wrong with "the minister will appoint experts"? Maybe the next minister will always fire the previous one's experts and replace them with new ones, so no longer project ever gets completed. Or maybe there will be a rule against firing the old experts (e.g. you can only add new ones, but each one stays there for 20 years), and then you will have new and old experts working together, disagreeing with each other, and nothing gets done and everyone blames the other side. If you are looking forward to your favorite party's minister appointing their experts, you should also shiver at the thought of the opposite party's minister appointing their experts and giving them the same power. Maybe a politician you respect will unexpectedly choose some snake oil salesman as an expert. Or maybe the nominations of experts will need to be anounced in advance, in which case the ministers will settle for non-controversial experts over the competent ones. Etc.
1. The libraries idea could be useful.
2. If no idea works "perfectly", then:
3. Can competence be measured?
The libraries are definitely a good idea. A friend of mine worked for London city hall. They had an exciting new computerised mathematical model to calculate where transport links were bad and the computer produced them a proposal for a new bus route. At a glance the route made perfect sense, and they couldn't understand how that bus route didn't already exist, its value was so obvious at a glance. They started setting the bus route up, then a load of people wrote letters of complaint saying they didn't want busses full of poorer people going past their posh houses, and these complaint letters killed the project. Only much later did they learn that substantially the same bus route had been shot down by the same objections on 2 or 3 previous occasions. A department historian could have told them they were repeating history, so they had a better idea to expect that obstacle.
This would seem to be such a common roadblock in any urban centre that I'm surprised nobody in your friend's department asked if that was why the bus route didn't exist already.
What could possibly go wrong with "the minister will appoint experts"? Maybe the next minister will always fire the previous one's experts and replace them with new ones, so no longer project ever gets completed.
I don't think that's a good summary of the issue. Even the US system does have experts like Kiesinger that serve multiple administrations and have power when the ruling party changes.
In my father's generation, (see David Lodge, Changing Places, 1975), British academia had this system where if you did really well on your degree exams, you could get tenure, with the expectation that you'd give some lectures, and one day, once you'd thought of anything interesting, you'd write it up as a PhD.
There were senior academics in Cambridge, well respected in their fields, who'd never got round to writing a PhD.
Tenure was defined as 'we give you a well paid job for life, from which you can only be dismissed for "gross moral turpitude"'. Nobody knew what gross moral turpitude actually was, but it probably included murder.
In those days, the complaint was always that "British scientists would invent things, and then American businessmen would turn then into commercial successes".
I don't know how much truth there was in that complaint, but that was the commonly heard complaint.
Of course, people being people, many British Academics decided that their job was to stay at home drinking themselves to death, and never got in touch with their university ever again.
And of course, a lot of more passionate academics decided that their job was to be a full time advocate of communist revolution. Generally this was thought to be 'the sort of thing that academics did'.
At any rate, the answer was deemed to be to go over the the American system of 'Publish or Perish'.
No no-one can get anything done, and our youngest and brightest people spend their lives teaching, filling out grant applications, and moving house,
And God help you if you say something controversial.
My sources inform me. I ran.
While the ministers are nominally in charge, the officials can subtly block the implementation of policies they dislike. Because the civil service is permanent and ministers transient, the ministers are always at the disadvantage (there is also too much happening for one person to keep track of all the details). Further, because of movement within the civil service, it is almost impossible to hold people to account for their mistakes as they will often only arise years later, but which point they may have moved on to another department. (The last two complaints seem like they are in contradiction, but maybe I am misunderstanding him here).
The issues are
1. The ministers are "accountable", but not in control.
2. Those in control are not accountable, and move around a lot.
Imagine you try to make things better by coming up with possible solutions as a 'leader'. Rather than approaches succeeding or failing on their own merits, they may be secretly vetoed or modified by a cabal, whose members serve as roving shadow judges.
Ultimately the interests of the cabal are advanced over the 'leaders' or the beneficiaries, even if one 'shadow judge' may not stay in the same place for a long time. (Yes, this might limit the cabal's ability to come up with good plans because 'shadow judges' switch a lot.*)
*Focus on this issue seems to be theme. I'm calling it: "Problem 1".
Nor does he see any attempt to learn from previous mistakes.
This lines up with the above. (Problem 1.)
he argues that in order to achieve the greatest successes we need to be willing to endure massive failures.
Failure is a part of learning. (Problem 2.)
(Another argument would be the constantly shifting priorities can result in enormous waste when projects are stopped, then restarting again).
(Problem 1.)
Another key property is openness so that people are comfortable enough to admit errors so that they can be learned from. Instead of punishing people for admitting errors, the focus should be on punishing people who don't admit errors.
(Problem 2.)
He worries that our educational system isn't set up to produce the kinds of leaders that are needed for such an environment.
That makes sense.
These leaders need to be trained quantitatively so that they understand things like "exponential functions, normal distributions and conditional probability". He argues that less than 1% of the population are capable of understanding complex systems from a cross-disciplinary perspective and these people are almost invariably not in charge of government.
This might be a good thing, though we do care about complicated things that might not be capturable by numbers - in the conventional sense. (What does it mean to give a movie 5 stars? That it was really good and you'll watch it again soon? Then shouldn't the rating be "really good, I'll watch it again soon"? If it's something else that 'can't be put into words' that exists only as a rough comparison, then why is it a number instead of the phrase "as good as movie X" for some movie that's a benchmark for really good?*)
*If someone doesn't like movie X, then why would their opinion matter as much to you as the opinion of someone who did?
My confusion was he seems to think the permanent nature of the civil service provides them an advantage over ministers, but if they are always shifting around then wouldn't this prevent them from gaining a local knowledge advantage.
1. There's a difference between having unchecked power/no accountability, and being good at your job.
a. The ministers also move around. (Perhaps they can't gain a large knowledge advantage, but if no one can fire you, and you're actually in control (if only via a 'succeed/fail switch') then you can have a power advantage.
b. They are at an advantage if they have (greater) career security - even in the form of moving around, while the elected officials may not have the same job security. For an analogy, how good is someone at a job on their first day? After 2 years? After switching departments?
2. They can have more information:
a. For example, if internally they are able to find predecessors, they could in theory communicate individually. At a group level, think of it as a conspiracy - maybe they don't write things down, or destroy records, but they can still operate over time and communicate (if they bother).
b. Knowing things like 'did the policy fail, or fail because it was never tried?' can convey a slight relative advantage. Additionally if they tend to 'serve' in the same capacity for longer periods (as compared to ministers), then even if they were starting from scratch, they'd have time to build up more of an advantage.
Re departmental historians: the UK's Foreign Office does have (or had) a small team of historians; someone I know was one of them for a year or two. Apparently they were writing up the history of the Foreign Office in chronological order, at slower than real time; hence were falling further and further behind. They had got up to 1947 or something, but would never catch up. When they completed the history of a year, it was published in an internal book (I assume not publicly available due to national security etc.), which went on a shelf and no-one ever read.
Each year the historians had to submit a justification for their continued existence. The guy I know said there was none, and they should just write: "The Foreign Office should close down its history department."
I suppose what this shows is that if internal historians have a use, it's important that they know the full departmental history, particularly recent decades; they are involved in departmental decisions; and the history is as up-to-date as possible.
Like him or loath him, Dominic Cummings is now the most politically successful rationalist-adjacent figure. For this reason, it seems like our community ought to try to understand him to see what we can learn. I'm sure there are people who have read more of his content than I have and who have read it more carefully than me, but I tend to see the perfect as the enemy of the good. At times I struggled to understand what he was saying because he was making references which I, an Australian not involved in the political world, wasn't familiar with, which likely limited the quality of my understanding. At other times I felt that his writing was extremely verbose and that I had to wade through a lot of content before the point became clear, so hopefully this saves you some time.
Dominic Cumming's main thesis seems to be that there are certain way of managing a team that are known to be highly effective, but which are widely ignored. In terms of politics, he argues that the government is ineffective because of self-interested bureaucrats blocking progress, who'd never think of or allow teams to be run in such a manner. One area he particularly focuses on is science funding where he argues for a British version of DARPA (now being created) is needed to take high-risk, high reward projects that could provide ridiculous amounts of economic value. One of his main critiques of the education system is that it doesn't provide the kind of leaders who are needed to run the country in the way he proposes - with an interdisciplinary background including technical skills in maths and science. As you can probably see, his views are highly interconnected.
I think the best place to begin is with his criticisms of bureaucracy. He particularly focuses upon dysfunction within the British civil service, but he also saw EU rules as just another layer upon this, which is why he pushed for Brexit.
I thought Hollow Men II was best post on this. The British Civil service is quite different from the American Civil Service in that ministries are run by Permanent Secretaries who aren't supposed to be political appointees. While the Prime Minister makes the final decision, the candidates for consideration are selected by the civil service on the basis of what is supposed to be an objective process.
The idea is that this will result in appointees on the basis of merit rather than politics, however Dominic Cummings believes that it does anything but, with internal civil services politics simply replacing external governmental politics. He argues that even if a minister ultimately makes an appointment, the civil service has sufficient control over criteria and evaluation of these criteria to ensure that the options are all favourable to them.
He further believes that ministers should be control both policy and implementation, as the current civil service is unaccountable. He writes that ministers being humiliated or fired doesn't "put the slightest dent in their [officials] day – never mind their career". While the ministers are nominally in charge, the officials can subtly block the implementation of policies they dislike. Because the civil service is permanent and ministers transient, the ministers are always at the disadvantage. There is just too much happening for one minister and a handful of special advisors to keep track of all the details. Further, because of movement within the civil service, it is almost impossible to hold people to account for their mistakes. These will often only arise years later, but which point they and their boss will have moved on to another department. (The last two complaints seem like they are in contradiction, but maybe I am misunderstanding him here).
He argues that the internal view of the civil service is much more dysfunctional than you would believe with ministers entering every half-hour to explain a new messup and ministers receiving letters with errors in facts, spelling and grammar which they'd have to correct themselves. (Another example, is a senior civil service member trying to get the lifts fixed and eventually just giving up). He argues that in order to better control the civil service, ministers would need control over hiring, firing and promotion. They'd also need the ability to recruit talented outsiders. Further, he argues that the civil service has an incentive to ensure that their performance is never evaluated, at least not in a way that they couldn't manipulate. The focus is almost entirely on process rather than performance and they are very reluctant to change existing processes in a way that would allow this dysfunction to be addressed. Nor does he see any attempt to learn from previous mistakes.
So what is his alternative? For the civil service, his answer is contained in What Is To Be Done? An answer to Dean Acheson’s famous quip. As already noted, he wants to bring in talent for the outside and provide ministers with control over hiring/firing. He thinks that the civil service should be open to hiring people with unusual talents who would never be hired according to the traditional, credentialist focus (not that he avoids credentials completely). He wants politicians to pay less attention to the media and focus on the long term. Further, he wants to shrink departments in order to simplify them and increase their focus. He argues that most politicians don't understand how ruthlessly they have to prioritise if they want to actually get things done. But beyond this, he wants to create sections of government that can be run as a startup without being smothered by other sections.
Dominic Cummings believes that most managers have failed to learn the lessons from highly successful projects such as the Manhattan Project, ICBMs and the Apollo project and indeed that most projects are run in a completely opposite manner "in the name of efficiency".
Probably the most informative documents you can read from him on management is his essay - On the referendum & #4c on Expertise: On the ARPA/PARC ‘Dream Machine’, science funding, high performance, and UK national strategy and The unrecognised simplicities of effective action #2: ‘Systems engineering’ and ‘systems management’ — ideas from the Apollo programme for a ‘systems politics’. He argues that PARC was successful because they found great people and provided them the freedom to do what they needed to do.
Dominic Cummings argues that you need great people and not good people because "Ten good people can't do what a single "great" one can". Further, he argues that far too often HR excludes most of the best people who don't fit their narrow criteria and spends time defending the incompetent. He worries that most organisations are only set up to deal with people of moderate ability, motivation and trust. He frequently quotes John Boyd's saying: "People, ideas, technology - in that order". (His desire for great people ties into his policies on immigration that allow talented people, particularly scientists much easier access to the country).
He worries about micromanaging people and money as this often puts barriers in the way of their success. The intent is often to ensure efficiency and to reduce the chance of failure, but he argues that in order to achieve the greatest success we need to be willing to endure massive failures.
He also stressed the importance of connected people, not isolated people. He cites Mueller requiring NASA teams to communicate with their functional counterparts in other teams in order to exchange information and best practices and not solely through their bosses.
Dominic Cummings also described how configuration management allowed massively complex projects to be delivered on schedule. A large amount of time was spent defining the initial design and interfaces and whenever a change was made to an interface everyone using it was notified. This ensures that the system as a whole works when everything is put together.
Another technique he favours is matrix management where people report both to their department heads and to a project manager. He argues that this allows coordination across departments that is impossible with the current system.
He believes that large departments should have libraries and internal historians. Some government departments had internal libraries, but abolished them which means that they no longer know if they've already tried a particular policy. He's also a huge fan of Tetlock's work on superforecasters.
Another of his principles is that using long-term budgets save money by allowing more investment in things that have long-term payoffs and enabling long-term planning. (Another argument would be the constantly shifting priorities can result in enormous waste when projects are stopped, then restarting again). He also worries about how the focus on efficiency means that redundancy to increase resilience is often seen as waste. He frequently talks about how going faster saves money, but I couldn't find an explanation of this as far as I could see.
He argues that both centralisation and decentralisation is important. He believes that the top needs to set an overall vision with goals and a strategy, it is also important that people down the bottom can execute in whatever manner they believe to be best.
Another key property is openness so that people are comfortable enough to admit errors so that they can be learned from. Instead of punishing people for admitting errors, the focus should be on punishing people who don't admit errors.
This brings us to Dominic Cummings vision for a British version of ARPA. He seems to believe that ARPA is responsible for the US being at the head of the internet revolution, with the countless billions of dollars of value that this brings. He believes that by creating a similar organisation Britain could be at the forefront of innovation again.
He worries that the current funding arrangements encourage incremental developments over the truly transformative. Projects that have the potential to truly transform a domain of science have a much higher chance of failure and when they fail open up the funders to criticisms of having not done their job of filtering correctly.
Short term grants mean that scientists have to waste far too much of their time filling out paperwork and reduce the viability of truly ambitious projects which often won't be able to demonstrate solid results until much further along in the process.
In addition, he worries that the recent focus on ensuring science has applications is short-sighted in that basic science often results in a huge amount of applications, but further down the line and beyond anyone's ability to predict.
He believes that it is important that it be its own agency, because if this funding were handled by an existing agency the grants would be subject to an excessive amount of bureaucracy and the selection process would tend to weed out anything too radical.
Dominic Cummings argues that there is a "mismatch between the power of our society and our ability to cope with it" (link). He believes that with all of its dysfunction the bureaucracy is completely unprepared for dealing with a world contuining "complex, nonlinear, interdependent networks with feedback" and the kinds of existential risks we will face in the future.
He worries that our educational system isn't set up to produce the kinds of leaders that are needed for such an environment. These leaders need to be trained quantitatively so that they understand things like "exponential functions, normal distributions and conditional probability". He argues that less than 1% of the population are capable of understanding complex systems from a cross-disciplinary perspective and these people are almost invariably not in charge of government.
His favoured approach to education would be to pick the biggest problems facing humanity in way of motivation and to explore connections between these problems to train people who can synthesise information. Anyway, he has a whole 240 page document on his preferred Odyssean Education System, but I've only read the first few pages.
Inside the Mind of Dominic Cummings: One aspect this article emphasises is his drive to get thing done, rules be damned.