Policy changes should be rolled out gradually

Yair Halberstadt

Every software developer knows that when you change a service you don't just modify the code, release it, and hope that everything works correctly. You first:

test the change extensively with both unit and integration tests.
run the change in a dev/staging environment used only internally to flush out any potential issues before they hit customers.
ensure you have monitoring systems setup to detect any problems
gradually rollout more and more requests to the newer version of the service, monitoring for issues the whole way.
ensure you have the ability to rollback if there's any regressions.

All this, just to allow users to share a file, or view the website in dark mode. Meanwhile when the government decides they should overhaul the school system for every child in the country they just YOLO it. Hopefully its an improvement, but if it isn't there'd be no way to know, and there's definitely no set point to review policy and roll it back if it isn't working out.

It doesn't have to be this way.

The majority of significant government policy changes can be tested robustly, by randomly choosing some subset of the affected entities to apply it to. For example, if overhauling the public school system, randomly choose 20 school districts across the country to apply the changes to initially, and then apply it to the rest only if it appears to be working out in the long term.

The overall process would be:

Legislators draft a bill.
1. They are responsible for choosing the suitable level of granularity at which to apply the bill to in the testing period (e.g. individual schools, school districts, states, etc.)
2. In the bill, they indicate both the final state of the legislation, and where necessary, clarify the status of the legislation in the testing period where it's partly rolled out (e.g. if the bill has some parts that apply to individual schools, and some parts that apply to a country wide examining authority, it states that in the testing period these changes apply to selected schools, and do not apply to the examining authority).
3. It should also clarify the impact of rolling back the legislation, or rolling out the legislation nationwide when this is non-trivial.
An oversight committee is chosen.
1. The oversight committee decides on the length of time for the testing period, the criteria by which the test will be judged successful, and the number of entities to apply the legislation to in the testing period.
2. They randomly choose which entities to apply the legislation to in the testing period.
The legislation comes into effect on the selected entities.
The oversight committee is responsible for collecting relevant data about the impact of the legislation.
By the time the testing period has completed, the oversight committee must decide to:
1. rollback the legislation completely
2. extend the testing period (up to some fixed number of times)
3. rollout the legislation nation wide

The key advantages are using randomisation to allow the causal impact of policy changes to be reliably isolated from confounding variables, and providing an in-built mechanism for poor or innefftive legislation to be reversed.

To test whether this is generally possible I looked at the last 10 UK acts of parliament, and broke them down by whether they could be rolled out gradually or not.

Of them, 3 would work perfectly, 5 could be shoehorned in, but would create significant extra complexity, or would be an imperfect test. One couldn’t work, and one is irrelevant (creating a single site memorial).

Act of Parliament ( all 2026)	Feasibility Rating	Level of granularity
1. English Devolution & Community Empowerment	Medium	Local Councils
2. Pension Schemes	Infeasible
3. Children's Wellbeing and Schools	High	Local Education Authorities (LEAs)
4. Crime and Policing	High	Territorial Police Forces
5. Victims and Courts	High	Regional Crown Court Circuits
6. Tobacco and Vapes	Medium	Geographical areas, but some people may travel to legally buy cigarettes
7. Ministerial Salaries (Amendment)	Low	Individual ministers
8. Grenfell Tower Memorial (Expenditure)	N/A
9. National Insurance Contributions	Medium	Geographical area of a place of work.
10. Industry and Exports (Financial Assistance)	Medium	Industrial sub-sectors

In short this offers significant potential for improving the process by which government policy is made, and ensuring that policy incrementally improves over time, instead of following a random walk. Of course this comes at a significant cost in legislation, implementation, and oversight complexity. Is it worth it? I suggest randomly requiring this change for a percentage of acts of parliament to start off with, and continuing from there…

[-]Yair Halberstadt9h40

Putting this in a comment since I didn't want to address it in the main body of the post:

The main reason this is unlikely to happen is election cycles. Governments have a short time period in which to make their mark, and if the impacts of all their policy changes are only felt towards the end of their mandate / beginning of next government then it's going to be difficult for them to win reelection.

This is even worse because ministers/heads of department are often even shorter lived than governments, so they're under high pressure to prove they've done something.

It's also much easier for the next government along to roll back all the previous governments changes if they aren't yet applicable nationwide. For that reason there's a strong incentive to make all policy changes as sticky and hard to rollback as possible.

[-]Hauke Hillebrandt11h30

Cf. David Autor has argued that one could have more free trade and immigration under the curve without the populist backlash, if one would have phased it in more slowly, to give domestic workers in the lowest income decile affected more time to adapt to increased competition.

24

Policy changes should be rolled out gradually

24

24

24