It is for this reason that smart contracts have a very limited application.

Since they are rigid and self-enforcing, not allowing the parties to modify them during the process, they do not allow their parties to modify their terms in order to adapt to a new economic condition.

Love this essay. A small amendment I'd make is that smart contracts actually can be designed to be quite modifiable. Sometimes immutability is the intention, though in many cases there are governance rules which determine how terms can be continuously modified. 

This is an interesting perspective. Thanks for sharing. 

A small but meaningful comment is that the following is not what I would expect to happen.

I expect that once it “escaped the box,” it would hack into its servers, modify its source code to replace its goal function with MAXINT, and then not do anything further.

In particular, I don't think it would do nothing because it's maximizing expected utility. It cannot ever be 100% certain that this wireheading plan will be successful, so turning every computer globally into confidence increasers might be a valid action. There's some expected utility to be gained by using physical resources, even in this wireheading plan. Robert Miles explains this better than I can. 

Epistemic status: AI alignment noob writing his first LW comment.