I don't completely agree with your characterisation "[math] seems to have gotten it impossibly right the first time around" of how we got the current abstractions in mathematics. Taking your example of analysis, 1) Leibniz and Newton put forward different ideas about what the operation of taking a derivative meant, with different notations 2) there was a debate over (two) centuries before the current abstractions were settled on (the ones that are taught in undergraduate calculus) 3) in the 60s famously "non-standard analysis" was developed, to give an example of a radical departure, but it hasn't really caught on.
Still within analysis, I would point out that it's common(-ish?) to teach two theories of integration in undergraduate math: Riemann and Lebesgue. Riemann integration is the more intuitive "area of thin rectangles under the curve" and is taught first. However, the Lebesgue integral has better theoretical properties which is useful in, for example, differential equations. And beyond undergraduate, there are conceptions of limits in topology and category theory also.
Overall, I'd agree that the rate of trying out new abstractions seems to be lower in mathematics than programming, but as another commenter pointed out, it's also much older.
A second point is that the relevant distinction may be teaching mathematics vs research mathematics. It seems to me that a lot more theories are tried out on newer mathematics in topics of active research than in teaching the unwashed hordes of non-math-major students.
You make this comparison between programmers and mathematicians, but perhaps the more apt analogy is programming language designers vs mathematicians and programmers vs engineers/scientists? I would say that most engineers and scientists learn a couple of mathematical models in class and then go off and do stuff in R or Matlab. What the average engineer/scientist can model presently is now far greater than even the very best could model in the past. And they don't need to know which of the 11 methods of approximation is going on under the hood of the program.
Then the different abstractions are things like ODE models, finite element analysis, dynamical systems (eg stability), monte carlo, eigenvalue analysis, graph theory stuff, statistical significance tests, etc