EU AI Policy / Mechatronics Engineer @ Independent Researcher
This was very interesting, looking forward to the follow up!
In the "AIs messing with your evaluations" (and checking for whether the AI is capable of/likely to do so) bit, I'm curious if there is any published research on this.
Hmm, in that case maybe I misunderstood the post, my impression wasnt that he was saying AI literally isn't a science anymore, but more that engineering work is getting too far ahead of the science part - and that in practice most ML progress now is just ML Engineering, where understanding is only a means to an end (and so is not as deep as it would be if it was science first).
I would guess that engineering gets ahead of science pretty often, but maybe in ML it's more pronounced - hype/money investment, as well as perhaps the perceived relative low stakes (unlike aerospace, or medical robotics which is my field) not scaring the ML engineers enough to actually care about deep understanding, and also perhaps the inscrutable nature of ML - if it were easy to understand, it wouldn't be as unappealing spend resources to do so.
I don't really have a take on where the in elegance comes in to play here
While theoretical physics is less "applied science" than chemistry, there's still a real difference between chemistry and chemical engineering.
For context, I am a Mechanical Engineer, and while I do occasionally check the system I am designing and try to understand/verify how well it is working, I am fundamentally not doing science. The main goal is solving a practical problem (i.e. as little theoretical understanding as is sufficient), where in science the understanding is the main goal, or at least closer to it.
So basically, post hoc, ergo propter hoc (post hoc fallacy)
If winning happened after rationality (in this case, any action you judge to be rational under any definition you prefer) it does not mean it happened because of it.
This was a great read! Personally I feel like it ended too quickly - even without going into gruesome details, I felt like 1 more paragraph or so of concluding bits in the story was needed. But, overall I really enjoyed it.
I'm trying to think of ideas here. As a recap of what I think the post says:
^let me know if I am understanding correctly.
I might have more thoughts later on.
(for context, I am recently involved in governance work for the EU AI Act)
Working at a startup made me realize how little we can actually "reason through" things to get to a point where all team members agree. Often there's too little time to test all assumptions, if it's even doable at all. Part of the role of the CEO is to "cut" these discussions when it's evident that spending more time on it is worse than proceeding despite uncertainty.
If we had "the facts", we might find it easier to agree. But in an uncertain environment, many decisions come down to the intuition (hopefully based on reliable experience - such as founding a similar previous startup)
To me it seems that there are parallels here. In discussions I can always push back on the intuitions of others, but where no reliable facts exist, I have little chance at getting far. Which is not always bad, since if we couldn't function well under uncertainty, we would likely be a lot less successful as a species.