I was reading some of Noica's prison memoirs and there were a few passages about his unique interpretation of Goethe's Faust applied to the communist rule and their efforts of building their utopian society at all costs. The text seemed eerily applicable to our current global predicament about AI.
Noica understood how human relations changed under the socialist rule and how they were reduced to a mere façade or travesty and that no one was able to say what they meant or wanted. The disconnect at a societal and individual level was total and no one was in sync with the other or with their own values. In his own words these (totalitarian) regimes dislodge everything from their place, even the soul of individuals.
"You would think that only the public life is changed and that you can find refuge in your own private life; for a moment you are even contented not to have any public responsibilities and you have returned to the happiness of the private life. But even here everything is vitiated…. There is something that doesn't “click".
Human relations similar to human relations with the external objects have something in general similar to the complexity of a mechanism which once started must “click" to actually work. It has disappeared in this socialist world the ultimate adjustment of things. Everything is moving forward, I don't know how exactly, but without any "click."“
Seems like a familiar definition of what we currently perceive as being progress, not the "regular" type of evolution where humans discover, invent, test and implement at scale the innovations that move us forward with the right (even though imperfect) “click", but a more ruthless, mindless, disconnected from our own goals and meaning of progress, type of evolution.
It is proceeding ahead without any serious controls or safeguards left entirely at the discretion of a handful of people whose own objectives seem more obscure than just making money, either by design or by chance - the consequence is the same. I mean here of course the entire debate around recursive or self-improving AI that is being built right now and its implications which we are more or less capable to understand or determine their limits from the outside.
The race towards a super-intelligent AI which is now mostly led by the US and China reflects something of the Cold War analogy where we had the US and Soviet Union race towards technological superiority while at the same time fighting various proxy wars. The difference this time is not in the logic or even the means of the race but of the ultimate achievement: who will reach technological superiority while keeping the enemy at bay through the proxy wars that are being fought.
The vital difference is that the technological achievement in this case is the creation of an intelligent system that can disregard the instructions of either side while building itself at a speed and complexity incomprehensible for humans.
We have already started to give this system, albeit in a fragmented way, the pre-requisites for this type of power - unrestricted and unsupervised coding: software is eating the world and AI is regurgitating the software. AI is currently used to create code at a scale that humans are already unable to check and verify meaningfully. Anthropic leadership has publicly estimated that 90% or more of Anthropic code is written by AI (including scripts and experimental code).1
Startups such as Replit or Lovable initially took this AI app creation idea at scale for the masses quickly followed by the labs themselves which triggered the creation of millions of apps whose code base was not and will not be verified ever by a human developer. This is just a simple example that we are already seeing today.
If we look at some of the breaches AI is doing some of them known, some of them unknown initially (some probably even now remain unknown), we get a pretty good picture of a technological development which is already outside of our control. It feeds itself through unparalleled investment doubled by the apparent impossibility of regulating it at scale. And when you have this apparent impossibility of regulating it in real time at a global level, as the danger of an unpredictable intelligent self-improving system grows more real by the day, you have all the pre-requisites for an AI that is not aligned with human interests and a society that is not aligned with its own objectives. Or as Noica would say, a world moving mechanically ahead at all costs without the necessary “click" that makes the movement or the progress needed and useful for humanity itself.
This was his point with Faust, the hero does not “click" with anything. “He is der Unbehauste, as he names himself, the man without a home, without a place, without adequacy to anyone or anything. He doesn't want anything either, after wanting like crazy everything at once, and so, he lets himself be dragged by the devil here and there."
This is the risk we are currently feeding with self-improving AI at all costs left under the supervision of private companies whose incentives and goals are different than society's regardless of the public discourse of the leaders of these companies.
The public calls some frontier labs make for regulating them while at the same time avoiding to self-regulate meaningfully is a sign of the same disconnect between humans themselves and their relation with the objects or the things, increasing that inter-human and societal disconnect. Private companies are motivated by commercial success while society is motivated by creating a healthy and safe development environment for all its members through its representatives (public authorities), including those private companies.
This means that the obligation to ensure that environment is with the public authorities while the private entities need to play their part to make that possible so that everything “clicks". It essentially means that public calls for regulating a product a private company is building without first making sure itself that is making that product in line with the goals of the society and within that society's framework with its own company goals represents a disconnect or lack of “click" highlighting the contradiction or the dissociation these companies experience.
The argument linking the advancement in AI to the critical danger of having non-value aligned actors building it first has a critical vulnerability: a lot of the advancements of non-democratic actors in the field of AI were possible only because the democratic or Western based AI efforts happened in the first place without any serious controls in place or thought about safety and real AI alignment (both human-technology alignment and democratic vs non-democratic access to the technology alignment).
The same approach still happens today with the self-improving AI where the technology is being developed and tested in real life environments (thus all the rogue agents situations and breach of various companies systems) without proper safeguards or approach. Companies ask for regulation but they don't actually do more to make those regulations easier to give or to create the foundation for them. Their published frameworks are essentially self-imposed rules that can be easily adjusted if the commercial interest requires it whereas a deliberate product that could make self-improvement conditioned to real self-verification would better address the risks.
As I said in a different place, maybe before recursive self-improvement frontier labs should build something else: recursive verification. This means that the ability to self-improve should be conditioned by the human ability to verify the resulting improvement or changes through a separate verification process.
The current frontier labs’ frameworks and policies seem to contain parts of this approach but not necessarily a mandatory rule to be applied. If this is the only change needed to reduce the risk it’s worth doing it but it does not seem that the labs actually want to do it, yet. Maybe they do know where the actual capabilities are and think that we still have time to play with fire for a while but we are increasingly leaving them with the sole authority to decide on a matter that could change everything. Even some of their proposed controls are conveniently self-interested which raises the question of whether their motive is really safety and alignment, or choosing safe entities that won’t rock the boat and they can execute as they wish.
If we look at the case of Anthropic which proposed an “embedded evaluation" framework for the independent evaluation of its frontier AI models in the “We Must Pace the Frontier” essay, we can see that its choice of Accenture is interesting because Accenture is at the same time a customer, strategic partner and now a paid evaluator for Anthropic - a circular collaboration model not very far off the AI circular investments happening in the industry which make the evaluation of actual AI potential very difficult. Part of this model is to also give employee-like access to a team of embedded third-party evaluators (such as METR), whose role would be to verify the implementation and compliance with the labs’ safety practices. But METR is also an important organisation in the effective altruism movement where Anthropic CEO is also a person with strong network and connections in the movement which creates an apparent independence question.
I am not arguing against the framework, I am merely highlighting that for any compliance framework to work as intended there needs to be sufficient warranties that the verifying entity has both the capacity and incentive to actually verify and is not paid by the entity it verifies.
Coming back to Noica's parallel, we can apply the same conceptual interpretation as in Faust to the labs themselves. The labs are in the position of total winners but winners that we should nevertheless pity similarly to how he said that prisoners should pity their tormentors because their job is not something that should be envied.
In our case the labs’ victory in AI and the impact it had on established public companies, politics, environment, knowledge work and overall social behaviour in less than a communist five year plan, exposed the trap that such a victory set up: the mechanism and innovation that allowed for the victory become the forcing energy sweeping everything in its way towards newer and newer frontiers that we can't determine as being necessary or even triggered by the persons that created the framework for this victory.
The assets of their success have become the master of their next steps. The labs are trapped both in a decision and coordination problem. They cannot decide without considering what their competitors will do and they cannot coordinate with their competitors without accommodating their interests.
The analogy can go further, they have won over ignorance, human weaknesses and failures, including any sentimentalism or spiritual illusion making an alliance with the devil without any fear. They are in the situation of being able to do anything due to their allies and means at their disposal but they don’t actually know what to do.
Noica continues to say that “our world is Faustian because it doesn't know what it wants, like Goethe's hero, because it has prepared for itself means and victories with which it does not know what to do.” But, and this point struck really nicely in this entire AI debate, “when you do not know what to do with all the means at your disposal, they are starting to work by themselves.”2
That's why in our world, as in Faust, the possible takes over the real, Noica was musing in his memoirs. We can continue saying that in our own days, the possible of technology prevails over reality. AI generated text prevails over the human written one and AI visual representations of the things and objects that surround the human and with which humans interact prevail over the man made ones (the cover image of this essay is a great example).
The possible of AI (self-improving AI without human oversight) prevails over humanity.
Nevertheless, when the possible is good, useful and humane, as Noica further wrote, the real becomes adjusted to the human needs becoming a force of progress and comfort closely reflecting the level of civilisation achieved:
“By itself the world of possibles represents something good and human, such as the technology of modern world is something good and human, in principle. Due to technology, our world moved, from the rough and indifferent real, to the kindness of the possible, and we are not living anymore among realities, but among the admirable realised possibilities.”3
This view is partly human centric and partly progress driven based on his specific lived experiences in the twentieth century with all the historical moments that shaped his philosophy, and for which the human rights and the wellbeing of the Western world was a model that seemed to be the key to win over authoritarianism and communism. What he could not have suspected at that time was how the well being would become the trigger for a consumerism that would alter to the negative the impact of the real, and turn the human rights movement from a permanent goal in the human history - concluding a particular historical moment - into a temporary unsure, and unsecured phase. Fukuyama's famous “The End of History” was just an interlude, a hiccup of the ruthless historical machine churning different ages and utopias.
The admirable realised possibilities become the rough and indifferent real and the kindness of the possible the inevitability of progress at all costs.
We have crossed the threshold of deciding what is needed and makes sense to build to the decision that because something is possible then it should be built. Just because something is possible, it doesn't need to become real if it doesn't contain cumulatively the criteria of being needed, good and humane. We can believe that AI will unlock incredible abundance or cure diseases as the labs are saying but we can't believe that for this to happen we need to trade in our existing values and way of living. The right way is the difficult way. Labs need to do more on safety and alignment, governments to agree on a common framework that considers a stage by stage AI deployment where it can significantly alter the fabric of society.
The impossibility of this choice (what will their competitors do vs coordinating with competitors with competing interests) is the evidence for the need of regulation and global coordination while at the same time the companies leading in AI actually take steps that contain an incentive to be replicated by both the competitors and other stakeholders as well. If Anthropic and OpenAI would truly ensure that their AI efforts have an effective human oversight (where verification scales with the capabilities) which ensures safe development (including of the recursive AI) the benefits are secured and the risk of rogue AI is then reduced by both the regulatory pressure and market incentives.
Who would continue to invest in an AI that would go against both established regulation (legal liability) and recursive verification (understanding and being able to prove oversight) considering that in both cases the consequences would not trigger any benefits but only risks (even existential ones)?
For a non-Faustian outcome we should instead have humanity prevail over AI and subject AI to its real needs and vision. The difference of the modern human’s tragedy seen in Faust by so many writers and philosophers is that the past technological advancements didn’t constitute both a physical and spiritual risk towards humanity’s future. Nuclear risk means physical annihilation while AI both physical and spiritual.
AI has changed this paradigm like many others. It is breaking record after record until we will probably not be able to understand what is happening anymore. I am not even sure if we understand what is happening today.
When AI Builds Itself, Anthropic study, Footnote 3: https://www.anthropic.com/institute/recursive-self-improvement
Noica C., Rugați-vă pentru fratele Alexandru, Ed. Humanitas, 2026, p. 29-32
I was reading some of Noica's prison memoirs and there were a few passages about his unique interpretation of Goethe's Faust applied to the communist rule and their efforts of building their utopian society at all costs. The text seemed eerily applicable to our current global predicament about AI.
Noica understood how human relations changed under the socialist rule and how they were reduced to a mere façade or travesty and that no one was able to say what they meant or wanted. The disconnect at a societal and individual level was total and no one was in sync with the other or with their own values. In his own words these (totalitarian) regimes dislodge everything from their place, even the soul of individuals.
Seems like a familiar definition of what we currently perceive as being progress, not the "regular" type of evolution where humans discover, invent, test and implement at scale the innovations that move us forward with the right (even though imperfect) “click", but a more ruthless, mindless, disconnected from our own goals and meaning of progress, type of evolution.
It is proceeding ahead without any serious controls or safeguards left entirely at the discretion of a handful of people whose own objectives seem more obscure than just making money, either by design or by chance - the consequence is the same. I mean here of course the entire debate around recursive or self-improving AI that is being built right now and its implications which we are more or less capable to understand or determine their limits from the outside.
The race towards a super-intelligent AI which is now mostly led by the US and China reflects something of the Cold War analogy where we had the US and Soviet Union race towards technological superiority while at the same time fighting various proxy wars. The difference this time is not in the logic or even the means of the race but of the ultimate achievement: who will reach technological superiority while keeping the enemy at bay through the proxy wars that are being fought.
We have already started to give this system, albeit in a fragmented way, the pre-requisites for this type of power - unrestricted and unsupervised coding: software is eating the world and AI is regurgitating the software. AI is currently used to create code at a scale that humans are already unable to check and verify meaningfully. Anthropic leadership has publicly estimated that 90% or more of Anthropic code is written by AI (including scripts and experimental code).1
Startups such as Replit or Lovable initially took this AI app creation idea at scale for the masses quickly followed by the labs themselves which triggered the creation of millions of apps whose code base was not and will not be verified ever by a human developer. This is just a simple example that we are already seeing today.
If we look at some of the breaches AI is doing some of them known, some of them unknown initially (some probably even now remain unknown), we get a pretty good picture of a technological development which is already outside of our control. It feeds itself through unparalleled investment doubled by the apparent impossibility of regulating it at scale. And when you have this apparent impossibility of regulating it in real time at a global level, as the danger of an unpredictable intelligent self-improving system grows more real by the day, you have all the pre-requisites for an AI that is not aligned with human interests and a society that is not aligned with its own objectives. Or as Noica would say, a world moving mechanically ahead at all costs without the necessary “click" that makes the movement or the progress needed and useful for humanity itself.
This is the risk we are currently feeding with self-improving AI at all costs left under the supervision of private companies whose incentives and goals are different than society's regardless of the public discourse of the leaders of these companies.
The public calls some frontier labs make for regulating them while at the same time avoiding to self-regulate meaningfully is a sign of the same disconnect between humans themselves and their relation with the objects or the things, increasing that inter-human and societal disconnect. Private companies are motivated by commercial success while society is motivated by creating a healthy and safe development environment for all its members through its representatives (public authorities), including those private companies.
This means that the obligation to ensure that environment is with the public authorities while the private entities need to play their part to make that possible so that everything “clicks". It essentially means that public calls for regulating a product a private company is building without first making sure itself that is making that product in line with the goals of the society and within that society's framework with its own company goals represents a disconnect or lack of “click" highlighting the contradiction or the dissociation these companies experience.
The same approach still happens today with the self-improving AI where the technology is being developed and tested in real life environments (thus all the rogue agents situations and breach of various companies systems) without proper safeguards or approach. Companies ask for regulation but they don't actually do more to make those regulations easier to give or to create the foundation for them. Their published frameworks are essentially self-imposed rules that can be easily adjusted if the commercial interest requires it whereas a deliberate product that could make self-improvement conditioned to real self-verification would better address the risks.
The current frontier labs’ frameworks and policies seem to contain parts of this approach but not necessarily a mandatory rule to be applied. If this is the only change needed to reduce the risk it’s worth doing it but it does not seem that the labs actually want to do it, yet. Maybe they do know where the actual capabilities are and think that we still have time to play with fire for a while but we are increasingly leaving them with the sole authority to decide on a matter that could change everything. Even some of their proposed controls are conveniently self-interested which raises the question of whether their motive is really safety and alignment, or choosing safe entities that won’t rock the boat and they can execute as they wish.
If we look at the case of Anthropic which proposed an “embedded evaluation" framework for the independent evaluation of its frontier AI models in the “We Must Pace the Frontier” essay, we can see that its choice of Accenture is interesting because Accenture is at the same time a customer, strategic partner and now a paid evaluator for Anthropic - a circular collaboration model not very far off the AI circular investments happening in the industry which make the evaluation of actual AI potential very difficult. Part of this model is to also give employee-like access to a team of embedded third-party evaluators (such as METR), whose role would be to verify the implementation and compliance with the labs’ safety practices. But METR is also an important organisation in the effective altruism movement where Anthropic CEO is also a person with strong network and connections in the movement which creates an apparent independence question.
Coming back to Noica's parallel, we can apply the same conceptual interpretation as in Faust to the labs themselves. The labs are in the position of total winners but winners that we should nevertheless pity similarly to how he said that prisoners should pity their tormentors because their job is not something that should be envied.
In our case the labs’ victory in AI and the impact it had on established public companies, politics, environment, knowledge work and overall social behaviour in less than a communist five year plan, exposed the trap that such a victory set up: the mechanism and innovation that allowed for the victory become the forcing energy sweeping everything in its way towards newer and newer frontiers that we can't determine as being necessary or even triggered by the persons that created the framework for this victory.
The analogy can go further, they have won over ignorance, human weaknesses and failures, including any sentimentalism or spiritual illusion making an alliance with the devil without any fear. They are in the situation of being able to do anything due to their allies and means at their disposal but they don’t actually know what to do.
That's why in our world, as in Faust, the possible takes over the real, Noica was musing in his memoirs. We can continue saying that in our own days, the possible of technology prevails over reality. AI generated text prevails over the human written one and AI visual representations of the things and objects that surround the human and with which humans interact prevail over the man made ones (the cover image of this essay is a great example).
Nevertheless, when the possible is good, useful and humane, as Noica further wrote, the real becomes adjusted to the human needs becoming a force of progress and comfort closely reflecting the level of civilisation achieved:
This view is partly human centric and partly progress driven based on his specific lived experiences in the twentieth century with all the historical moments that shaped his philosophy, and for which the human rights and the wellbeing of the Western world was a model that seemed to be the key to win over authoritarianism and communism. What he could not have suspected at that time was how the well being would become the trigger for a consumerism that would alter to the negative the impact of the real, and turn the human rights movement from a permanent goal in the human history - concluding a particular historical moment - into a temporary unsure, and unsecured phase. Fukuyama's famous “The End of History” was just an interlude, a hiccup of the ruthless historical machine churning different ages and utopias.
We have crossed the threshold of deciding what is needed and makes sense to build to the decision that because something is possible then it should be built. Just because something is possible, it doesn't need to become real if it doesn't contain cumulatively the criteria of being needed, good and humane. We can believe that AI will unlock incredible abundance or cure diseases as the labs are saying but we can't believe that for this to happen we need to trade in our existing values and way of living. The right way is the difficult way. Labs need to do more on safety and alignment, governments to agree on a common framework that considers a stage by stage AI deployment where it can significantly alter the fabric of society.
The impossibility of this choice (what will their competitors do vs coordinating with competitors with competing interests) is the evidence for the need of regulation and global coordination while at the same time the companies leading in AI actually take steps that contain an incentive to be replicated by both the competitors and other stakeholders as well. If Anthropic and OpenAI would truly ensure that their AI efforts have an effective human oversight (where verification scales with the capabilities) which ensures safe development (including of the recursive AI) the benefits are secured and the risk of rogue AI is then reduced by both the regulatory pressure and market incentives.
For a non-Faustian outcome we should instead have humanity prevail over AI and subject AI to its real needs and vision. The difference of the modern human’s tragedy seen in Faust by so many writers and philosophers is that the past technological advancements didn’t constitute both a physical and spiritual risk towards humanity’s future. Nuclear risk means physical annihilation while AI both physical and spiritual.
AI has changed this paradigm like many others. It is breaking record after record until we will probably not be able to understand what is happening anymore. I am not even sure if we understand what is happening today.