This post is crossposted from my Substack,Structure and Guarantees, where I explore how formal verification and related ideas might scale to more complex intelligent systems. Here I argue that modern AI ecosystems are shaped not only by engineering trade-offs but also by signaling incentives grounded in evolutionary psychology. Engineers build status by solving socially legible hard problems, while organizations build status through socially legible expensive deployments. The result is a subtle pressure toward AI approaches that are unusually effective as prestige signals, even when alternative system designs could produce solutions that are cheaper, faster, or more reliable.
I’ve been making the case that today’s popular style of generative AI is fundamentally slow and unreliable. To unlock applicability of other styles (most importantly those based on symbolic reasoning and proof), I’ve also argued for a different sort of learning loop, where we go beyond repeatedly improving a model based on mistakes it makes, also making strategic changes to the world and the problem formulation, tending toward simplifying AI challenges. Examples include simplifying or avoiding natural-language processing (e.g. by using computer-oriented communication methods), computer vision (e.g. by rearranging environments to be more geometrically regular), and challenging programming tasks (e.g. by adopting better programming languages).
This perspective is so far from the mainstream that it isn’t even on a menu of possible positions that folks are used to being prepared to argue with. One natural hypothesis is that success in the real world fundamentally requires solving familiar hard AI problems, so we really need to stick with techniques like deep learning that have the best track record solving those problems. That hypothesis is compatible with an engineering mindset, where in building better artifacts, we may need to trade off between dimensions like generality, speed, and reliability – and deep learning’s success in generality justifies weaknesses in other dimensions.
My subject in this post is how such claims may often be after-the-fact rationalizations. There are other reasons, grounded in evolutionary psychology, why we should expect the popular approach, epitomized by LLMs and the systems built on them, to be seductive. By surfacing those reasons, we can better inform ourselves to choose the right technology for each job. The key will be that familiar aspects of large-scale AI deployment are helpful to individuals looking to build status, in ways that create perverse incentives at odds with traditional engineering goals.
Signaling: Expensive Displays
I previously covered the phenomenon of signaling, where animals perform costly displays of fitness, so that observers are convinced that they are worthy as mates, coalition partners, and so on. I presented signaling as a cool trick to accelerate evolution, where observers don’t have to wait for individuals to encounter rare situations where their skills can shine, since instead regular opportunities are created to show off the same skills in artificial circumstances. Having reliable information sooner helps evolutionary processes improve more quickly, by directing more resources to the most-promising individuals. Humans are particularly flexible in being able to learn new signaling games, which might not even have existed at their births, let alone in the bulk of our evolution before we converged into anatomically modern humans only a few hundred thousand years ago. I suggested “programmable evolution” as a useful way to describe that flexibility.
A textbook example of signaling through conspicuous consumption is the potlach, a kind of feast among the indigenous people of North America. At such a party, a tribal leader gives away or flat-out destroys many items considered valuable. The point is precisely that only a very rich leader or tribe could afford to part with so much wealth. In turn, the wealth is considered to have arisen through effectiveness in leadership and planning, producing a highly legible signal of leadership quality. We couldn’t trust a leader to give a speech explaining his triumphs. We all know how easy self-aggrandizement can be, picking and choosing topics. Instead, we need a signal that is expensive to fake, and what better than wanton destruction of objects that are known to require great coordination and talent to create?
This kind of social ritual is far from having disappeared among us. Think of a Great Gatsby-style mansion party, still built around conspicuous consumption and over-the-top hosting competency. A relatively recent study that I enjoyed was Very Important People: Status and Beauty in the Global Party Circuit, which looks at rituals in nightclubs today.
The point is that we find easy analogies between old-school anthropology and wealth-signaling displays today. Signaling behavior around a variety of admirable qualities is pervasive in our world, for instance accounting for art as a way to signal cognitive ability. Now we can turn to the specifics of these phenomena around AI-powered systems.
Difficult Problems and Expensive Solutions
Let’s start by thinking about how elite software engineers build and maintain their status in the global tech industry. Imagine that an engineer has applied for a job, and the cornerstone of his portfolio is a totally new technology stack, from custom hardware on up, that he built to solve an important problem. On one level, we’re suitably impressed by his versatility. However, we also find it very hard to tell which parts of his work were hard – and of course we want to hire smart people who are good at solving hard problems. We know the popular technology stacks of today and which of their aspects require which skills to execute competently. Our job evaluating the candidate would be much easier if he picked one of the established areas to focus on.
Candidates appreciate this dynamic, perhaps not always fully consciously. With AI as the hottest area today, it is not only valuable for engineers to build up portfolios of AI work, but also it benefits them to choose among relatively small menus of “standard” AI problems that are widely understood by people who make hiring decisions. So now we have the most innocuous form of signaling dynamics leading to lock-in for choice of AI problems.
However, the effects are more perverse than that narrative suggests. Signaling games are more effective when they involve harder problems, so the tech industry tends toward elite engineers choosing the hardest problems to work on. This incentive exists even if the hardest problems are not aligned with the best engineering solutions to real-world problems in any dimensions, like cost, speed, or decision quality. Difficulty is valuable in its own right. Just to avoid the impression of taking potshots at one technical community from my own ivory tower in academia, I’ll mention that I have observed a similar dynamic in the world of programming languages, where many engineers in my circles prefer programming languages like Haskell and particularly abstract ways of using them that turn programming into challenging puzzles, applying more-complex code than is needed to solve a problem, for love of the mathematically grounded obstacles that must be overcome. A programmer with this mindset typically isn’t consciously thinking “I want to make my job harder,” but I argue that evolutionary dynamics explain why it would feel natural and fulfilling to seek out hard problems that most people can’t solve.
I also want to emphasize that current frontier AI work is doing amazing things, with highly successful efforts to solve hard problems in new ways delivering phenomenal value to society. Still, we should remain wary of the effect of signaling on incentives. We should also keep an eye out for the possibility to redesign systems at higher levels, so that lower levels no longer contain AI problems that are as challenging – as sad as it may feel to lose the opportunity to solve those problems heroically.
We’ve now gone two steps up the ladder of ways that signaling drives engineering choices, progressing through increasingly less noble-sounding root causes. The signaling we’ve covered so far takes place within relatively narrow specialist communities: engineers evaluating engineers. It can be an uphill battle to convince modern knowledge workers to forsake the valorization of solving hard problems, even if engineers will generally agree in the abstract that it is better to find ways to decompose a goal into subproblems that cost the least to solve. OK, but let me next consider another variety of signaling that stands in conflict with many of our stated values, where, while specialists may generate the signals, the signals can be evaluated by a general audience.
The last examples focused on costs from the labor of rare expert software engineers. Harder AI problems require paying more to rarer experts. However, another proxy for problem hardness is instructive: the cost of hardware needed to implement competitive solutions. Today’s token commodity in that category is the GPU, the cousin of CPUs that is the overwhelmingly popular choice for implementing deep learning. There is a striking convergence of prices for GPUs and luxury watches, with popular models costing tens of thousands of dollars each (see sources for GPUs and watches). It’s probably the case that GPU prices are driven by genuine supply-chain challenges ramping up to serve the extraordinary demand, while high-end watch prices reflect intentional scarcity and extra features added precisely because they increase price. Still, we wind up with GPUs in a secondary role to signal conspicuous consumption, like works of art ceremonially destroyed at potlachs. That last statement can coexist with the great fit between GPUs and implementation of deep learning, which just helps hide the signaling motive, without reducing its potency.
Then we have the data center, overwhelmingly important today for housing GPUs. Naturally, the cost of a data center is significantly higher than the cost of just one computer inside it, creating an even more potent signal of wealth. Data centers also have a significant advantage for signaling to society at large. Only geeks know enough about GPUs to compare them and decide which are most impressive, beyond just looking at the price tag (which isn’t typically hanging off of the deployed GPU on a piece of string!). There can be a similar dynamic with luxury watches, where only a connoisseur can tell the difference between models of radically different price levels. However, a data center is a large facility consuming a large amount of electricity. It is very easy for members of the global elite across industries to understand that such an object must be expensive and indicate success by whatever organization owns it.
Across tech-company sizes, we can see a common dynamic of signaling with AI-hardware deployments. The biggest companies compete to announce larger and larger build-outs of data centers, as well as breakthrough solutions to ever-harder AI problems (connecting to the prior example of incentives to solve hard problems). Scrappy start-ups can go through hard times and then, after new success in sales or attracting investment, signal their success very clearly by buying GPUs. The cofounders feel a sense of relief that, in contrast to earlier periods when no one could tell if they were “for real,” now everyone can tell “they’ve really made it” when their GPU stockpiles are large enough. Release of open-source software or open-weight models can be a good signaling trick, too, if it’s clear that the artifacts could only have been produced by burning enough compute, for instance in training the model using your GPU farm.
By the way, even individual engineers can feel the call of more-expensive solutions in their daily work. There was quite a stir recently around the revelation of tech companies maintaining internal leaderboards of, basically, which software engineers were spending the most on AI tools. There is certainly a correlation between cost of AI services used to build some technical artifact and the inherent challenge (or business value) of building that artifact, but we can wind up in strange places if measuring and incentivizing just the former.
Conclusion
Engineering involves navigating trade-offs between dimensions like cost, speed, and accuracy. However, the signaling phenomenon I’ve outlined pushes tech-industry participants, often without realizing it consciously, to optimize perversely for focusing on hard problems (the spirit of engineering prefers to simplify problems) and expensive solutions (while maximizing cost is rarely part of a classical engineering trade-off). Participants from CEOs to junior engineers can build status, both within their specialist communities and in society at large, by affiliating with expensive solutions to hard problems. The pull is strongest for problems and solutions that are obviously hard or expensive to a broad educated audience, and we should work to counteract that pull.
Evolution has left us with many bad habits – or ways of thinking that are clearly misaligned with modern life. For instance, we learn as kids that it is not a shrewd move to adopt the all-candy diet, even if our ancestors 100,000 years ago encountered sugar so rarely that they always won by consuming as much as they could find. Today’s engineered systems are vastly more complex than anything those ancestors dealt with, weakening the relevance of their instincts to signal success by displaying expensive solutions to hard problems. We should follow the true spirit of engineering and always appreciate a chance to simplify a problem by changing a system design at a higher level. Sometimes simplifying the world is itself an engineering challenge, and we should watch out for perverse incentives in choosing to do it, but often the investment of changing the world pays off in simpler engineering subproblems ever after.
Deep learning and friends are especially likely to excel at hard problems produced by evolution that we can work around today, as I will explore in considering how the compute stack should be different to take advantage of problems with elegant logical structure. If everything goes well, we won’t need to depend anymore on computer-hardware-world equivalents of luxury watches! It won’t hurt that we get more-reliable systems as a bonus, in a world with more of our economy handed off to AI agents in a carefully curated ecosystem.
This post is crossposted from my Substack, Structure and Guarantees, where I explore how formal verification and related ideas might scale to more complex intelligent systems. Here I argue that modern AI ecosystems are shaped not only by engineering trade-offs but also by signaling incentives grounded in evolutionary psychology. Engineers build status by solving socially legible hard problems, while organizations build status through socially legible expensive deployments. The result is a subtle pressure toward AI approaches that are unusually effective as prestige signals, even when alternative system designs could produce solutions that are cheaper, faster, or more reliable.
I’ve been making the case that today’s popular style of generative AI is fundamentally slow and unreliable. To unlock applicability of other styles (most importantly those based on symbolic reasoning and proof), I’ve also argued for a different sort of learning loop, where we go beyond repeatedly improving a model based on mistakes it makes, also making strategic changes to the world and the problem formulation, tending toward simplifying AI challenges. Examples include simplifying or avoiding natural-language processing (e.g. by using computer-oriented communication methods), computer vision (e.g. by rearranging environments to be more geometrically regular), and challenging programming tasks (e.g. by adopting better programming languages).
This perspective is so far from the mainstream that it isn’t even on a menu of possible positions that folks are used to being prepared to argue with. One natural hypothesis is that success in the real world fundamentally requires solving familiar hard AI problems, so we really need to stick with techniques like deep learning that have the best track record solving those problems. That hypothesis is compatible with an engineering mindset, where in building better artifacts, we may need to trade off between dimensions like generality, speed, and reliability – and deep learning’s success in generality justifies weaknesses in other dimensions.
My subject in this post is how such claims may often be after-the-fact rationalizations. There are other reasons, grounded in evolutionary psychology, why we should expect the popular approach, epitomized by LLMs and the systems built on them, to be seductive. By surfacing those reasons, we can better inform ourselves to choose the right technology for each job. The key will be that familiar aspects of large-scale AI deployment are helpful to individuals looking to build status, in ways that create perverse incentives at odds with traditional engineering goals.
Signaling: Expensive Displays
I previously covered the phenomenon of signaling, where animals perform costly displays of fitness, so that observers are convinced that they are worthy as mates, coalition partners, and so on. I presented signaling as a cool trick to accelerate evolution, where observers don’t have to wait for individuals to encounter rare situations where their skills can shine, since instead regular opportunities are created to show off the same skills in artificial circumstances. Having reliable information sooner helps evolutionary processes improve more quickly, by directing more resources to the most-promising individuals. Humans are particularly flexible in being able to learn new signaling games, which might not even have existed at their births, let alone in the bulk of our evolution before we converged into anatomically modern humans only a few hundred thousand years ago. I suggested “programmable evolution” as a useful way to describe that flexibility.
A textbook example of signaling through conspicuous consumption is the potlach, a kind of feast among the indigenous people of North America. At such a party, a tribal leader gives away or flat-out destroys many items considered valuable. The point is precisely that only a very rich leader or tribe could afford to part with so much wealth. In turn, the wealth is considered to have arisen through effectiveness in leadership and planning, producing a highly legible signal of leadership quality. We couldn’t trust a leader to give a speech explaining his triumphs. We all know how easy self-aggrandizement can be, picking and choosing topics. Instead, we need a signal that is expensive to fake, and what better than wanton destruction of objects that are known to require great coordination and talent to create?
This kind of social ritual is far from having disappeared among us. Think of a Great Gatsby-style mansion party, still built around conspicuous consumption and over-the-top hosting competency. A relatively recent study that I enjoyed was Very Important People: Status and Beauty in the Global Party Circuit, which looks at rituals in nightclubs today.
The point is that we find easy analogies between old-school anthropology and wealth-signaling displays today. Signaling behavior around a variety of admirable qualities is pervasive in our world, for instance accounting for art as a way to signal cognitive ability. Now we can turn to the specifics of these phenomena around AI-powered systems.
Difficult Problems and Expensive Solutions
Let’s start by thinking about how elite software engineers build and maintain their status in the global tech industry. Imagine that an engineer has applied for a job, and the cornerstone of his portfolio is a totally new technology stack, from custom hardware on up, that he built to solve an important problem. On one level, we’re suitably impressed by his versatility. However, we also find it very hard to tell which parts of his work were hard – and of course we want to hire smart people who are good at solving hard problems. We know the popular technology stacks of today and which of their aspects require which skills to execute competently. Our job evaluating the candidate would be much easier if he picked one of the established areas to focus on.
Candidates appreciate this dynamic, perhaps not always fully consciously. With AI as the hottest area today, it is not only valuable for engineers to build up portfolios of AI work, but also it benefits them to choose among relatively small menus of “standard” AI problems that are widely understood by people who make hiring decisions. So now we have the most innocuous form of signaling dynamics leading to lock-in for choice of AI problems.
However, the effects are more perverse than that narrative suggests. Signaling games are more effective when they involve harder problems, so the tech industry tends toward elite engineers choosing the hardest problems to work on. This incentive exists even if the hardest problems are not aligned with the best engineering solutions to real-world problems in any dimensions, like cost, speed, or decision quality. Difficulty is valuable in its own right. Just to avoid the impression of taking potshots at one technical community from my own ivory tower in academia, I’ll mention that I have observed a similar dynamic in the world of programming languages, where many engineers in my circles prefer programming languages like Haskell and particularly abstract ways of using them that turn programming into challenging puzzles, applying more-complex code than is needed to solve a problem, for love of the mathematically grounded obstacles that must be overcome. A programmer with this mindset typically isn’t consciously thinking “I want to make my job harder,” but I argue that evolutionary dynamics explain why it would feel natural and fulfilling to seek out hard problems that most people can’t solve.
I also want to emphasize that current frontier AI work is doing amazing things, with highly successful efforts to solve hard problems in new ways delivering phenomenal value to society. Still, we should remain wary of the effect of signaling on incentives. We should also keep an eye out for the possibility to redesign systems at higher levels, so that lower levels no longer contain AI problems that are as challenging – as sad as it may feel to lose the opportunity to solve those problems heroically.
We’ve now gone two steps up the ladder of ways that signaling drives engineering choices, progressing through increasingly less noble-sounding root causes. The signaling we’ve covered so far takes place within relatively narrow specialist communities: engineers evaluating engineers. It can be an uphill battle to convince modern knowledge workers to forsake the valorization of solving hard problems, even if engineers will generally agree in the abstract that it is better to find ways to decompose a goal into subproblems that cost the least to solve. OK, but let me next consider another variety of signaling that stands in conflict with many of our stated values, where, while specialists may generate the signals, the signals can be evaluated by a general audience.
The last examples focused on costs from the labor of rare expert software engineers. Harder AI problems require paying more to rarer experts. However, another proxy for problem hardness is instructive: the cost of hardware needed to implement competitive solutions. Today’s token commodity in that category is the GPU, the cousin of CPUs that is the overwhelmingly popular choice for implementing deep learning. There is a striking convergence of prices for GPUs and luxury watches, with popular models costing tens of thousands of dollars each (see sources for GPUs and watches). It’s probably the case that GPU prices are driven by genuine supply-chain challenges ramping up to serve the extraordinary demand, while high-end watch prices reflect intentional scarcity and extra features added precisely because they increase price. Still, we wind up with GPUs in a secondary role to signal conspicuous consumption, like works of art ceremonially destroyed at potlachs. That last statement can coexist with the great fit between GPUs and implementation of deep learning, which just helps hide the signaling motive, without reducing its potency.
Then we have the data center, overwhelmingly important today for housing GPUs. Naturally, the cost of a data center is significantly higher than the cost of just one computer inside it, creating an even more potent signal of wealth. Data centers also have a significant advantage for signaling to society at large. Only geeks know enough about GPUs to compare them and decide which are most impressive, beyond just looking at the price tag (which isn’t typically hanging off of the deployed GPU on a piece of string!). There can be a similar dynamic with luxury watches, where only a connoisseur can tell the difference between models of radically different price levels. However, a data center is a large facility consuming a large amount of electricity. It is very easy for members of the global elite across industries to understand that such an object must be expensive and indicate success by whatever organization owns it.
Across tech-company sizes, we can see a common dynamic of signaling with AI-hardware deployments. The biggest companies compete to announce larger and larger build-outs of data centers, as well as breakthrough solutions to ever-harder AI problems (connecting to the prior example of incentives to solve hard problems). Scrappy start-ups can go through hard times and then, after new success in sales or attracting investment, signal their success very clearly by buying GPUs. The cofounders feel a sense of relief that, in contrast to earlier periods when no one could tell if they were “for real,” now everyone can tell “they’ve really made it” when their GPU stockpiles are large enough. Release of open-source software or open-weight models can be a good signaling trick, too, if it’s clear that the artifacts could only have been produced by burning enough compute, for instance in training the model using your GPU farm.
By the way, even individual engineers can feel the call of more-expensive solutions in their daily work. There was quite a stir recently around the revelation of tech companies maintaining internal leaderboards of, basically, which software engineers were spending the most on AI tools. There is certainly a correlation between cost of AI services used to build some technical artifact and the inherent challenge (or business value) of building that artifact, but we can wind up in strange places if measuring and incentivizing just the former.
Conclusion
Engineering involves navigating trade-offs between dimensions like cost, speed, and accuracy. However, the signaling phenomenon I’ve outlined pushes tech-industry participants, often without realizing it consciously, to optimize perversely for focusing on hard problems (the spirit of engineering prefers to simplify problems) and expensive solutions (while maximizing cost is rarely part of a classical engineering trade-off). Participants from CEOs to junior engineers can build status, both within their specialist communities and in society at large, by affiliating with expensive solutions to hard problems. The pull is strongest for problems and solutions that are obviously hard or expensive to a broad educated audience, and we should work to counteract that pull.
Evolution has left us with many bad habits – or ways of thinking that are clearly misaligned with modern life. For instance, we learn as kids that it is not a shrewd move to adopt the all-candy diet, even if our ancestors 100,000 years ago encountered sugar so rarely that they always won by consuming as much as they could find. Today’s engineered systems are vastly more complex than anything those ancestors dealt with, weakening the relevance of their instincts to signal success by displaying expensive solutions to hard problems. We should follow the true spirit of engineering and always appreciate a chance to simplify a problem by changing a system design at a higher level. Sometimes simplifying the world is itself an engineering challenge, and we should watch out for perverse incentives in choosing to do it, but often the investment of changing the world pays off in simpler engineering subproblems ever after.
Deep learning and friends are especially likely to excel at hard problems produced by evolution that we can work around today, as I will explore in considering how the compute stack should be different to take advantage of problems with elegant logical structure. If everything goes well, we won’t need to depend anymore on computer-hardware-world equivalents of luxury watches! It won’t hurt that we get more-reliable systems as a bonus, in a world with more of our economy handed off to AI agents in a carefully curated ecosystem.