Since there's a very broad spectrum of different kinds of computer programs with different constraints and desiderata, I think the transition will be very gradual. Consider the following things that are all computer programming tasks:
I have no doubt that sufficiently fancy AI can do or help human programmers do all these tasks, but probably in different ways and at different rates.
As an experienced programmer that can do most of these things well, I would be very surprised if my skillset were substantially obsolete in less than 5 years, and somewhat surprised if it was substantially obsolete in less than 10 years. It seems like GPT-3 and GPT-4 are not really very close to being able to do these things as well as me, or close to being able to help a less skilled human do these things as well as me.
What I expect to change quickly is that "programming languages" will go away completely. LLMs or similar tech will get us way closer to the DWIM level. Directly translating from a spec to executables will be something AI can excel at. The missing piece is the feedback: writing and executing unit tests and changing the executable (not the code!) to pass the tests.
Note that a lot of current CS concepts are human-oriented and make no sense when the human is not a part of the process: "architecture" is a crutch for the limitation of our brains. "Design" is another crutch. This can all be streamlined into "Spec->Binary".
Even further, there is no reason to have a general-purpose computer when it is easy for an AI to convert a spec into actual hardware, such as FPGA.
Next on the list (or maybe even first on the list) is not needing the low-level executables at all: the LLM or equivalent just does what you ask of it.
Architecture is not JUST for brain limitations. Dividing a large task into separate testable (and formally provable now that we have AI to do the immense labor this takes) modules interconnected by message passing through shared memory with an eye towards performance is architecture.
It's not simple either, for example performance requires someone or something to have a flow graph in their head or represented somewhere to know where the bottlenecks are.
I agree with you on the idea of AI bytecode authors: once you have a program into a representation tight ...
There are architectural problems with LLMs that I think prevent the future you are describing; they can only output so many tokens, and actual binaries are often up to thousands of times the token size of the actual programming languages, especially when compiled from high level languages. The compilation process is thus a useful compression step, and I don't expect designed-for-LLM programming languages because the terabyte datasets currently necessary to train LLMs to use them won't be available. In addition, at least for the next few years, it will be important for humans to be able to inspect and reason directly about the software the LLM has made.
But it's possible these problems get solved soon.
I can foresee a near future where people can "program" by passing pseudo-code in plain English to LLMs, but I still think that we are nowhere near the point where programmer truly becomes an unskilled job. "Writing correct algorithms in plain English" is not a common skill, and you can't have LLMs writing perfect code 100% of the times. At the very least, you will still need a competent human to find bugs in the machine-generated code... my best guess is that the software industry will hire less programmers rather than less competent programmers.
I have a major problem with the framing of your question.
Say we invented construction robots that given a blueprint for a building and an auto generated list of materials deliveries, take the materials off trucks and assemble the building. This means you no longer need 'shift bosses', the computers do that. You are essentially down to 5 main roles:
You may notice that the people replaced, welders and tradesmen and crane operators etc, are less skilled than the remaining people. (not claiming blue collar work is unskilled but the time to learn to do it 'ok' enough to work independently is a few months of on the job training, with some skill gain over the years since)
This would be true for software also. The remaining people required have to have more skills. The idea of a "pointy haired boss" with no understanding of software designing a whole app that works to production scale reliability is false.
You are imagining a scenario in which computer programmers are completely automated away, rather than one where the intellectual ceiling for becoming a computer programmer is reduced and thus more people migrate to software engineering from other jobs. I don't find your scenario as plausible as my scenario but I suppose it could happen.
The main issue I see with this prediction is that 'computer programming becoming easier' has already happened before, and has not had this effect.
Programming has become easier as new languages sanded the rough edges off and automated a lot of rote Assembly work, and as widespread Internet use allowed you to find solutions fast. It was much harder to code on punch cards, or in Assembly, than to code today with StackOverflow's help.
However, this didn't lead to programming becoming less of a career, and I don't think it led to programmers being less well paid either.
There's a possibility that 'the average programmer' might become less well-paid as the definition of 'programmer' expands, but I don't anticipate a given level of programming skill becoming less valuable until AIs reach a pretty-much-singularity level.
(Disclosure: am a programmer)
As a programmer, I extensively use GPT models in my work currently. It speeds things up. I do things that are anything but easy and repeatable, but I can usually break them into simpler parts that can be written by AI much quicker than I would even review documentation.
Nevertheless, I mostly currently do research-like parts of the project and PoCs. When I sometimes work with legacy code - GPT-3 is not that helpful. Did not yet try GPT-4 for that.
What do I see for the future of my industry? Few things - but those are loose extrapolations based on GPT progress and knowledge of the programming, not something very exact:
>The consequence is the higher performance of programmers, so more tasks can be done in a shorter time so the market pressure and market gap for employees will fall. This means that earnings will either stagnate or fall.
Mostly agree with your post. Historically higher productivity has generally lead to higher total compensation but how this affects individuals during the transition period depends on the details (eg how much pent-up demand for programming is there?).
You're not accounting for an increase in demand for software. The tools to automate "basically every job on earth" are on the horizon but they won't deploy or architect themselves. Plenty of work remaining.
And there are larger jobs you are not even considering. How many people need to supervise and work on a self replicating factory or a nanoforge research facility or a city replacement effort?
There are these big huge immense things we could do that we had nothing even vaguely close to the labor or technical ability to even try. Just because humans are more efficient per hour worked doesn't mean the work won't scale up even faster.
I wouldn’t want to be getting into the software engineering business right now. I have been doing this for close to 40 years. Current systems can’t replace a senior level developer, but CharGPT is close to a junior developer. I expect that in 2-3 years, we’ll have systems that can do the work of a junior dev.
I expect that my job will become describing specs and test cases, and more of the architectural stuff. I expect to be mostly obsolete in 6 years, but maybe as long as 10 for niche solutions.
Doesn't this make you more valuable? I can understand feeling like you want to do other things in life but if you were planning to stay don't these new tools expand what you can do in a day in a way that favors someone at your skill level more than someone junior?
The most accurate answer is also the least helpful: none of us really know. Guido van Rossum has an opinion about GitHub Copilot in this interview:
but he's really just talking about what LLMs can do know, not what they'll be able to do in five or ten years.
Chris Lattner has an opinion about Software 2.0 here:
but Software 2.0 isn't really the same thing. But he's talking about Software 2.0, which is a little different. More info about Software 2.0 here:
and if you watch Chris Latter and Lex talk for a little while longer, you'll see that Chris has no idea about how you can tell a computer to build you a webpage with a red button using just text, and admits that it's out of his area of expertise.
I bring up these examples mostly to illustrate that nobody has any clue. Sam Altman addresses the topic the most out of all the people I've linked, in this video:
and the TLDR is that Lex and Sam both think LLMs can make programmers 10x more productive. Sam also thinks that instead of hiring 1/10th the number of programmers, we'll just have 10x more code. He thinks there's a "supply problem" of enough software engineers.
One thing I would advise is to make yourself more than just a software engineer. Lex says in his talk with Sam that he's not worried because he's an AI guy, not just a programmer. You might want to learn more about how AI works and try to get a job in the space, and ride the wave, or learn about information security in addition to software engineering (that's what I'm doing, in no small part because of the influence of a one-on-one chat with 80,000 Hours), or maybe you learn a lot about oceanography or data science or something else in addition to software engineering.
Then I'd also just say that we have no idea and if anyone says they know, they really don't, because look a bunch of smart people discussed it and they have no clue either.
"Computer programming" describes a pretty wide range of cognitive work. Stack Overflow has already reduced the training and knowledge required for many tasks, and LLMs push this a lot further. The next level of abstraction, understanding program flow and correctness of results hasn't seen much evidence that LLMs can plan or understand things well enough to do. And the elements of understanding user and business needs and deciding what to build in the first place hasn't really had a start.
As far as timelines, I suspect it'll transform some amount of coding into being focused on prompt generation and testing, speeding devs up by a lot, but the work will expand to fill the space rather than putting very many out of work. As we figure out how to bolt on planning, modeling, and strategic modules coordinated with LLMs, it'll move up the stack, but this is probably decades in the making.
Note that some of this coincides with "when does GPT become self-improving"?
Note gpt (or any other AI model) can become self improving while still lacking the skill to write a large and complex software package from spec.
Here's how I'm tentatively thinking about it:
I think that in general, people who work in tech will be OK as long as they're keeping up with the new LLM-based tools and ways of working.
The shift we're looking at is going from program code that's very close to a computer's inner workings to natural human language for specifying systems, but where the specification must still unambiguously describe the business interest the program needs to solve. We already have a profession for unambiguously specifying complex systems with multiple stakeholders and possibly complex interactions between its parts in natural language. It's called a legislator and it's very much not an unskilled job.
To summarize my other posts, I think the answer is "never!" because the unskilled part will not be done by humans.
Many answers contesting the basic premise of the old title, "When will computer programming become an unskilled job?" I'm fine with this and the title of the post has been updated accordingly.
"able to generate code I expect income and jobs to shift away from people with no credentials and skills to people with lots of credentials and political acumen and no skills"
What do you mean by this? What do you define as skills? In swe ing right now, "talent" means "ability to leetcode" which we already know is about to vanish as a relevant metric once someone fine tunes gpt-4 generation models on programming.
I am kinda imagining that age discrimination will matter less in coding because that 55 year old software architect with lots of credentials doesn't need anyone else for whole applications. They design the high level architecture, with more experience than anyone younger, and these architecture text files get made into code, where the design stacks the probabilities of failure in such a way that mistakes made by the AI are usually automatically found and fixed.
(This is by unit testable module design, and a different session of the same or different AI models is writing the unit tests, where you must have a double failure of a fault in the code and the unit test letting it through in order for a bug to make it to production)
So in a way this "all star" who is slower than a younger person at actually writing code, and is not up to date on the latest languages and apis, is more skilled than anyone else where it counts. But they have a good design and half their workday they spend in meetings. The AIs use whatever apis they feel like which are often the latest.
Whatever you think of current hiring practices at large companies, I think it is pretty fair to say that they are much more meritocratic than hiring practices for law positions at large law companies, for example. It is literally impossible in most industries for young highly skilled professionals to shoot to the top of the payscale based on past or tested performance. My sense is that the default for white collar positions is more like a 0.1 correlation with ability versus a 0.3 in computers.
That's a claim but is something meritocratic if it doesn't test merit at all?
Leetcode has been goodharted long past the point where what it is selecting for has any signal.
See back when the questions were more reasonable and practice sites didn't exist, you could be measuring "can someone invent an algorithm to this novel problem and code it up in an hour". That's real talent.
The question difficulty is such even for a "medium" that this is impossible. The actual original solution made by a real genius of CS at some elite university decades ago took them days to months to invent.
Anyone solving it in an hour, much less the 20 minute new standard is cheating through pattern recognition or direct knowledge of the solution.
If they tested young lawyers on ace attorney speedruns would you consider this meritocratic?
I've done a LOT of interviewing for all levels of software engineering, at big and small companies, and it's simply wrong to say it's all leetcode-style ranking. Short-duration coding challenges are a big part of most interview processes, but it's not graded the same way as competitions are. (at the better employers, at least) It's not about the right answer, it's about the explanation, follow-up questions, and understanding of the algorithm and code. And perhaps a bit about the right answer and the fluency of coding, but that's more pass/fail than requiring tons of practice. I routinely give hints to get the candidate on the right path to remember/figure out a working solution.
It's disturbing just how many applicants can get to an interview without having done minimal practice on those sites (or coding regularly in their previous job). It's a necessary part of the interview, just to weed out the non-starters. For more senior roles, the design discussions and tech dives into resume topics ("I see you've worked with distributed caching - tell me how you managed varying invalidation needs.") are more useful for final decisions.
More importantly, interview and hiring is a pretty small part of the career impact of a developer. In-role impact is also never 100% meritocratic, but at a lot of places is pretty good.
I have no clue if young attorneys are tested by actually knowing what cites to start with or how to prepare for a relevant case, but I kind of hope they are.
TL;DR leetcode-style interview coding is (or should be, if done well) satisficing, not ranking. Being competent at it is just as good (possibly better, if it lets you show other strengths) as being great at it.
TL;DR leetcode-style interview coding is (or should be, if done well) satisficing, not ranking. Being competent at it is just as good (possibly better, if it lets you show other strengths) as being great at it.
I would agree with this and upthread said basically the same thing. It's a goodharted metric. Sure, if something can do it at all that's one signal, but deciding between 10 candidates based on going to "2 mediums, 40 minutes" where only 1-2 will pass is essentially arbitrary.
The 1-2 who passed may have been better at LC than the failures, or may have been lucky this round.
Say 6 candidates finished at least 1 medium and were somewhere dealing with bugs on the second. Your test doesn't realistically distinguish between that set of 6.
Leetcode has been goodharted long past the point where what it is selecting for has any signal.
Yes, Leetcoding is essentially unrelated to actual software engineering, but isn't one's Leetcoding ability also an indirect test of intelligence + conscientiousness? I doubt that it's the entire industry standard only because it's fashionable.
Intelligence - there may be a small amount of correlation conscientiousness - no. LC is competitive programming, a sport where you memorize from a finite amount of patterns and practice sloppy methods of coding to minimize typing.
So consistent with "a small amount of correlation". The biggest correlation is competitive programming practice.
So you now have to maintain 2 skillsets - the one for the job, and a second one to remain employable.
And then each interview spend many hours "battling" for each offer, since even if you are an expert you can get unlucky or get ghosted after passing for unclear reasons that may be illegal discrimination you can't prove.
It's dumb.
But yeah, maybe not as bad as the lawyer interview process, where it's basically the reputation of your school and word of mouth and how good you are at golf or something.
I expect for there to be a delay in deployment, but I think ultimately OpenAI is aiming as a near term goal to automate intellectually difficult portions of computer programming. Personally, as someone just getting into the tech industry, this is basically my biggest near-term concern, besides death. At what point might it be viable for most people to do most of what skilled computer programmer does with the help of a large language model, and how much should this hurt salaries and career expectations?
Some thoughts:
Edit: Many answers contesting the basic premise of the old title, "When will computer programming become an unskilled job?" The title of the post has been updated accordingly.