I don't know if the metaphor is "goalpost moving" or "frog boiling" but if you dropped a top model into 2006 it would very clearly be "AGI".
I don't know how we could check who's right but I don't believe this is true at all, much less clearly true.
All depiction of AGI in fiction I know of is stronger than today's LLMs are and I think that is the central example people would compare LLMs to. I can provide some examples.
My favorite example is the sci-fi roleplaying system Eclipse Phase from 2009, as it has both AGI and non-general AI called Muses, with Muses... (read more)
Thanks for clarifying. I think "we could try what OP proposes, if it sucks it's probably not that expensive to LLM-rewrite in cleaner" is a reasonable argument.
Personally I'm just less certain than you are I feel. If I had to bet on it I would assume that the things you listed are objectively useful, not just for humans. But I'm not actually any sort of sure of it, and if a bunch good new studies came out tomorrow showing that LLMs do extremely good without linters or immutability or something I would not be surprised either.
Same with clean, good, cheap LL... (read more)
The points your making do not seem to adress the discussion I believe we're having, so it seems likely we're talking past each other.
I'll try to give a brief summary of what I believe the point of dicussion is and why I believe your points do not address it. I may be wrong in this, if so hopefully you can use this to tell me why your points are relevant.
To oversimplify a bit for brevity, I believe the chain of argument goes approx as follows
If I understand correctly, you propose that if the LLM ends up creating slop that works but is unmaintainable, then we could use that codebase as the starting point for reimplementation?
That seems plausible, but the question is, should we assume that reimplementation will be more maintainable? After all, the first codebase was presumably generated in a similar fashion and did end up unmaintainable.
I'm not confident this is true, but I can imagine a world in which reimplementing codebases larger than, say, 300k LOC by LLM results in unmaintainable slop, no ... (read more)
I've been wondering about that.
Basically it's been possible for a long time now, to not care about maintainability of your codebase and the result would be faster progress in implementing features.
This is deemed, in my view, the correct approach for some parts of the job: building a POC and writing a one-off script are two examples of tasks for which caring about maintainability is wrong most of the time, as maintainability costs significant time and effort.
There have also always been people that believed that caring about and investing in maintainability ... (read more)
I though the answer was "assume you're simulated and follow FDT to save real-you the trouble of ever getting into this situation in real life". Out of the all arguments to take the bomb this is the only one I've ever heard which I can at least understand where it's coming from.
If that is not the FDT response then, I guess I don't know why you'd ever really blow yourself up. I did read the whole post, including rereading the example against just now.
But what I got from it was mostly the insight that FDT kind of answers a different question than CDT, in that it's goal is to shape what situations you end up in, not necessarily how to get the best result out of a given situation.
I believe they way it works is that FDT tells you to make the bad decision (here: suicide) if faced with the actual situation, but the argument is that you won't get into the situation at all or less often because you're playing FDT.
By the time you find yourself facing the problem in which FDT recommends killing yourself something's already gone wrong, because the situation ought to be prevented by playing FDT in the first place.
But of course the question always is: should you actually find yourself in the decision problem, despite being an FDT-agent that ... (read more)
As an agent when you think about the aspects of the future that you yourself may be able to influence, your predictions have to factor in what actions you will take. But your choices about what actions to take will in turn be influenced by these predictions.
To achieve an equilibrium you are restricted to predicting a future such that your predicting it will not cause you to also predict yourself taking actions that will prevent it (otherwise you would have to change predictions).
Do you have an example for that? It seems to me you're descr... (read more)
(I'll admit, there's another reason for programmers to want to use AI even if it did make them worse at their jobs: it outsources some of the most unpleasant programming labor, so even if it's slower, it's worth it in the eyes of a programmer because their experience of programming feels better when they use AI because they didn't spend a lot of time doing tasks they didn't enjoy doing, like typing out the code changes they already figured out in their head.)
Basically that's proposing to take the programmer job description and move it from hands-on w... (read more)
I suppose you are right. Fiction rarely labels whether a given AI system is meant to be general or not.
There's definitely AI stronger then LLMs in fiction and AI weaker than LLMs in fiction and I cannot show that when people think of AGI they think of the examples stronger than LLMs.
So let me make another argument:
- Even careful and thoughtful people are generally bad at predicting what other people believe outside of trivial examples. See for example thoughtful Democrats being often unable to articulate accurately what thoughtful Republicans believe and vi
... (read more)