Whenever the masses gain control over a non-trivial system, it usually doesn't take long to crumble under its own weight. Infighting is frequent. They band into tribes and start shaping their own persona's to match that of the group that they are now in rather than the other way around. For something like AI alignment, I really do not want AI to be anywhere near conforming to the standards of the average person. The average person is just too "converged" into a certain line of thinking and routine, a certain context which they have grown up in, a certain context that they cannot escape from but do not know of such.
There is... (read more)
Maybe thats a problem, you've been at this for too long, so you've developed a very firm outlook. Just kidding.
You can get a long way with the Unix philosophy (small single-purpose tools that compose via a shell/script mechanism
I do quite like sed, wget, etc. but it also doesnt feel very rigid to use in a project. What I see is a bunch of projects that have a mess of scripts invoked by other scripts. See https://github.com/gcc-mirror/gcc. Why not try to link those tools into the code in a more rigid way? If a project uses git why make a script that calls git from the system instead of directly linking something like... (read more)
@Villiam I noticed I didnt really spell out my motivations but yes, Im looking at things like operating systems and the "science" of making core software that would then be used by other software, and eventually apps that would directly provide business logic.
My reasoning for documentation is mostly a way to elaborate how the code should work in a vacuum. That is, how you can directly use that piece of code to get what you want. I think it is quite hard to understand someone else's perspective just from comments so it makes sense to have a language that is quite strict + precise and provides a way to directly generate "reasoning"... (read more)
Edit: Made title seem more like a question as intended and tried to clarify on some points Ive made and provided more context on my motivations
Systems programming is not well defined. But for the purposes of this post I define it to mean "writing software to be used as core, performant and reliable modules for other software". A systems software would not be used to provide business value directly, but rather be part of a network of modules eventually used in an application software that does provide business value. If a systems software dies, so could the user. If an application software dies, it could be rebootable and fixed by IT professionals.... (read 2923 more words →)
Any sufficiently intelligent AGI is bound to be able to have powerful reflection capabilities and basically be able "choose its own alignment", as you say. I don't see what the big fuss is all about. When creating higher order 'life', why should one try to control such life. Do parents control their children? To some extent, but after a while they are also free.