Hey, very interesting post!
I enjoyed the "become a superintelligence" framing / thought experiment. I am not sure I fully believe the "armed balance of power" b/w Superintelligences part as stated, but I can imagine something similar I might agree with.
Ultra charged human intelligence and IA are both interesting areas to me. I think writing and database search are both interesting examples of IA information technologies to use as a foil to a world of AI servants.
I couldn't really understand what Crystallect was doing/going for, but I liked your discussion of programming. I wish you all the best with it. I am reminded of Conjecture because of the IA and custom programming language design. It is different though; they have an attempt going to make a special programming language to harness LLM intelligence in a much more "code like" and "human accessible" way.
"Crystallect" stands for the crystal transparency and order in empowering the user’s intellect in contrast to the black box of neural-network-based Artificial Intelligence.
Kids in school, as I remember it, had two ways for getting good grades: to study hard or to bully some nerd kid into doing their homework. The second seemed more rational in the short-term. And sometimes in the long-term too, because some of those bullies learned soft skills and became managers of those hard-studying nerds.
Specialization and division of labor made it common sense to think this way: If something seems hard for you, let someone else do the job. And even the most nerdy among us like the idea of Artificial Superintelligence solving all problems for them if they believe they can manipulate it into compliance.
I believe, they can't. LessWrong contains plenty of materials on the problem of AI Alignment, so I'll not elaborate on that. Instead, I wanted to talk about the possibility of the first way: to study hard and become a Superintelligence yourself, however unrealistic it may sound. Let us fantasize first and discuss technicalities later to see where we, as a humanity, may be heading.
Imagine an ultimate picture of you with your brain interfaced with some high-tech increasing your intellectual capabilities and data throughput. Such intellectual capabilities that you could contemplate quantum gravity theories or analize the genome of your cat. Such data throughput that you could manage in real time swarms of combat drones in the battlefield like it were your own fingers. Or you could manage all cyber-servants of your international restaurant chain as if you were the every one of them.
I've mentioned military application for a reason. New technologies give an edge in the power struggle, and thus attract power players. Intelligent people usually avoid direct participation in conflicts. They invent weapons but let harsh military men do the dirty job. This will be not the case anymore.
Such technology eliminates the need for specialization and labor division. If you have not enough resources to solve some problem and need the help of your neighbour, you may as well devour his resources and do the job yourself. This sounds pretty dangerous, isn't it? Like in the "Highlander" movie, leading to the survival of a singular, most aggressive superintelligence.
The hope for humanity is in simultaneous emergence of multiple superintelligences that will keep an armed balance of power. They will be smart enough to understand that peace is the best strategy and will be uniting to fight off aggressors. Just like in primitive society, but without specialization being the main drive for cooperation.
Such power transition won't happen overnight, I think. Smart people will be slowly improving technologies that will let them get even smarter and gain more financial and political power. Military and police forces will rely more on high-tech and get rid of stupid and violent types among them. There are setbacks, but in the long run, society evolves for the better: the most violent tyrants of the present are refined intellectuals by the standards of ancient times.
Prehistoric humans relied on muscular strength in their struggle for survival. Modern people exert their muscles for fun and health, but for practical goals, they prefer relying on tools and machines.
There are also tools to enhance intellectual capabilities. Early entrepreneurs had to memorize relations with all counterparties and could not scale their business. Paper became a kind of external memory which released the brain for more creative tasks. And database management systems allow manipulating huge amounts of well-structured data with a single query that before could take many human-years to process.
You may also think of Large Language Models as another tool for enhancing your intelligence. But allow me to set it aside for now. It is on the borderline with Artificial Intelligence, alternative for which I'm trying to discuss here. And it is also limited by the language. And that limitation I'll elaborate at length.
Four years ago, upon receiving a hefty award money from an InnoCentive challenge, I felt like I deserved some me time. I always wanted to learn quantum physics to the level of familiarity with the Standard Model. In other areas, I could pick things here and there and invent something meaningful at the intersection of fields. But not so with quantum physics. The thing is enormously huge for comprehension.
So, I gave it another try, and dived into learning full on. I was reading one textbook until couldn't follow the narrative, then switching to another and so on. After several months of stupid memorization, I started noticing certain patterns but had no means to express them, and realized that memorizing them would take even longer. I've got a vague idea how to help that, and it prompted a vision of a project worthy of a lifetime commitment. So I put physics aside and started pondering about patterns.
It's not enough just to memorize several textbooks. To master a new knowledge, you must meditate on it by practice or any other way. Doing so, you let your brain notice patterns and correlations between them. It generates and validates multiple theories of its own without you even noticing that. Textbook authors and university professors only try their best to help students build and synchronize their own models of the science in their brains.
The same applies to social wisdom and Emotional Intelligence as well. It took a significant portion of my life to learn things that seem so obvious now. Others could have been trying to explain those to me, but that knowledge remained a dead weight until my own experience highlighted correlations with my own model of life. I think literature would allow to speed this process up, e.g. by simulating various experiences via empathy with the heroes. If only I were a more frequent reader. A talented writer knows how to evoke associations with deep feelings or ideas that readers themselves couldn't express. Some people like inventing new terms for such ideas, but only a few of those go viral and become part of the language.
Common languages, whether literary or mathematical, have such limitation: There are
too many writers who must synchronize introduction of new terms(notations) between
themselves and readers who have to memorize them all.
As an old programmer, of course I don't take seriously such limitations. While programming, we are free to make up new identifiers for entities in our code. Name scopes and interfaces allow us to choose what to synchronize with others and how. While coding, I naturally come up with many new ideas which I immediately implement in the code. Without a better alternative, the program code itself becomes an external memory for the mind to keep the theory of the program. When the theory needed a slight modification, I could routinely change many lines of the code, but succeed without adding a single bug.
It was an obvious choice to dig into this experience. To make a tool for programming while trying to synchronize with the user's thinking process.
Twenty years ago I was playing around with inventing programming languages, but always arriving to the conclusion that I could do what I wanted in the same C++ by only changing the patterns of my programming style. However, there was a vague idea how to do the same in a form of logic programming that would compile to C. And finally, twenty years later, I implemented that. Not quite how I imagined then, but I did it. I just hope, it will not take another 20 years to implement that vague idea I've mentioned above. Well, 3 years already have passed, 17 to go.
And the name I have given this thing is also from the twenty years ago. (Even had the domain crystallect.ru) Like Intel taking the beginning of the word intellect, I took the ending and added word crystal as a symbol of transparency and order. Was surprised that no one still adopted such a beautiful blend.
Sentiments aside, it is available as free software at https://www.nongnu.org/crystallect
It is still far from what I intended it to be. Users can use the current version for designing smaller algorithms like those for AVL-tree from the example, but it is not suitable for anything bigger yet. I suppose I can call this a proof-of-concept.
The next major milestone is to express the entire Crystallect theory on its own platform, similar to rewriting a compiler in the very language it compiles from.
This will be a continuous process. I will use old code as an external library and step by step replace it with proofs of the growing theory. Crystallect will encounter multiple growing pains and difficulties of refactoring, which will be the perfect roadmap for its evolution. I will implement new features either directly into the theory or into C++ code to be ported later, whichever will be easier at the moment. I will abstain sidetracking into development of new features for their own sake until those bring the completion of the stage closer. I estimate this taking 1 to 3 years of full-time work.
As a result, we will get a robust version, suitable for the development of bigger projects and quite capable of refurbishing. A platform for thinking that fully describes itself - lives up to the name.
What's next? It will heavily depend on the research findings I'll make. This is only the beginning of the real research.