This is a special post for quick takes by Coafos. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
As I heard the current discourse about scaling laws, is that we trained the models on the entire (textual) internet, and still haven't found superintelligence. They started to use video and audio data, but these sources are less meaning-dense (encoding a picture of a room takes as much data as two books of of characters), so we could be at the edge of scaling unless a new, more data-efficient model arises.
Let's assume humans can generate 40 bit/sec meaningful data. Estimates of information density in natural spoken languages converges around this number cross-culturally, so it could be a natural processing limit in the brain. Quarter-third of the day is spent sleeping, and not every interaction is captured (yet. Distopian surveillence will increse it, but) for now let's take the half of it for an avarege day. 20 bit/sec * 86400 sec/day ≈ 210 KB/day/human. Population of 8G means 1640 TB/day. A post I found with DuckDuckGo says 402M TB/day is created on the net, which would be a 0.4% mening-to-data efficiency, and given a large amount of it is video, it is beliveable. GPT-2 was released in 2020, so let's take 5 years, this would be 2.8 EB of meaning, and given a token is 16 bit, this would be 1.4E of tokens in the past five years.
Plugging it into the Chinchilla scaling it would mean an 0.00048 loss-floot-contribution, which would be insignificant compared to the ideal generative model, which has a 1.87 lossfloor. D/N=20 means 700P of paramaters, about 1000 times larger than current models. Let's assume a 3.7x increase/year, it would mean at the end of transformers will be reached in 5-6 years. But this is some back-of-the napkin calculation, I think it's more like an upper-bound.
The (ancient) Greek form of debate or dialogue was based on the notion of common good. If ONE of the participants feel bad about it, then EVERYONE loses it. Yeah, during the dialogue the partner (opponent?) will look dumb, but afterwards they reach a conclusion, they learn something and part happily.
Trolling on the other hand is just a quick crack at the other's worldview. The point is provoking a response from other's, not educating and lifting them up. The motive of ending with MUTUAL respect is missing.
Like, dialogue is a martial art; you can fight with it but point is mastering one's own body. On the other hand trolling is pure optimization for K.O. punch. Pure optimization in the misalignment sense: The troller would even hurt/humiliate themselves if it would lead to a quicker victory.
There is the notion of singular truth in the world, but accepting this cuts back too. Two side goes into the debate, one side comes out, and they will have truth. It is of course your side. Does the debate optimizes for truth?
If the truth is independent of the participants (e.g. correct interpretation of quantum mechanics), then one (of course the other) side goes into the debate with a clearly losing position. By trying to defend their villanious statements, they have to lie about the world, and muddy the waters. With chance they could win, and only repeated debate could lead to a statistical conclusion.
If the truth is dependend on the participants, (e.g. Lighthaven's colorscheme is bad. We should paint the whole blue/green, let's have a debate on it.), the the truth doesn't even exists prior to the debate. An outcome will be reached, but will it become accepted as truth? The losing side can ask always ask for a remach, until concensus is reached.
For something to accepted by a concesus, a not warring dialogue is faster method. And while a dialogue has the goal of not necessery truth but a social concensus, by allowing the participants to freely abandon positions which they no longer hold, over time people will argue only defendible positions, which is the truth.
However, truth is only a byproduct, and power dynamics sometimes can overwrite it. Yet dialogue is the best I know. If you a quicker-faster-easier way to reach the truth, please let me know.
Platonic Dialogues which are the most famous example of Ancient Greek dialogues while certainly having a pedagogical function for the audience were polished and refined texts by writers who had the lessons they intended to impart before they began writing this. It is not a quick and easy method for the truth - it a a byproduct of having arrived at one's own truth. A literary genre. As such they a martial art (a liberal art, maybe, but not martial) - they are more like watching training film or a manual for martial art rather than being a form of oratory combat in and of themselves. (In the end of the Topics Aristotle does propose a system for training philosophical ability - but this is not a "dialogue" - it isn't intended to be written down, and he claims his school invented it: suggesting it is a totally different beast to the Platonic Dialogues or even his own Exoteric works which while lost were known to be dialogues.).
Not only that, I'm sure that in Gorgias at least there was no intent to "reach a conclusion" with his interlocutors, it is adversarial. Calllicles even calls out Socrates for laying "traps" for Gorgias - trying to get him to admit that he will teach anyone who pays, even a man who isn't trained in virtue, rhetoric. This is not because Gorgias is a poor naive soul who doesn't realize the harm of what he's doing, Socrates knows it, Gorgias knows it, even Callicles knows it. But of course the last two won't admit it because... well why would they?
I'm convinced Plato was very much intended to skewer and satirize the Sophists. Plato is actually very funny.
Why you're trying to distinguish Ancient Greek dialogues from trolling then, and then to say they are martial art is very confusing to me.
Let's say A is smarter than B if A knows about topic X, but B doesn't.
Step 1: Let's say you don't know about quantum biology. But Charlie knows, because they currently do a phd in it.
Step 2: Go to Charlie. Say: "I heard about quantum biology, and it sounds interesting. Could you give quick intro on it?"
Step 3: Charlie says (eager to talk about the cool idea they found): "Sure. Quantum biology is [two hour forty-seven minute long monologe]."
Step 4: Important! Listen to it.
Step 5: BOOM! Now you also know about quantum biology.
Repeat it for every topic. If you partition humanity along every topics, you will always be in the in-the-know part. By Zorn's lemma[1] you will be one of the smartest person in the world.
To be fair, in real life there are time and energy bounds, not everyone has time to talk about their topic, and active listening can be a hard mental work. But it worked for me a surprising amount of times. Well, surprising at first, then I adjusted my expectations.
actually you might not need Zorn's lemma for this, but it sounds so cool
It's evening, the sun is set. A man walks up to a scholar:
"Scholar, the sun rose yesterday and today morning. Will it rise again tomorrow?"
"Man, I don't know, it's kinda dark right now. Have you heard about the no free lunch theorem?"
One of my favourite Gettier-like problems is about black holes.
Say you have a very dense star. It is so dense, that the gravitational force on its surface is capable of pulling back even the particles of its light, leaving only a black hole in the sky. How large can it be with a given mass?
It's an easy exercise using Newtonian mechanics. Take a light particle with mass . Its gravitational energy at a distance is , and its kinetic energy is at the start. If the total energy is negative, then the path of the light particles will stay within a boundary. Therefore, the answer to the question is , if the object is smaller than this, then it will be a black hole.
Of course, for that dense objects, Newtonian predictions break down. We should care about curved spacetime and use general relativity in our calculations. The answer (to my knowledge) is the Schwarzschild radius, which is .