Over the last few posts, I talked about expansion of general human processing power by growing an AI to do whatever computations extra biological neurons would. This is powerful and flexible, but also has bigger value-drift risk and is probably technically harder than other approaches.
There might be technically simpler ways of augmentation. It does seem that evolution made modular structures; perhaps subcomponents of human cognition are similarly separable.
For example, individual neurons correspond to individual episodic memories[1], and visual and tactile stimuli are ~topologically continuously mapped to brain surface. Maybe working memory has similar regularities?
People can use mnemonics to get extraordinarily good at remembering simple structures like digits, but those don't scale to the sort of abstract concepts a technical AI safety researcher would find useful. We need better flexibility.
One great thing about computers is that, unlike humans, if you give a Western Digital SN8100 a 20 million digit vector[2], it sticks. If you need a 20MB sparse connectivity matrix to represent a single abstract thought, that's fine, your server has a terabyte of VRAM.
There's a mesh of neuron tails which sits in the outermost micrometers of cortical tissue. It's called neuropil.
Unlike other long-range brain connectors, neuropil has quite slow conduction velocities; it's unmyelinated, so signals travel slowly and are more metabolically expensive to send. Evolution found unmyelinated neuropil worth the processing lethargy and ATP costs; why?
Time coding could be a good area to look, since temporal data is really important for neuron semantics. Timing improves the decoding of macaque working memory items on spatially complex tasks, for example. And physical distance is codedas temporal difference, in memory cells.
As a further sign of time-coding's relevance, signal delay in neuropil causes large, diffuse waves which spread across brain surface[3]; a difference in conduction velocities alone is sufficient to create large-scale oscillations in what would otherwise be a pretty chill system.
I fear I should first explain FM radio.
Constructive interference, where multiple signals combine into a single loud signal, looks like this:
when you sum across separate signals. This is how we get musical chords.
We can also loop one signal and see how it interferes with itself; by tuning the loop frequency, we can isolate the power of a carrier signal.[4]
If we smooth the summing process (added EMA parameter), we can trade spectral for temporal precision; the receiver is less sensitive to frequency changes, but can come to a conclusion faster.
By turning the carrier on and off, we can encode and decode lower-frequency signals. This is how AM (amplitude modulation) radio works.
Shifting the carrier frequency likewise changes how well it sums on the receiver, and we can use it to encode signals too.
In this last example, you can see that "frequency" modulation actually looks to the receiver like phase modulation; slightly bumping the frequency causes each new beat to anticipate its rolling average. Harder anticipation quickly flattens the received power.
For neurons, if we had multiple simultaneous carrier frequencies (bands), we could correlate firing activity so that sub-concepts congealed into bigger and more abstract ideas. Any parts of the brain oscillating in the same band would, over time, get lots of activity from the conglomerate concept and average close to nothing for unrelated ones.
Let's say that some part of my brain represents my dog, some other part represents sensations of running, and a third holds how sunlight filters through trees. Three modular circuits.
When I imagine trail running with my dog, the three circuits "bind" together on a shared carrier wave. If you "tune your radio" to that carrier wave, you get exactly the signal "sunny dog visuals, proprioception of running".
The timing information here is flexible in a way that the underlying physical circuits can't be. Flexible time codes are probably in some strong sense required for holding things in working memory[5]; if working memory entries come with large-scale brainwaves, it's probably pretty easy to locate which patterns are objects in working memory.
One might say that large-scale waves hold the control information about representations; wave timing reorganizes existing primitives into new concepts.
By scanning for characteristic control waves, we can infer that <pattern> is something in memory; then gently re-stimulate <pattern> while the expensive large-scale biological waves address other objects for working memory.
But just re-stimulating this sort of neural contraption, without extra control circuitry, is probably not nearly as helpful as one might hope if they heard "10x native working memory capacity". Native implies finer control.
As mentioned, we want to not restrict what thoughts could be computed by a BCI. So one might think "native addressing" means "never decoding to explicit control actions"; that we should find a language where memory control macros (CRUD) are "learned" by the digital component of the augment, just as neurons learn from nearby neurons.
I don't think it must be so. Indeed, memory control seems low-dimensional; we need only solve create, read, update, and delete. "Read" is implicit if we're replaying a representation (though perhaps we'd want a command for say it louder), so we're left with create-update-delete.
Existing low-bandwidth neurotech is fantastic at decoding low-dimensional features of this sort. We could train a model to decode brainwave-coded control information, which tells us where to find memory entries; if <dog> and <forest> are harmonically coupled in a way that they normally aren't, we can tell <dog in forest> is a memory entry without knowing what <dog> or <forest> mean.
Since large-scale brainwave data like this is low-dimensional, we have very few degrees of freedom, so fitting a decoder is extremely cheap.
In the best case, we wouldn't even need to train a decoder because brainwaves straightforwardlycarry control information about sub-concepts. For example, if 20-30hz waves mean DELETE.
Well, that was the hope, but while writing this I found that we probably don't have an OOM of free native-memory lunch (see [4]).
We might still link a sort of "embedding search" over cached WM contraptions, where SEARCH is ~the only macro, or any other query scheme; but this isn't native in the same sense; explicit queries impose a big extra cost.
We'd ideally have something which monitors activity and replays <pattern> when <pattern> is predicted by some system to be relevant; i.e. SEARCH is implicit. The training signals for this could be way sparser than simulating extra brain matter, especially if harmonic similarity is a strong inductive bias.
And/or maybe credit assignment for rationality-important thoughts? Inasmuch as items in working memory describe ongoing cognition, reactivating patterns once you've realized they were reasoning errors could help metacognition.
Segmenting items from working memory is almost certainly upstream of accelerated skill acquisition, which I'll cover in the next post of this sequence.
I'm around 30% confident that SCT-based working memory augmentation alone would be extremely productive. Seems worth exploring, but not as much as raw compute.
In real physical radios (and interferometers etc) the antenna converts the electromagnetic wave into currents which still carry the harmonics you'll notice if you set the carrier to, say, 880hz and the receiver to 440. I'm not a radio expert, but from a conversation with GPT, it seems to me that integer overtones are very hard to filter. This, along with the temporal-spectral uncertainty tradeoff linked above, would physically limit how many items could stay in working memory for any response time; I hadn't considered Fourier uncertainty as a potential limiter until writing this all out, so I tested it.
If we allow 20-40hz addressing (one octave to avoid harmonic issues), the maximum working memory capacity scales linearly with response time, at about 6 items per 100ms decode window (at 99.6% decode accuracy).
But this only accounts for boolean "are we bound or not?"; phase also very likely encodes semantic relationships. Each bit we add halves the number of possible carriers, so our information throughput is exactly identical in expectation across all configurations; decoding time and signal-to-noise ratio (which is also linear) fully describe our input.
I don't think anyone will measure phase-modulated working memory decode time with existing tech. Absent such data, humans seem pretty close to the Pareto frontier of ~30 binding bits per 500ms, especially if binding data carries >1bit per concept.
Speculation.
Over the last few posts, I talked about expansion of general human processing power by growing an AI to do whatever computations extra biological neurons would. This is powerful and flexible, but also has bigger value-drift risk and is probably technically harder than other approaches.
There might be technically simpler ways of augmentation. It does seem that evolution made modular structures; perhaps subcomponents of human cognition are similarly separable.
For example, individual neurons correspond to individual episodic memories[1], and visual and tactile stimuli are ~topologically continuously mapped to brain surface. Maybe working memory has similar regularities?
People can use mnemonics to get extraordinarily good at remembering simple structures like digits, but those don't scale to the sort of abstract concepts a technical AI safety researcher would find useful. We need better flexibility.
One great thing about computers is that, unlike humans, if you give a Western Digital SN8100 a 20 million digit vector[2], it sticks. If you need a 20MB sparse connectivity matrix to represent a single abstract thought, that's fine, your server has a terabyte of VRAM.
There's a mesh of neuron tails which sits in the outermost micrometers of cortical tissue. It's called neuropil.
Unlike other long-range brain connectors, neuropil has quite slow conduction velocities; it's unmyelinated, so signals travel slowly and are more metabolically expensive to send. Evolution found unmyelinated neuropil worth the processing lethargy and ATP costs; why?
Time coding could be a good area to look, since temporal data is really important for neuron semantics. Timing improves the decoding of macaque working memory items on spatially complex tasks, for example. And physical distance is coded as temporal difference, in memory cells.
As a further sign of time-coding's relevance, signal delay in neuropil causes large, diffuse waves which spread across brain surface[3]; a difference in conduction velocities alone is sufficient to create large-scale oscillations in what would otherwise be a pretty chill system.
I fear I should first explain FM radio.
Constructive interference, where multiple signals combine into a single loud signal, looks like this:
when you sum across separate signals. This is how we get musical chords.
We can also loop one signal and see how it interferes with itself; by tuning the loop frequency, we can isolate the power of a carrier signal.[4]
If we smooth the summing process (added EMA parameter), we can trade spectral for temporal precision; the receiver is less sensitive to frequency changes, but can come to a conclusion faster.
By turning the carrier on and off, we can encode and decode lower-frequency signals. This is how AM (amplitude modulation) radio works.
Shifting the carrier frequency likewise changes how well it sums on the receiver, and we can use it to encode signals too.
In this last example, you can see that "frequency" modulation actually looks to the receiver like phase modulation; slightly bumping the frequency causes each new beat to anticipate its rolling average. Harder anticipation quickly flattens the received power.
For neurons, if we had multiple simultaneous carrier frequencies (bands), we could correlate firing activity so that sub-concepts congealed into bigger and more abstract ideas. Any parts of the brain oscillating in the same band would, over time, get lots of activity from the conglomerate concept and average close to nothing for unrelated ones.
Let's say that some part of my brain represents my dog, some other part represents sensations of running, and a third holds how sunlight filters through trees. Three modular circuits.
When I imagine trail running with my dog, the three circuits "bind" together on a shared carrier wave. If you "tune your radio" to that carrier wave, you get exactly the signal "sunny dog visuals, proprioception of running".
This is what an MIT-Sweden collaboration terms "spatial computing theory".
The timing information here is flexible in a way that the underlying physical circuits can't be. Flexible time codes are probably in some strong sense required for holding things in working memory[5]; if working memory entries come with large-scale brainwaves, it's probably pretty easy to locate which patterns are objects in working memory.
One might say that large-scale waves hold the control information about representations; wave timing reorganizes existing primitives into new concepts.
By scanning for characteristic control waves, we can infer that <pattern> is something in memory; then gently re-stimulate <pattern> while the expensive large-scale biological waves address other objects for working memory.
But just re-stimulating this sort of neural contraption, without extra control circuitry, is probably not nearly as helpful as one might hope if they heard "10x native working memory capacity". Native implies finer control.
As mentioned, we want to not restrict what thoughts could be computed by a BCI. So one might think "native addressing" means "never decoding to explicit control actions"; that we should find a language where memory control macros (CRUD) are "learned" by the digital component of the augment, just as neurons learn from nearby neurons.
I don't think it must be so. Indeed, memory control seems low-dimensional; we need only solve create, read, update, and delete. "Read" is implicit if we're replaying a representation (though perhaps we'd want a command for say it louder), so we're left with create-update-delete.
Existing low-bandwidth neurotech is fantastic at decoding low-dimensional features of this sort. We could train a model to decode brainwave-coded control information, which tells us where to find memory entries; if <dog> and <forest> are harmonically coupled in a way that they normally aren't, we can tell <dog in forest> is a memory entry without knowing what <dog> or <forest> mean.
Since large-scale brainwave data like this is low-dimensional, we have very few degrees of freedom, so fitting a decoder is extremely cheap.
In the best case, we wouldn't even need to train a decoder because brainwaves straightforwardly carry control information about sub-concepts. For example, if 20-30hz waves mean DELETE.
Well, that was the hope, but while writing this I found that we probably don't have an OOM of free native-memory lunch (see [4]).
We might still link a sort of "embedding search" over cached WM contraptions, where SEARCH is ~the only macro, or any other query scheme; but this isn't native in the same sense; explicit queries impose a big extra cost.
We'd ideally have something which monitors activity and replays <pattern> when <pattern> is predicted by some system to be relevant; i.e. SEARCH is implicit. The training signals for this could be way sparser than simulating extra brain matter, especially if harmonic similarity is a strong inductive bias.
And/or maybe credit assignment for rationality-important thoughts? Inasmuch as items in working memory describe ongoing cognition, reactivating patterns once you've realized they were reasoning errors could help metacognition.
Segmenting items from working memory is almost certainly upstream of accelerated skill acquisition, which I'll cover in the next post of this sequence.
I'm around 30% confident that SCT-based working memory augmentation alone would be extremely productive. Seems worth exploring, but not as much as raw compute.
Here's a good paper for the theoretical background.
Worst bedtime story ever
See here (or here if the free link breaks) for a broader treatment.
In real physical radios (and interferometers etc) the antenna converts the electromagnetic wave into currents which still carry the harmonics you'll notice if you set the carrier to, say, 880hz and the receiver to 440. I'm not a radio expert, but from a conversation with GPT, it seems to me that integer overtones are very hard to filter. This, along with the temporal-spectral uncertainty tradeoff linked above, would physically limit how many items could stay in working memory for any response time; I hadn't considered Fourier uncertainty as a potential limiter until writing this all out, so I tested it.
If we allow 20-40hz addressing (one octave to avoid harmonic issues), the maximum working memory capacity scales linearly with response time, at about 6 items per 100ms decode window (at 99.6% decode accuracy).
But this only accounts for boolean "are we bound or not?"; phase also very likely encodes semantic relationships. Each bit we add halves the number of possible carriers, so our information throughput is exactly identical in expectation across all configurations; decoding time and signal-to-noise ratio (which is also linear) fully describe our input.
I don't think anyone will measure phase-modulated working memory decode time with existing tech. Absent such data, humans seem pretty close to the Pareto frontier of ~30 binding bits per 500ms, especially if binding data carries >1bit per concept.
This is a stronger claim than I've seen SCT researchers make.