The simplest architectural prior for improving human intelligence is to simulate what a brain would do given more compute. This is usually more time ("what conclusion would I come to given 100 years of thought"), but more brain mass seems more tractable given current technology.
Consider that models of long-term cognition get very sparse feedback. You can't finetune an LLM on your reflective trajectory across a century because that would take an entire lifetime. With high-resolution neural connections (and a robust understanding of how neurons learn), we could close a mass-based feedback loop in milliseconds-to-minutes.
Given better genetic engineering tech, we might give people a virus which 10x'ed their neuron count without terrible physiological side effects. We very probably can't do that this decade because genetic edits are hard.
Grafting biological neurons to brain surface probably just causes seizures, and cultured brain tissue won't integrate in a developmentally natural way.
But by simulating those added neurons in software, we can edit hyperparameters and connectivity based on inferred anatomy. Software gives flexibility for gradual experimentation and reversion (excluding structures the underlying biological tissue learns from implant stimulation).
If we train an AI to send similar patterns as lots of extra (developmentally integrated) neurons would, we've effectively boosted someone's brain mass. I strongly suspect that the relevant algorithms for training such an AI are compactly specifiable, since it seems like neurons learn from simple local firing statistics[1].
It's like extending brain maturation beyond 2-to-3-decade typical timelines. Existing biological circuits specify some signal similar to a gradient error, and the computronium updates commensurately, just as any section of the universally-learning parts of the brain do.
An incorrect learning rule almost certainly doesn't cause competent agentic misalignment, but it won't make a superintelligent human either. Locating the right rule seems very hard; theredoesnotseemto be consensus on how local updates are implemented.
Also, distilling human credit assignment would probably accelerate AI development; I'm supportive of open-access brain research by default, but credit assignment in particular shouldn't be published.
If we're only imputing cortical neurons. The full subcortical credit assignment system seems impenetrably complex, based on how long it's taken to map fruit fly neurons.
The simplest architectural prior for improving human intelligence is to simulate what a brain would do given more compute. This is usually more time ("what conclusion would I come to given 100 years of thought"), but more brain mass seems more tractable given current technology.
Consider that models of long-term cognition get very sparse feedback. You can't finetune an LLM on your reflective trajectory across a century because that would take an entire lifetime. With high-resolution neural connections (and a robust understanding of how neurons learn), we could close a mass-based feedback loop in milliseconds-to-minutes.
Given better genetic engineering tech, we might give people a virus which 10x'ed their neuron count without terrible physiological side effects. We very probably can't do that this decade because genetic edits are hard.
Grafting biological neurons to brain surface probably just causes seizures, and cultured brain tissue won't integrate in a developmentally natural way.
But by simulating those added neurons in software, we can edit hyperparameters and connectivity based on inferred anatomy. Software gives flexibility for gradual experimentation and reversion (excluding structures the underlying biological tissue learns from implant stimulation).
If we train an AI to send similar patterns as lots of extra (developmentally integrated) neurons would, we've effectively boosted someone's brain mass. I strongly suspect that the relevant algorithms for training such an AI are compactly specifiable, since it seems like neurons learn from simple local firing statistics[1].
It's like extending brain maturation beyond 2-to-3-decade typical timelines. Existing biological circuits specify some signal similar to a gradient error, and the computronium updates commensurately, just as any section of the universally-learning parts of the brain do.
An incorrect learning rule almost certainly doesn't cause competent agentic misalignment, but it won't make a superintelligent human either. Locating the right rule seems very hard; there does not seem to be consensus on how local updates are implemented.
Also, distilling human credit assignment would probably accelerate AI development; I'm supportive of open-access brain research by default, but credit assignment in particular shouldn't be published.
If we're only imputing cortical neurons. The full subcortical credit assignment system seems impenetrably complex, based on how long it's taken to map fruit fly neurons.