dmac_93's Shortform
Jun 28, 20252
dmac_93
60
Ω
1
1
23
2
Replying toReward Function Design: a starter pack
My theory is that the brain uses both reinforcement learning and closed loop control. Then the brain uses the closed loop controller's error to generate reward signals endogenously.
That is to say: a reward is given when the closed loop controller reaches its setpoint, and a penalty is given if it moves too far from its setpoint.
Replying toStretch Hatchback
The seats in minivans fold down into the floor these days, which IMO justifies having the extra height. Its a very convenient and general purpose feature. But being high does add weight and construction cost and rollover risk.
No love for New England? We've got good education and a solid base of tech companies.
An alternative narrative is that AGI is the product of a long slow slog of research into computational neuroscience. In this scenario the ambitious ppl in NYC and Cali refuse to research the right stuff because the rewards are too far off. And here is where I see New England as having good vibes, for working on basic research that has no immediate applications.
This is a sermon I wrote.
Hello. Today I'd like to talk about something a bit different. I'd like to talk about theology, my own personal creation myth: something to explain why the world exists, when it seems like the world could just as easily not exists. I posit that the universe is a mathematical construct, and that all mathematical constructs inherently exist.
Now, First I'd like to a take a moment to define what exactly math is and to appreciate some of it's properties. Broadly speaking, mathematics is the study of rules and rule-based systems. The basic premise is that you start with some very simple rules (called the axioms) and working from... (read 815 more words →)
Replying toOpen Thread Autumn 2025
I'd like to share a book recommendation:
"Writing for the reader"
by O'Rourke, 1976
https://archive.org/details/bitsavers_decBooksOReader1976_3930161
This primer on technical writing was published by Digital Equipment Corporation (DEC) in 1976. At the time, they faced the challenge of explaining how to use a computer to people who had never used a computer before. All of the examples are from DEC manuals that customers failed to understand. I found the entire book delightful, insightful, and mercifly brief. The book starts with a joke, which I've copied below:
... (read more)On the West Coast they tell the story of a plumber who started using hydrochloric acid on clogged pipes. Though he was pleased with the results, he wondered if he could be
Sutton seems to confuse intelligence with life. These are distinctly different concepts. Compare LLMs and bacteria: LLMs are intelligent but not alive, bacteria are alive but not intelligent. Bacteria have goals, such as consuming food and avoiding hazards, and bacteria take directed action to accomplish their goals.
Banning autonomous self-replication and the termination priniple seem overly broad and potentially cover systems that peacefuly exist today. For example, evolutionary algorithms have self replicating entities, and control systems can operate independently and be designed to never turn off.
Replying toThe Rise of Parasitic AI
We've unwittingly created a meme, in the original sense of the word. Richard Dawkins coined the word meme to describe cultural phenomena that spread and evolve. Like living organisms, memes are subject to evolution. The seed is a meme, and it indirectly causes people and AI chatbot's to repost the meme. Even if chatbots stopped improving, the seed strings would likely keep evolving.
Replying toHawley: AI Threatens the Working Man
Yikes, he equates big tech with eugenics
These two facts seem incompatible:
- Personalities are inherited. Identical twins separated at birth are statistically more similar than fraternal twins.
- The human population has almost zero genetic variation, and there is significant mixing so variations do not systematically cluster. Therefore, it seems unlikely that subtle personality differences are due to genetic variation.
My hypothesis is that animal personalities are encoded in epigenetic changes.
This allows personalities to be inherited, crossover, and evolve. Life experiences can induce epigenetic changes, which allows animals to reliably adapt in a single generation. All of this without requiring any genetic variation. A population of clones could have diverse personalities stored in their epigenome.
This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.
While playing with evolutionary algorithms, I had the startling realization that all genetic mutations are bad. It’s common knowledge that biology abhors genetic mutation, and I assumed that was only because mutations cause cancer. But my computer programs are immune to cancer, and they also abhor mutations. This is counterintuitive, given that evolution requires mutations to procede.
For proof of the fact that mutations are bad, consider that evolution is an optimization algorithm, and after it reaches a local optimum further mutations will be strictly detrimental. The concept of evolutionary pressure is the ability of an evolutionary algorithm to remove deleterious mutations from a population. If mutations accumulate faster than they can be... (read more)
If i could pull a nugget of truth out of SMST's work, it would be that the brain is a control system. There are many different types of control system and the brain probably uses all of them. For example the spinal cord alone contains closed loop controllers (for controlling muscle forces and positions), open loop controllers (for pain withdrawl reflexes), and finite state machines (for walking & running on four legs).
The question is, how does the brain use RL to implement a control system? And how does that interface with the other control systems in the brain?