I think just showing would be better than reconstruction score metric because is very noisy.
there is a validation metric called reconstruction score that measures how replacing activations change the total loss of the model
That's equivalent to the KL metric. Would be good to include as I think it's the most important metric of performance.
Patch loss is different to L2. It's the KL Divergence between the normal model and the model when you patch in the reconstructed activations at some layer.
It would be good to benchmark the normalized and baseline SAEs using the standard metrics of patch loss and L0.
What is Neuron Activity?
Does anything think this could actually be done in <20 years?
materialism where we haven't discovered all the laws of physics yet — specifically, those that constitute the sought-for materialist explanation of consciousness
It seems unlikely that new laws of physics are required to understand consciousness? My claim is that understanding consciousness just requires us to understand the algorithms in the brain.
Without that real explanation, “atoms!” or “materialism!”, is just a label plastered over our ignorance.
Agreed. I don't think this contradicts what I wrote (not sure if that was the implication).
The Type II error of behaving as if these and future systems are not conscious in a world where they are in fact conscious.
Consciousness does not have a commonly agreed upon definition. The question of whether an AI is conscious cannot be answered until you choose a precise definition of consciousness, at which point the question falls out of the realm of philosophy into standard science.
This might seem like mere pedantry or missing the point, because the whole challenge is to figure out the definition of consciousness, but I think it is actually the central issue. People are grasping for some solution to the "hard problem" of capturing the je ne sais quoi of what it is like to be a thing, but they will not succeed until they deconfuse themselves about the intangible nature of sentience.
You cannot know about something unless it is somehow connected the causal chain that led to the current state of your brain. If we know about a thing called "consciousness" then it is part of this causal chain. Therefore "consciousness", whatever it is, is a part of physics. There is no evidence for, and there cannot ever be evidence for, any kind of dualism or epiphenomenal consciousness. This leaves us to conclude that either panpsychism or materialism is correct. And causally-connected panpsychism is just materialism where we haven't discovered all the laws of physics yet. This is basically the argument for illusionism.
So "consciousness" is the algorithm that causes brains to say "I think therefore I am". Is there some secret sauce that makes this algorithm special and different from all currently known algorithms, such that if we understood it we would suddenly feel enlightened? I doubt it. I expect we will just find a big pile of heuristics and optimization procedures that are fundamentally familiar to computer science. Maybe you disagree, that's fine! But let's just be clear that that is what we're looking for, not some other magisterium.
If consciousness is indeed sufficient for moral patienthood, then the stakes seem remarkably high from a utilitarian perspective.
Agreed. If your utility function is that you like computations similar to the human experience of pleasure and you dislike computations similar to the human experience of pain (mine is!). But again, let's not confuse ourselves by thinking there's some deep secret about the nature of reality to uncover. Your choice of meta-ethical system is of the same type signature as your choice of favorite food.
The subfaction veto only applies to faction level policy. The faction veto is decided by pure democracy within the faction.
I would guess in most scenarios most subfactions would agree when to use the faction veto. Eg. all the Southern states didn't want to end slavery.
If this is accurate then I agree that this is not the same as "the KL Divergence between the normal model and the model when you patch in the reconstructed activations". But Fengyuan described reconstruction score as:
which I still claim is equivalent.