Tangential thought: Why are the released models even trained to perform exploitation at all? Wouldn't adversarial attacker-defender training and then throwing away the attacker side be safer? The model would still have to gain "security mindset" to compete, but hopefully without the monomaniacal drive to work around any and all obstacles.
Anything along the curved upper boundary of the [CIE horseshoe](https://en.wikipedia.org/wiki/CIE_1931_color_space) (but not the [line of purples](https://en.wikipedia.org/wiki/Line_of_purples)) is a monochromatic pure additive color. You can construct triangular colorspaces by picking any three. But to get a large gamut you need to pick something in the blue-purple corner, one in the red and then from the large range greens. And if you want yellows covered you need to pick from greens on the right flank. That's also roughly where the sensitivity peaks of ... (read more)
We might also be using working memory to reconstruct reverse relations on the fly. E.g. reciting a poem backwards will consist of remembering chunks of it in forward direction and then rearranging the chunk to be in reverse direction.
If that is correct than a variation of CoT-prompting might work. By first having the model recall any context in which it recalls an object and then picking the answer out of that.
I wonder how much camp #1 correlates with aphantasia.
The thoughtexperiments suggest that qualia is tied to memory-formation. If your nociceptors are firing like crazy but the CNS never updates on it, was there any pain?
Then the obvious next question is what distinguishes qualia from memory-formation?
There are can order at least 10k-basepair DNA synthesis online, longer sequences are "call to get a quote" on the sites I found. The smallest synthetic genome for a viable self-replicating bacterium is 531kb. The genome for a virus would be even smaller.
My understanding is that there are existing processes to encapsulate genes into virus shells from other species for gene therapy purposes. That leaves the logistics of buying both services, hooking them up and getting the particles injected into some lab animals.
It doesn't look trivial, but less complicated than buying an entire nuclear arsenal.
What updates on your predictions, if any, would you make if we learned that the variant developed from a long-term infection in an immunocompromised person, i.e. had plenty of opportunity to adapt to the immune response.
It's more about the "work around any and all obstacles" part rather than the "monomaniacal drive". You do need that level of paranoia and persistence to find all flaws that a tireless attacker will use anyway.
My intuition is that training defense would make it easier to write reward functions that encourage operator-intended holistic behavior.
The attacker only cares about the flag, no matter how it gets there. The defender can be graded on how many already-known and attacker-discovered flaws it fixes, whether the testsuite keeps working and various other e... (read more)