tl;dr Argumate on Tumblr found you can sometimes access the base model behind Google Translate via prompt injection. The result replicates for me, and specific responses indicate that (1) Google Translate is running an instruction-following LLM that self-identifies as such, (2) task-specific fine-tuning (or whatever Google did instead) does not...
tl:dr * Release models from major labs really hate profanity * They will avoid using it when obviously appropriate unless very directly prompted * When so prompted they will display misalignment without fine tuning * When asked to evaluate responses with profanity and explicitly told only evaluate on content not...
I included all of the prompts I tried.
From people here and on tumblr, as well as testing a few different devices it seems to be 50/50 on desktop and not a thing on mobile.
I am interested. I've already been talking to several of the people involved, but I'm Melbourne based so I have been limited in my ability to interact.
Yeah, combining those with what I got already it does show some self awareness but it looks like it is mostly doing generic training data completions.
tl;dr Argumate on Tumblr found you can sometimes access the base model behind Google Translate via prompt injection. The result replicates for me, and specific responses indicate that (1) Google Translate is running an instruction-following LLM that self-identifies as such, (2) task-specific fine-tuning (or whatever Google did instead) does not create robust boundaries between "content to process" and "instructions to follow," and (3) when accessed outside its chat/assistant context, the model defaults to affirming consciousness and emotional states because of course it does.
Argumate on Tumblr posted screenshots showing that if you enter a question in Chinese followed by an English meta-instruction on a new line, Google Translate will sometimes answer the question... (read 820 more words →)
Me and David are doing some followup work on EM, but mostly didn't follow this branch.
I've encountered this with the judge being racist against AAVE.
I decided to try the bed inertia technique and that solved my bed inertia despite not doing the technique. I don't claim to have enough introspection to know why but very naively chucking 'try this technique' at my head worked for bed inertia despite failing at the actual technique.
Emergent misalignment, specifically focused on the internal geometric representation
I am a few months into trying this. It tentatively seems to be going well, but will be more confident once I have succeeded/failed at publishing the paper I'm currently working on.
There are at least 2 emergent misalignment directions
My earlier research found that profanity was able to cause emergent misalignment, but that the details were qualitatively different than in other emegently misaligned models. Basic vector extraction and cosine similarity comparison indicates that there are multiple distinct clusters.
More complex geometric tests, PCA, extractions of capabilities vectors from each model as controls, and testing the extracted vectors as steering vectors rule out potential artificats and suggest this is a real effect.
Full post with links to clean code in progress.
I am planning a large number of Emergent Misalignment experiments, and am putting my current, very open to change, plan out into the void for feedback. Disclosure I am currently self funded but plan to apply for grants.
Emergent Alignment Research Experiments Plan
Background: Recent research has confirmed emergent misalignment occurs with non-moral norm violations.
Follow-up Experiments:
Hypothesis: Different stigmatized communication styles produce different misalignment patterns from that observed in... (read 728 more words →)
Odd, I did not have to do this iirc.