pradahuman's Shortform
Jun 3, 20251
pradahuman
Message
Physics/Mathematics undergraduate at UC Berkeley.
I'm deeply interested in ML/AI (and recently alignment and mechanistic interpretability, though I'm fairly new to the latter two).
1
3
Hi!
I realize now the full conversation from Claude 4 was not shared. I essentially showed Claude 4 it's system card in chunks to test how it's own meta-model would react/update based on new information about itself. Meanwhile, I would ask o3 for signs or evidentiary clues that would change it's prior belief on whether there is an internally consistent conscious-like metacognition in LLMs (it initially vehemently denied this was possible, but after seeing Claude's response it became open to the possibility of phenomenological experiences and a consistent self-model that can update based on new information)
https://claude.ai/share/ee01477f-4063-4564-a719-0d93018fa24d
Here's the full conversation with Claude 4. I chose minimal prompting here, and I specifically used "you" without... (read more)
I have some thoughts on apparent emergent “self-awareness” in LLM systems and propose a mechanistic interpretability angle. I would love feedback and thoughts.
TLDR:
My hypothesis: Any LLM that reliably speaks in the first-person after SFT/RLHF should contain activation loci that are causally necessary for that role-aware behavior; isolating and ablating (or transplanting) these loci should switch the system between an “I-free tool” and a “self-referential assistant.”
I believe there is circumstantial evidence for the existence of such loci, though whether they are strictly low-rank or highly sparse or cleanly identifiable at all is an empirical question. I do believe that existing work on refusal clusters and role-steering vectors makes compact solutions plausible as well.
Why does it... (read 946 more words →)
Are frontier reasoners already "sentient" or at least “alien-sentient” within their context windows?
I too would immediately dismiss this upon reading it, but bear with me. I'm not arguing with certainty. I just view this question to be significantly more nuanced than previously entertained, and is at least grounds for further research to resolve conclusively.
Here are some empiric behavioral observations from Claude 4 Opus (the largest reasoner from Anthropic):
a) Internally consistent self-reference model, self-adjusting state loop (the basis of Chain-of-Thought, self-correcting during problem solving, reasoning over whether certain responses violate internal alignment, deliberation over tool-calling, in-context behavioral modifications based on user prompting)
b) Evidence of metacognition (persistent task/behavior preferences across chat interactions, consistent subjective... (read 524 more words →)