Epistemological Status: Correct, but lazily written. Basically, a while ago John made a post attempting to fix the Good Regulator Theorem. I've refrained from writing this simply because I know it may come off as "why does anyone care isn't this trivial"? However, more recently John has set up a...
Epistemological Status: I attempt to distill the minimal map argument and provide a natural way to chain a sequence of queries together. I argue that this definition, together with reasonable symmetry constraints on the minimal map, provides an intuitive explanation for the appearance of neural networks. Minimal Maps Say we...
Epistemological Status: I ran a basic simulation and this checks out. Specific details need refinement. Probably just a curiosity, but I think there's a chance there's a deeper connection here. I opted to post in the earlier stages incase I'm missing something big. In biology, the quasispecies equation is used...
Hi, I think this is incorrect. I had to wait 7 days to write this comment and then almost forgot to. I wrote a comment critiquing a very long post (which was later removed) and was down-voted (by a single user I think) after justifying why I wrote the comment with AI-assistance. My understanding is that a single user with enough karma power can effectively "silence" any opinion they don't like by down-voting a few comments in an exchange.
I think the site has changed enough over the last several months that I am considering leaving. For me personally, choosing between having a conversation with a random commenter on this site vs. an AI model is just about at a wash. I even hesitate to write this comment given how over-confident your comment seemed i.e. I won't be able to interact with this site again for another week.
This is my endorsed review of the article.
This seems like a rhetorical question. Both of our top-level comments seem to reach similar conclusions, but it seems you regret the time spent engaging with the OP to write your comment. This took 10 min, most spent writing this comment. What is your point?
At over 15k tokens, reading the full article requires significant time and effort. While it aims to provide comprehensive detail on QACI, much of this likely exceeds what is needed to convey the core ideas. The article could be streamlined to more concisely explain the motivation, give mathematical intuition, summarize the approach, and offer brief examples. Unnecessary elaborations could be removed or included as appendices. This would improve clarity and highlight the essence of QACI for interested readers.
My acquired understanding is that the article summarizes a new AI alignment approach called QACI (Question-Answer Counterfactual Interval). QACI involves generating a factual "blob" tied to human values, along with a counterfactual "blob" that could... (read more)
[Deleted]
[Deleted]
[Deleted]
[Deleted]
So are you suggesting that ChatGPT gets aligned to the values of the human contractor(s) that provide data during finetuning, and then carries these values forward when interacting with users?
You are correct that this appears to stand in contrast one of the key benefits of CIRL games. Namely, that they allow the AI to continuously update towards the user's values. The argument I present is that ChatGPT can still learn something about the preferences of the user it is interacting with through the use of in-context value learning. During deployment, ChatGPT will then be able to learn preferences in-context allowing for continuous updating towards the user's values like in the CIRL game.
Epistemological Status: Correct, but lazily written.
Basically, a while ago John made a post attempting to fix the Good Regulator Theorem. I've refrained from writing this simply because I know it may come off as "why does anyone care isn't this trivial"? However, more recently John has set up a research program centered around creating more results of this type so perhaps it's worth introducing a new opinion.
I guess I'm also slowly becoming a cynical grad student. Since I'm at UIUC and Conant/Ashby were affiliated I simply went around and asked a few ECE friends about the internal model principle. While it's a skewed sample, the friend and teacher I know didn't... (read 963 more words →)
Epistemological Status: I attempt to distill the minimal map argument and provide a natural way to chain a sequence of queries together. I argue that this definition, together with reasonable symmetry constraints on the minimal map, provides an intuitive explanation for the appearance of neural networks.
Say we sample an observation from a state-observation pair and want to decide based only on whether has a property . We can define the function to indicate whether or not an observation has the property we have in mind. Concretely, I've chosen to suppress the reference to the hidden state since it's not an observable. This could alternatively be viewed as... (read 1250 more words →)
Epistemological Status: I ran a basic simulation and this checks out. Specific details need refinement. Probably just a curiosity, but I think there's a chance there's a deeper connection here. I opted to post in the earlier stages incase I'm missing something big.
In biology, the quasispecies equation is used to model population structures of viruses. States are usually interpreted as genotypes for an individual. Commonly, these states represent proto-viral DNA/RNA strands. Transitions are mutations between these genotypes. Usually, the state genotype confers the replication rate and transitions are equivalent to mutations. The literal equation states that we can study the population dynamics with the mutation matrix using, Let's switch to the... (read 612 more words →)
If we're taking the idea that arguments are paths in topological space seriously, I feel like conditioned language models are going to be really important. We already use outlines to effectively create regression data-sets to model arguments. It seems like modifying GPT-2 so that you can condition on start/end prompts would be incredibly helpful here. More speculative, I think that GPT-2 is near the best we'll ever get at next word prediction. Humans use outline like thinking much more often then is commonly supposed.
I'm interested in converting notes I have about a few topics into posts here. I was really trying to figure out why this would be a good use of my time. The notes are already rather readable by myself. I thought about this for a while and it seems as though I'm explicitly interested in getting feedback on some of my thought processes. I'm aware of Goodhart's law so I know better than to have an empty plan to simply maximize my karma. However, on the other end, I don't want to simply polish notes. If I were to constrain myself to only write about things I have notes on then it
Just wanted to give some validation. I left a comment on this post a while ago pointing out how one (or apparently a few) users can essentially down vote you however they like to silence opinions they don't agree with. Moderation is tricky and it is important to remember why. Most users on a website forum are lurkers meaning that trying to gather feedback on moderation policies has a biased sampling problem. The irony on likely not being able to leave another comment or engage in discussion is not lost on me.
At first, I thought getting soft-banned meant my "contributions" weren't valuable. For context, I study AI and integrate it into my... (read more)