I have been researching for a while, and it seems to me that there isn't that much progress on "automating" MI using Program Synthesis. The only source I could find is a paper from Max Tegmark's lab. However, this paper has been about for quiet a while, and not that...
I am really not sure of why neuro-symbolic systems are considered as alternatives to the current black-box ones? A concrete example I have found (and currently studying) is HOUDINI (https://arxiv.org/pdf/1804.00218). Essentially, it implements neural networks using higher order combinators (map, fold etc.) that were found via enumeration/genetic programming searches. When...
Recently, I have been trying to reason why I belive what I belive (regarding AGI). However, it appears to me that there is not enough discussion around the arguments against AGI (more specifically AGI skeptisim). This might be of benefit, especially given that Would this be because the arguments are...
I have been researching for a while, and it seems to me that there isn't that much progress on "automating" MI using Program Synthesis. The only source I could find is a paper from Max Tegmark's lab. However, this paper has been about for quiet a while, and not that much progress has been happening since then (or maybe I am unaware of it).
The inspiration of using Program Synthesis comes from Stephen Casper suggesting that program synthesis could lead to the process of automating MI. In the post itself, he mentions some papers that transform RL policies into programs, while the rest are some obscure methods. However, none of them are actually... (read 187 more words →)
These are some very valid points, and it does indeed make sense to ask "who would actually do it/advocate it/steer the industry etc.". I was just wondering what are the chances of such approach to take-off, but maybe the current climate does not really allow for such major changes to the systems' architecture.
Maybe my thinking is flawed, but the hope with this post was to confirm whether it would harmful or not to work on neuro-symbolic systems. Another point was to use such a system on benchmarks like ARC-AGI to prove that an alternative to dominating LLMs is possible, while also being to some degree interpretable. The linked post by @tailcalled is a good point, but I also noticed some criticism in the comments regarding concrete examples of how interpretable/less interpretable such probabilistic/symbolic system really are. Perhaps, some research on this question might not be harmful at all, but I think that is my opinion.
I am really not sure of why neuro-symbolic systems are considered as alternatives to the current black-box ones?
A concrete example I have found (and currently studying) is HOUDINI (https://arxiv.org/pdf/1804.00218). Essentially, it implements neural networks using higher order combinators (map, fold etc.) that were found via enumeration/genetic programming searches. When the programs are found, the higher order combinators are "transformed" into trainable networks and added to an ever growing library of "neural functions". The safety provided by such systems comes in the form of understanding the combined functions that form a the solution to a problem. Perhaps, mechanistic interpretability could be further used to dissect the inner workings of the trained networks.
Please, describe... (read more)
My focus would have wanted to be purely on AGI. I guess, the addition of AGI x-risks was sort of trying to hint at the fact that they would be coming together with it (that is why I mentioned Lecunn).
I feel like there is this strong belief that AGI is just around the corner (and it might very well be), but I wanted to know what is the opposition against such statement. I know that there is a lot of solid proof that we are going towards more intelligent systems, but understanding the gaps in this "given prediction" might provide useful information (either for updating timelines, changing research focus etc.).
Personally, I might... (read more)
Recently, I have been trying to reason why I belive what I belive (regarding AGI). However, it appears to me that there is not enough discussion around the arguments against AGI (more specifically AGI skeptisim). This might be of benefit, especially given that
Would this be because the arguments are either too weak or the Ai Safety is biased (understandably) towards imminent AGI?
This might also come out as a reaction from the recent advancements (such as o3) and the alarmant short timelines (less than 3 years). I want to understand the other sides points as well.
Based on what I found on the internet, the main arguments are roughly the following (not exact, given... (read more)
I like this post, especially as I think that o3 went under the mainstream radar. Just took notice of this announcement today, and I have not seen many reactions yet (but perhaps people are waiting to get their hands on the system first?) Is there a lack of reactions (also given that this post does not have a lot of engagement), or is my Twitter just not very updated?
Mike Knoop also mentioned in his Twitter post that this shows proof of how good deep learning program synthesis is. Does this refer to the way o3 was prompted to solve the ARC questions? Otherwise, what suggests this paradigm?
That's quite an interesting analysis. I feel like a lot of AI safety field, either intentionally isolates itself from academia, or simply does not want to engage with the academic research. Perhaps the feeling is both ways (academics might distrust AI Safety Research, especially the independent ones).
Regardless, I feel like this post motivated me to engage with a deeper / similar analysis between interpretability and the fields of genetic programming, program synthesis and neuro symbolic Ai. There might be some connections there, and on top of that formal analysis seems to be another layer of helpfulness.