All of Sven Nilsen's Comments + Replies

I think that most people imagine themselves as gatekeepers for controlling the world, when in reality the world is controlled by systems we don't fully understand. For example, I don't think people really know what "money" means in the real world. You can imagine yourself as a person who knows it, but can you verify that your understanding matches reality?

If we want to have a chance to control the future to some degree, then it means that humans need to take over the world. It is not sufficient to stop AGI from taking over the world, but we also need to ensure that the future of the world is aligned with human values. For now, it seems that stuff like "money" is running the show.

Obviously, problems are not exclusive! I find it easier to imagine a civilization that has survived for a long time and made significant technological progress: How would a such civilization approach ASI? I think they will analyze the problem to death and use automated theorem proving as much as possible and having a culture where only a tiny amount of ideas ever get implemented, even if most of those ideas never implemented would seem very good to us. In short: Higher standards for safety.

One challenge with the "people will use it for bad stuff"-situation... (read more)

It is also worth thinking if you put in context that people said "no, obviously, humans would not let it out of the box". Their confident arguments persuaded smart people into thinking that this was not a problem.

You also have the camp "no, the problem will not be people telling the AI do bad stuff, but about this hard theoretical problem we have to spend years doing research on in order to save humanity" versus "we worry that people will use it for bad things" which in hindsight is the first problem that occurred, while alignment research either comes too... (read more)

The error margin LeCun used is for independent probabilities, while in an LLM the paths that the output takes become strongly dependent on each other to stay consistent within a path. Once an LLM masters grammar and use of language, it stays consistent within the path. However, you get the same problem, but now repeated on a larger scale.

Think about it as tiling the plane using a building block which you use to create larger building blocks. The larger building block has similar properties but at a larger scale, so errors propagate, but slower.

If you use i... (read more)

The error margin is larger than it appears as estimated from tokens, due to combinatorial complexity. There are many paths that the output can take and still produce an acceptable answer, but an LLM needs to stay consistent depending on which path is chosen. The next level is to combine paths consistently, such that the dependence of one path on another is correctly learned. This means you get a latent space where the LLM learns the representation of the world using paths and paths between paths. With other words, it learns the topology of the world and is able to map this topology onto different spaces, which appear as conversations with humans.

I up-voted your post because I think this is a useful discussion to have, although I am not inclined to use the same argument and my position is more conditional. I learned this lesson from the time I played with GPT-3, which seemed to me as a safe pathway toward AGI, but I failed to anticipate how all the guardrails to scale back deployment were overrun by other concerns, such as profits. It is like taking a safe pathway and incrementally make it more dangerous over time. In the future, I expect something similar to happen to GPT-4, e.g. people develop ha... (read more)

If ChatGPT is asked questions like "should we put in safeguards in the development of self-improving AIs" then is it likely to answer "yes". Now, if ChatGPT was given political power, it becomes a policy that world leaders need to solve. Do we need constraints on GPU computing clusters? Maybe ChatGPT answers "STOP", because it thinks the question is too complex to answer directly. It is always more difficult to decide on what actions to do in order to implement general policies, than agreeing about the overall policy. However, if we can align our overall p... (read more)

Maybe you can talk to Eric Weiser, who kindly provided me a proof in Lean 3: https://github.com/advancedresearch/path_semantics/blob/master/papers-wip/semiconjugates-as-satisfied-models-of-total-normal-paths.pdf

The experts in dependent types I know, think Path Semantics might help provide a better foundation or understanding in the future, or perhaps languages with some new features. We don't know yet, because it takes a lot of work to get there. I don't have the impression that they are thinking about Path Semantics, since there is already a lot to do in ... (read more)

The paper about the counter-example to Leibniz's First Principle in Path Semantics has been released: https://github.com/advancedresearch/path_semantics/blob/master/papers-wip2/counter-example-to-leibniz-first-principle.pdf

OK, that thread is from last year, before HOOO EP was completed. HOOO EP is now completed.

You should not be confused by the term "Path Semantics", as this is the name of my project. The foundational level of Path Semantics is a set of logical languages consisting of IPL, HOOO EP, PSQ and PSI.

Of course HOOO EP is a made up term. I am the person who invented it. However, the Prop library contains stuff about IPL, which is not something I invented. You don't have to understand HOOO EP to trust that IPL is previous work, as it is well established field. Consid... (read more)

Path semantics is built upon previous works, e.g. Homotopy Type Theory: https://homotopytypetheory.org/

This kind of previous work is cited all of the place. I have no idea why you think there is no preexisting discussion going on.

Yes, all you needed was simply to ask: https://www.academia.edu/45073917/Asymmetries_of_Complementary_Nilsen_Theories?email_work_card=title

Here is a theory based on the work of 3 people in collaboration: https://github.com/advancedresearch/joker_calculus

I can give you more examples.

Let's say you don't trust the Pocket-Prover library. Yet, this library is a brute force theorem prover that no person so far has reported bugs. I understand you won't trust it without further investigation, but there are no other people who have reasonable doubt about it. Where ... (read more)

I assume you can assume for yourself that what I write is nonsense, but this does not prove that I am writing nonsense. Claiming that my work is generated by GPT is false, when it isn't.

That's not how logic works. If no evidence I provide can convince you otherwise, then your condition for changing your mind is predicated on that the agent proving you evidence is not me. This is not acceptable for me, because if I give you link to somebody who uses my work or a computer program that uses my theory, then you will not accept it as a reason to investigate further.

You didn't make a claim that some of my work is wrong, you made the claim that the ENTIRE thing is wrong, which is a gaslighting argument.

I'll give short introduction here to some of the background in constructive logic required to provide constructive criticism, as I saw one person here using a gaslighting argument:

Constructive logic is a weaker logic than classical logic, in which a statement A => B is modeled as a lambda expression in simple type theory. Propositional equality is A == B, consisting of two maps A => B and B => A. The reason A == B is technically an assumed position in logic, is that lambda expressions can capture variables from their environment. It means, A == B ... (read more)

Author here. This is based on research from implemented theorem proving. See the links in the post.

Thank you! Post updated to include the definition.