AI Risk & Opportunity: Strategic Analysis Via Probability Tree

[-]othercriteria14y80

I'm not well-versed in these probability trees but something weird seems to be happening in the third one. The conditional probabilities under "private push" and those under no "private push" should each separately sum to 1. Changing them to joint probabilities would resolve the problem. Alternatively, you could put a probability on "private push" and then put FAI, uFAI, and Other on each branch.

[-]lukeprog14y40

Oops, fixed.

[-]Bugmaster14y50

This is kind of a weird article... It explains how to use decision trees, but then it just stops, without telling me what to expect, why I should care, or, assuming I did care, how to assign probabilities to the nodes. So, the only feeling I'm left with at the end is, "all righty then, time for tea".

In addition, instead of saying "X | private push" and "X | no private push", it might be clearer to add the nodes "private push" and "no private push" explicitly, and then connect them to "FAI", "uFAI", etc. An even better transformation would be to convert the tree into a graph; this way, you won't need to duplicate the terminal nodes all the time.

[-]othercriteria14y10

Moving to a graph makes elicitation of the parameters a lot more difficult (to the extent that you have to start specifying clique potentials instead of conditional probabilities). Global tasks like marginalization or conditioning also become a lot harder.

[-]Bugmaster14y10

Moving to a graph makes elicitation of the parameters a lot more difficult (to the extent that you have to start specifying clique potentials instead of conditional probabilities).

I think you can still get away with using conditional probabilities if you make the graph directed and acyclical, as I should've specified (my bad). The graph is still more complex than the tree, as you said, but if we're using software for the tree, we might as well use one for the graph...

[-]XiXiDu14y50

But this is much too large a project for me to undertake now.

Too bad. I was excited about this post and thought it was a good sign that you took that path and that it would be highly promising to pursue it further.

[-]lukeprog14y90

Another worry is that putting so many made-up probabilities into a probability tree like this is not actually that helpful. I'm not sure if that's true, but I'm worried about it.

[-]Dmytry14y40

I'm rather pessimistic about that. I think that basically the tree branches into a very huge number of possibilities; one puts the possibilities into categories, but has no way of finding total probabilities within the categories.

Furthermore, the categories themselves do not correspond to technological effort; there is the FAIs that resulted from regular AI effort via some rather simple insight by the scientist that came up with AI, the insights that may be only visible up close when one is going over an AI and figuring out why the previous version killed half of itself repeatedly instead of self improving, or other cases the probability of which we can't guess at without knowing how the AI is being implemented and what sort of great filters does the AI have to pass before it fooms. And there are uFAIs that result from the FAI effort; those are the uFAIs of entirely different kind, with their own entirely different probabilities that can't be guessed at.

The intuitions are often very wrong, for example the intuitions about what 'random' designs do; firstly, our designs are not random, and secondarily, random code predominantly crashes, and the non crashing space is utterly dominated by one or two simplest noncrash behaviours, and same may be true of the goal systems which pass the filters of not crashing, and recursively self improving over enormous range. The filters are specific to the AI architecture.

The existence of filters, in my opinion, entirely thwarts any generic intuitions and generic arguments. The unknown , highly complex filters are an enormous sea between the logic and the probability estimates, in the land of inferences. The illogic, sadly, does not need to pass through the sea, and rapidly suggests the numbers that are not, in any way, linked to the relevant issue-set.

[-]keefe14y20

I think that you are you are on a solid research path here. I think you have reached the bounds of business oriented software and it's time to look into something like apache mahout or RDF. Decision tree implementations are available all over, just find a data structure and share them and run inference engines like owlim or pellet and see what you can see.

RDF is a good interim solution because you can start encoding things as structured data. I have some JSON->RDF stuff for inference if you get to that point.

Here is one way to represent these graphs as RDF.

Each edge becomes an edge to a blank node, that blank node has the label, arrival probability and could link to evidence supporting. Representing weighted graphs in RDF is fairly well studied.

The question is, what is your net goal of this from a computational artifact point of view?

[-]wedrifid14y10

I like the numbers plugged into that tree. Only slightly optimistic. :)

[-]lessdazed13y00

Teaching tree thinking through touch.

These experiments were done with video game trees showing evolutionary divergence, and this method of teaching outperformed traditional paper exercises. Perhaps a simple computer program would make teaching probability trees easier, or the principles behind the experiments could be applied in another way to teach how to use these trees.

LESSWRONG
LW

LESSWRONG
LW

20

AI Risk & Opportunity: Strategic Analysis Via Probability Tree

20

20

A very simple tree

How the tree could be expanded

References