LESSWRONG
LW

muhia_bee — LessWrong

Answer Set Programming for Automated Verification of Intent Consistency

Brian Muhia, August 2023

The causal influence diagrams introduced here (see the appendix, also here), and the accompanied reasoning that favours certain diagrams over others based on links to the "I" node, are simple enough that we can devise automated rules that check if a diagram is correct or wrong. We call this property "intent consistency". Here we introduce three simple rules written in the Answer-Set Programming (ASP) formalism that

find paths between any two nodes X and Y, then
check if a path exists from the input node "I" to any decision node, and
fails if there is no direct link from the node "I" to any

... (read more)

muhia_bee3yQuick Take

When chaining parallel and sequential calls to large language models (like LangChain), you implicitly create a causal graph that can be analyzed visually if you have the right tracing tools (https://github.com/oughtinc/ice). This notebook describes different agents using an explicit formalism based on causal influence diagrams, which we can treat as a notation for describing the data flow, components and steps involved when a user makes a request. We use the example diagrams to explain and fix risk scenarios, showing how easy it is to debug agent architectures if you can visually reason about the data flow, and ask questions about intent alignment for AGI in the context of such agents.

Examples and Theory... (read more)

muhia_bee's Shortform

muhia_bee

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Visualizing Learned Representations of Rice Disease

muhia_bee

By Brian Muhia

Status: Mid-project progress report after developing some thinking about potential future research directions. This is also part of a grant application [private communication].

https://github.com/poppingtonic/semisupervised-rice-africa

Motivation

The historical amount of pesticide use in agriculture has been a constant producer of illness as a secondary effect, that can be reduced by early plant disease detection. This document by Our World In Data shows which countries are most affected by massive amounts of pesticide use. Artificial intelligence should be helpful in more targeted, early interventions that help people to avoid using large amounts of pesticides. This project is to ensure the intelligence is interpretable, trustworthy, reproducible and generative. First we train, then we explain a family... (read 907 more words →)