Follow-up to: Argument Maps Improve Critical Thinking, Software Tools for Community Truth-Seeking
We are here, among other things, in an attempt to collaboratively refine the art of human rationality.
Rationality is hard, because the wetware we run rationality on is scavenged parts originally intended for other purposes; and collaboration is hard, I believe because it involves huge numbers of tiny decisions about what information others need. Yet we get by, largely thanks to advances in technology.
One of the most important technologies for advancing both rationality and collaboration is the written word. It affords looking at large, complex issues with limited cognitive resources, by the wonderful trick of "external cached thoughts". Instead of trying to hold every piece of the argument at once, you can store parts of it in external form, refer back to them, and communicate them to other people.
For some reason, it seems very hard to improve on this six-thousand-year-old technology. Witness LessWrong itself, which in spite of using some of the latest and greatest communication technologies, still has people arguing by exchanging sentences back and forth.
Previous posts have suggested that recent software tools might hold promise for improving on "traditional" forms of argument. This kind of suggestion is often more valuable when applied to a real and relevant case study. I found the promise compelling enough to give a few tools a try, in the context of the recent (and recurrent) cryonics debate. I report back here with my findings.
The first tool I tried was Argunet, an Open Source offering from the Institute of Philosophy in Berlin. I was seduced by the promise of reconstructing the logical structure of an argument, and by the possiblity of collaborating online with others on an argument.
Like other products in that category, the basic principle of operation of Argunet is that of a visual canvas, on which you can create and arrange boxes which represents statements, portions of an argument. Relationships between parts of an arguments are then materialized using links or arrows.
Argunet supports two types of basic relationship between statements, Supports and Attacks. It also supports several types of "inference patterns".
Unfortunately, when I tried using the Editor I soon found it difficult to the point of being unusable. The default expectation of being able to move boxes around by clicking and dragging is violated. Further, I was unable to find any way to move my boxes after initially creating them.
I ended up frustrated and gave up on Argunet.
II. bCisive Online
I had somewhat better luck with the next tool I tried, bCisive online. This is a public beta of a commercial offering by Austhink, the company already referenced in the previous posts on argument mapping. (It is a spin-off of their range of products marketed for decision support rather than argument support, but is also their only online, collaborative tool so far.)
The canvas metaphor proved to be implemented more effectively, and I was able in a relatively short time to sketch out a map of my thinking about cryonics (which I invite you to browse and comment on).
bCisive supports different types of statements, distinguished by the icons on their boxes: questions; arguments pro or con; evidence; options; "fixes", and so on. At present it doesn't appear to *do* anything valuable with these distinctions, but they proved to be an effective scheme for organizing my thoughts.
III. Preliminary conclusions
I was loath to invest much more time in updating my cryonics decision map, for two reasons. One is that what I would like to get from such a tool is to incorporate others' objections and counter-objections; in fact, it seems to me that the more valuable approach would be a fully collaborative effort. So, while it was worthwhile to structure my own thinking using the tool, and (killing two birds with one stone) that served as a test drive for the tool, it seems pointless to continue without outside input.
The other, more important reason is that bCisive seems to provide little more than a fancy mindmapping tool at the moment, and the glimpse I had of tool support for structuring a debate has already raised my expectations beyond that.
I have my doubts that the "visual" aspect is as important as the creators of such software tools would like everyone to think. It seems to me that what helped focus my thinking when using bCisive was the scheme of statement types: conclusion, arguments pro and con, evidence and "fixes". This might work just as well if the tool used a textual, tabular or other representation.
The argument about cryonics is important to me, and to others who are considering cryonics. It is a life decision of some consequence, not to be taken lightly and without due deliberation. For this reason, I found myself wishing that the tool could process quantitative, not just qualitative, aspects of my reasoning.
IV. A wish list for debate support
Based on my experiences, what I would look for is a tool that distinguishes between, and support the use of:
- a conclusion or a decision, which is to be "tested" by the use of the tool
- various hypotheses, which are offered in support or in opposition to the conclusion, with degrees of plausibility
- logical structure, such as "X follows from Y"
- challenges to logical structure, such as "X may not necessarily follow from Y, if you grant Z"
- elements of evidence, which make hypotheses more or less probable
- recursive relations between these elements
The tool should be able to "crunch numbers", so that it gives an overall indication of how much the total weight of evidence and argumentation contributes to the conclusion.
It should have a "public" part, representing what a group of people can agree on regarding the structure of the debate; and a "private" part, wherein you can adduce evidence you have collected yourself, or assign private degrees of belief in various statements.
In this way, the tool would allow "settling" debates even while allowing disagreement to persist, temporarily or durably: you could agree with the logical structure but allow that your personal convictions rationally lead you to different conclusions. Highlighting the points of agreement and contention in this way would be a valuable way to focus further debate, limiting the risk of "logical rudeness".
Comment in a nutshell: "Practice with very simple tools optimized for swift use in a culture that values logic and evidence are likely to be better than specialized tools with hard coded abstractions."
There's a specialized form of note taking called "flowing" within the policy/CEDA/NDT debate community. Here is a wikipedia article on the subject if anyone wants to hunt in the concept space for keywords and links and such :-)
In this speech community (especially at the higher levels) people tend to speak very swiftly because there is a background theory that "dropping arguments is conceding arguments" which creates an incentive to make many parallel arguments with the same conclusion that reach towards different sets of evidence. It is possible to win simply by making many adequate arguments that your opponent is incapable of handling with enough speed or concision.
(It can be dangerous to spin out many arguments without paying attention to how they interact, because sometimes a response will be something like "I concede argument 2 and 5 which constitute a turn on the general position that works like X. Arguments 1, 3, and 4 were about A, B, and C wh... (read more)
First, my own observation agrees with GreenRoot. My view is less systematic but much longer, I've been watching this area since the 70s. (Perhaps longer, I was fascinated in my teens by Leibnitz's injunction "Let us calculate".)
Empirically I think several decades of experiment have established that no obvious or simple approach will work. Unless someone has a major new idea we should not pursue straightforward graphical representations.
On the other hand we do have a domain where machine usable representation of thought has been successful, and where in fact that representation has evolved fairly rapidly. That domain is "programming" in a broad sense.
Graphical representations of programs have been tried too, and all such attempts have been failures. (I was a project manager for such an attempt in the 80s.) The basic problem is that a program is naturally a high-dimensional object, and when mapped down into a two dimensional picture it is about as comprehensible as a bowl of spagetti.
The really interesting aspect of programming for representing arguments isn't the mainstream "get it done" perspective, but the background work that has been d... (read more)
Thanks for looking into this for us!
One of my ever-pending posts to write is on what sort of simple interface might prevent online arguments from retracing the same points over and over. I suspect it will not be graphical with boxes, because that makes poor use of screen real estate. I suspect it will not have lots of fancy argument types and patterns, because no one really uses that stuff. I think it does need to have a karma system, because otherwise there's no way to find the good stuff.
Reading that Argunet was both open source and flawed, I decided to download the code (signals geekiness) in order to fix the problem you have encountered (signals generosity). I didn't have to change anything: it took me a while (signals perseverance) but I finally figured out how to move a box in Argunet (signals superior problem-solving abilities).
It turns out that the result of dragging a box depends on whether the box was selected prior to dragging it. If the box was selected, it is moved, otherwise, an arrow is created. So, before you move a box, select it by clicking on it.
I created a wikipage listing all of the debate tools that were mentioned so far in this comments thread:
Please feel free to edit this wikipage.
I would recommend that we try to create our own debate-mapping tool. It might end up being surprisingly easy.
I've already used PHP, GraphViz, and MediaWiki to implement a vaguely similar project, the Transhumanist Wiki Scenarios Map.
Unfortunately, that project ended up being less useful than I had hoped, and has been abandoned for now.
Today, I made a rough sketch of what a debate-mapping tool based on these tools might look like.
A VERY rough sketch.
Pretty much every detail is probably going to need to be changed in order for it to be useable.
Anyway, here'... (read more)
I strongly suspect this is highly personal. People can relate to thoughts in different ways. The developers might primarily 'see' argument structure in their heads, where you might think less visually. This tools helps them clarify their ideas and show them to other visually oriented people.
So you're right that it might not catch on, but it may be that the visual aspect is hugely useful to the people it's adapted for.
I looked into this sort of software a couple years ago as part of a information visualization seminar at Stanford. People have been creating research prototypes and betas for a long time, aiming to support everything from education to the formalization of legal dispute resolution ("computational law"). I wasn't impressed with what I saw in my review then, and looking around again today, it doesn't look like things have changed much. I have never seen anything quantitative.
You can find descriptions of research systems and small-scale studies in... (read more)
One problem with these sorts of tools is that they encourage people to split their arguments into lots of little pieces, which means arguments for C often end up being A->B, B'->C where B and B' look alike but on closer inspection turn out to be different.
Here's another interesting and potentially useful tool I found recently. I'm not sure if this qualifies as a debate tool, but it seems like it's in the general category of what we're looking for:
Summary: Canonizer.com is a wiki system with added camp and survey capabilities. The system provides a rigorous way to measure scientific / moral expert consensus. It is designed for collaborative development of concise descriptions of various competing scientific or moral theories, and the best arguments for such. People can join the camps representing such, givi... (read more)
Today I did a google search for "debate map", and this was the very first result:
This... is exactly what we're looking for, isn't it?
Though it still doesn't actually do anything with numbers.
I still haven't gotten around to continuing my own project for a debate tool that actually does calculations involving probabilities, though it has finally risen to the top of my to-do list. I was planning to get back to work on it last weekend, but ended up getting distracted by other things again.
A point about using diagrams to make arguments: If you are attempting to convince a person that something is true, rather than just launching into your evidence and favorite arguments it is often most efficient to begin by asking a series of questions to determine precisely how the person disagrees with you. The questioning allows you to hone in on the most important sticking points that prevent the other party from coming to your conclusion. These points can then be attacked individually, preventing you from wasting time making arguments that the other pa... (read more)
Debate mapping is part of TakeOnIt, a publicly editable database of expert opinions introduced in a previous post ( http://lesswrong.com/lw/1kl/takeonit_database_of_expert_opinions/ ). It's deliberately very simple. Here's how it works:
1) Every debate is expressed as a yes-no question.
2) Every yes-no question has experts on both sides of the debate.
3) Every debate can link to a sub-debate (recursively).
A "simple debate" is one where '1' and '2' are sufficient. You can determine who is right in a simple debate by judging which experts have the b... (read more)
Hello everyone, I hope you don't mind me joining in on this 8 year old post. I've been working on ideas like this since 2012 and just found this. My current experiment is Reason Score where I am working on a way to measure the reasonableness of a claim based on the pro and con claims added to it. This will hopefully reduce cognitive biases by forcing people to add reasons to affect the score instead of votes. In the least it will encourage people to think through their claims.
It's not documented well so it might be best if someone has some time to debate me on a topic and see if it provides benefit. Any takers?
Some relevant links: structured debate is about how this should work, and dispute resolution technology lists some attempts to implement a solution.
Those pages are both GNU-FDL licensed (soon to be Creative Commons as soon as I can get around to it), so please feel free to snag anything useful and repost it on the LW wiki.
Thanks for writing this, this is an interesting area, and improving decision making is a worthwhile goal.
However, what I am a bit skeptical about is the extent to which people want to improve the decision making process. In LW-circles I can see that desire, but in the world at large it seems more important to win the argument than necessarily be right; there is some more work to do before they would even want to use the tools.
If we focus on LW-readers which are (hopefully) more interested in truth seeking, it would be interesting to see if there has been a... (read more)
There is an option in the bCisive application, under the "spaces" tab to turn on guest access. It should supply you with an URL you can include in your post here. Without turning that option on, we would have to register, and you would have to invite each of us to view the argument map.
So: "spaces" -> "cryonics" -> "manage" -> turn on guest access
Hey everyone! It appears I'm six years late to the party, but better late than never.
I've been building a website for the last few months which is very close to the ideas presented in this article. I've summarized some features of it, and added an entry to the wiki page:
Debate Map: Web platform for collaborative mapping of beliefs, arguments, and evidence.
Here's a new one which just crossed my radar: Wrangl. The link goes to a representative "argument".
There is a web-based tool being worked on at MIT's collective intelligence lab. Couldn't find the direct link to the project, but here's a video overview: Deliberatorium
There is also Scott Aaronson's worldview manager. This is designed to point out hidden contradictions (or at least tensions) between one's beliefs, by using programmed in implications to exhibit (possibly long) inferential chains that demonstrate a contradiction.
As such, it does have an OK DSL for such inferences.
EDIT: unfortunately, the site seems to be down, with broken SQL queries...
I'd love to see more tools like Bcisive! Thanks!
I have using Rationale from Austhink back when I was taking Philosophy classes. It was actually pretty good compared to what others offer but unfortunately limited to Windows. Ever since that time I have been thinking on and of about what software could be developed to both share ideas and refine our collective thinking in a better way. While there are tons of projects attempting such things most are either horribly designed or targeted at a non existing market.
I've read through the discussion above and I wonder why a semantic web approach wasn't mentioned... (read more)
I thought a lot about creating such a system and how it would look a number of years ago, but never did make any good progress on it. The point where I got stuck was to take a particular blog post with lots of debate in the comments and try to dissect it in different ways and see what ended up being the most useful. I found I didn't have the focus to do so.
Anyway, there's Truth Mapping, which I think sucks for quite a number of reasons.
I would love to see this turn into a software development project. I would be happy to participate, though I won't have much time any time soon.
Sentences are powerful because we are hard-wired to use language, so if we can leverage this then all the better. It sounds like TakeOnIt is going down this path (though I haven't looked at that project).
Re crunching numbers: Ensuring the argument is always crunchable will mean the software will have to place strong restrictions on the structure and relations at all points during the argument's con