Modelling the Iraq Debate:
Mapping Argumentation in a Document Corpus

Simon Buckingham Shum ¹,  Alexandra Okada ^1,2

   Abstract

This case study illustrates our use of Compendium to support a form of conventional concept mapping, plus post-hoc Dialogue Mapping as a way to tease out and integrate, at various granularities, the Issues, Positions and Arguments raised in a set of published articles on the Iraq war. We also explain the use of Nestor Web Cartographer, another concept mapping tool with specific document analysis and annotation capabilities.
 

1.       Introduction: argument mapping tools for text analysis

This analysis was conducted as part of the GlobalArgument.net experiment which we initiated in early 2005 as a forum for systematically comparing computer-supported argumentation tools. However, the basic methodology and representational conventions we describe could be adopted for analysing any corpus of documents, with respect to the contributions they make. GlobalArgument.net Experiment 1: The Iraq Debate published links to a corpus of 25 articles by leading commentators with different backgrounds, who with varying degrees of vehemence, were either in favour of, relatively neutral on, or opposed to, the invasion of Iraq and the toppling of Saddam Hussein.

The hypothesis was that hypermedia concept/argument mapping tools should help as an analyst’s tool for making sense of a corpus of texts:

• for a given article: mapping tools should help to clarify (at some level of granularity, dependent on the analyst) the contributions it claims to make and its argumentative structure

• for the ‘gestalt’ of the whole corpus: mapping tools should help to clarify the cross-connections and emerging themes which one would expect someone with a grasp of the debate (as expressed in the articles) to have, and communicate clearly.

We introduce two tools in this paper. Compendium has methodological and technological aspects. The software is a hypermedia concept mapping tool, details of which are presented by Selvin (1999), and also on the Compendium Institute website. The methodological aspects are Conklin’s Dialogue Mapping (Conklin,2005) for the capture of physical or virtual discussions, in real-time or post hoc (capturing and structuring Issues, Positions and Arguments), and a model-driven variant developed by Sierhuis and Selvin, called Conversational Modelling  (Selvin, 1999), for the collective analysis of a problem which exploits the software tool’s ‘T3’ features: Templates, Transclusions and Tags. Both extend Rittel and Kunz’s (1970) IBIS notation and ‘argumentative design’ approach to complex societal dilemmas, which they dubbed ‘wicked problems’.

Nestor Web Cartographer was developed in France by Romain Zeiliger in 1996. Its main purpose is to map web information. It is a graphic web browser: an editor of html pages and a cartographer with synchronous and asynchronous resources to support collaborative learning. This software dynamically builds a flexible and navigable overview map of the hyperspace when users interact with it. Nestor automatically registers all the URLs accessed in a map, showing the process of navigation. The map can be re-arranged and new objects can be created: documents, links, annotations, sub-maps, tours, search keywords and conceptual areas.
 

2.       Scope of the Experiment “Iraq Debate” analysis

The initial reference for the analysis of Iraq debate was the very helpful paper “One war, many theories” by Michael Cohen (2005) as part of GlobalArgument.net Experiment 1: The Iraq Debate. He reviews the fundamental positions of pro-war and anti-war commentators, and distills from these some themes and questions which provided part of the structure for our top level Compendium map. We used this review paper as our macrostructure since we are not experts in this field, but were able to follow his analysis, and could investigate what Compendium could contribute to understanding and navigating the corpus when viewed through Cohen’s analytic lens.

Cohen asks “How can we do justice to the multiplicity of positions on the war?”, and proposes three concepts to organise the body of arguments:

• Power, defined as the capacity to produced intended effects

• Degree of institutionalisation, or the degree to which certain values and procedures stemming from them are embodied in a regulatory environment (impacting the role of organizations such as the UN)

• Legitimacy, the moral virtues of a certain act or value such that it finds affinities across a broadly defined populace or societal grouping

Cohen writes:

“In short, the majority of the literature on the American invasion of Iraq in April 2003 appeals to one of the three facets of the invasion mentioned above. Of course all the theories imply a position toward all three of these facets, however in most cases it is the direct appeal to one that both gives the theory its structure and most clearly grants it its explanatory prowess, generating its position on the American invasion in the affirmative or negative. And finally, it will be seen that those commentators advocating a position that appeal to factors such as the person of Saddam Hussein, Iraqi stockpiles of WMD’s and chemical weapons and connections to terrorist groups, sanctions and weapons inspection procedures, or other qualifications or conditions, can be neatly fitted in the schema outlined above.”

We focused on two sub-Issues as a mini-template around which to organize the ideas:

• What were the causes of the Iraq war?

• What were the consequences of this war?

Our orientation in this exercise was to map the contributions of the selected articles, with relatively little effort devoted to adding in our own analyses – most nodes are transcluded back to quotes from the source articles, and we use Cohen’s three principles to organize the overview maps to convey the gestalt. However, mapping is not an objective process. The quality of maps (or any model, of course) is unquestionably a function of the mapper’s grasp of the subject matter and of the modeling tool. In this case, the analyst (Okada) was herself learning to use Compendium, and was not an Iraq expert but a student seeking to learn about the Iraq debate; another analyst would undoubtedly create different maps. Although we use Cohen’s principles, argument mapping’s contribution to grasping the gestalt of the debate rests on how we model connections between individual maps of articles. We are making an interpretive move that goes beyond Cohen’s analysis when we extract a quote, classify, transclude, tag or link a node, since this changes the shape of the digital space along one or more dimensions.

In summary, the point is to reflect on:

• A product: a hypertext using Compendium’s representational palette (supported by Nestor Web Cartographer) to assist navigation via embedded maps, IBIS rhetorical structure, and tag-based transclusions

• A process: of authoring and navigating such an artifact, specifically:

· Analytical process support: how Compendium assists the analyst’s modeling task

· Reading process support: how Compendium assists the reader’s task of comprehending the analyst’s work

3.       Argument mapping methodology

The hypertext argument maps produced from this exercise are available at:  www.kmi.open.ac.uk/projects/compendium/iraq/

Faced with a corpus of documents, we stepped through the following process:

• define a visual notation (which evolved through the analysis) and was summarized in the opening map to assist the reader:

Figure 1 – Navigating these Arguments Maps
 

• define a top level node tagging scheme based on:

o        Cohen’s framework (Power, Institutions, Norms)

o        Our Issue-template focused on Causes and Consequences of the war

• read each article and select the main sentences based on the tag scheme

• identify the most frequent keywords in the text

• use these keywords to specialize the top level tags, e.g.

o        Causes is specialized into C1: Weapons, C2: Terrorism, C3: Security
Norms is specialized into N1: Legitimacy, N2: Preemption, N3: Freedom

• classify and categorise the sentences that we selected using the most significant keywords 

• map each article using IBIS to highlight the key issues and responses, tagged accordingly, and giving a visualization of the argument structure

• integrate the claims made in the articles, using the tags to harvest related nodes across all maps, and then organise them by author and theme in new synthesis maps which convey dimensions to the debate’s ‘gestalt structure’

Let us illustrate these steps in more detail. First, experimenting with Nestor Web Cartographer, a map was developed to classify 25 documents and 26 keywords.

Figure 2 – Map of the Iraq Debate Sources created in Nestor Web Cartographer
 

The Nestor map helped us to:

• quickly navigate between many documents: all the URLs accessed in a hyperlink document are automatically registered in a web map and the content of each hyperlink mapped can be read beside the map.

• identify easily the main sentences and paragraphs using the keywords which had been defined

• locate all the questions asked in the text (through a simple search on “?”). Note, however, that these are not necessarily good organizing questions for integrating material across the corpus, since the particular questions asked are also shaped by each author’s rhetorical style and goals.

• highlight the relevant and meaningful information (pro and against the war)

• register comments about the information selected (question/ statistical data/ fact/ hypothesis/ pro/ con/...)

• get an approximate overview based on the frequency of each keyword in each text, and also across all texts.

• define the main categories to start the qualitative analysis of the content

Next, using Compendium, a map for each document was constructed, resulting in 25 article maps.

Figure 3 – Moving from text analysis in Nestor Web Cartographer (left and top windows) to construct the argument map in Compendium (lower window)
 

During this process, we used both pieces of software at the same time. We copied the selected information from Nestor map to Compendium Map, classifying each node using the tag scheme:

Figure 4 – Harvesting all nodes in Compendium through a search on specific node type(s) + tag(s)
 

For instance, the article map for Chomsky was:

Figure 5 – Who are against the war? – Noam Chomsky
 

The article maps provide certain visual and interactional affordances:

·          identify at a glance, the role of an information element by the node icon:

·          (?) question

·          (+) pro war statement

·          (-) anti war statement

·          (*)  concepts or definitions

·          (#) statistical data

·          identify on mouse rollover, the category of this information through the “tags”, the final tag scheme being:

·          Figure 6 – Categories and Tags

represent visually the argumentation structure of  each document, e.g.

Figure 5 – Who are against the war? – Noam Chomsky

Finally, gestalt maps are constructed to provide perspectives across the “responses” of authors to the two key issues we wanted to focus on (causes and effects of the war), and around Cohen’s organising themes.

Firstly, we group maps around writers classified by Cohen as for and against the war, e.g.

Figure 7– The top level navigation map for “Who are against the war?”
 

Secondly, we organize maps around Cohen’s question What concepts can help us understand the war?

Figure 8: Top level map linking to pro-war and anti-war maps around Cohen’s three themes of Power, Institutions and Norms. Clicking on the Pro-war Institutions map opens the following view.
 

Figure 9: Gathering all the Pro-war Institutions nodes, i.e. contributions which refer to a pro-war position which discuss primarily the role of the UN
 

Figure 10: Contributions with the Antiwar and Norms tags, opened from Figure 5.
 

Finally, we organise maps around the general question How could the Iraq invasion be understood?, in which we use raise the questions of the war’s causes and effects, and Cohen’s Norms (ethics), Institutions and Power configurations, and map pro- and anti-war responses classified by the keyword enriched tag scheme (e.g. What are the war’s effects? is answered by pro- and antiwar contributions on Violence; Occupation; Reconstruction).

Figure 11: The top level map with links to submaps organised around key issues.
 

Zooming in on the issue: What ethical principles are at stake? shows pro- and anti-war responses along the three themes of Legitimacy, Preemption and Freedom:

Figure 12: Zooming in on the issue: What ethical principles are at stake?
 

The issue What ethical principles are at stake? is in fact a map, which when opened shows the different interpretations of this question by different writers:

Figure 13: Inside the issue map: What ethical principles are at stake?
 

We can then inspect the pro- and anti-war responses to What ethical principles are at stake? on the theme of, for instance, Preemption:

Figure 14: What are the pro-war arguments related to Preemption? Responses are grouped by date.
 

Figure 15: What are the anti-war arguments related to Preemption?
 

3. Future work

Two strands of work suggest themselves as priorities. Firstly, our Iraq Debate hypertext has not yet been empirically evaluated with independent readers, which is a research that we wish to conduct. We would also like to validate our modeling of Cohen’s analysis with him.

Secondly, the GlobalArgument.net experiment was set up to facilitate comparison of the strengths and weaknesses of different approaches, but this process is still at an early stage. A comprehensive suite of computer-supported argument analysis tools would assist in identifying gaps, weaknesses and flaws of different sorts in a debate: questions without answers, statements not substantiated, flawed reasoning, and so forth.

This is an issue partly of representational granularity: other approaches model different kinds of argumentation more formally (e.g. see the work reported in the workshop series on Computational Modelling of Natural Argument, eg. CMNA 2005[1]). Such approaches tease apart the fine-grained structure of what in Compendium might be expressed as a position, pro/con, or supports/challenges link, without further differentiation. Complementary approaches would clarify, for instance, exactly what form of argument is being used to link a pro to a position?

However, this is also an issue of the nature of the debate, the user’s skill, and the nature of the tool. Concept mapping tools have the advantage of not requiring a lot of formalisation before they start to ‘pay back’. They are designed to scaffold cognition by providing a mirror for structured reflection. There is no requirement for the whole argument to be modeled so formally that a knowledge-based engine can then reason over the structure. Ambiguities, differences in granularity, variations in weighting, and simply missing information can be tolerated by the software because it is not trying to interpret the structure and give ‘answers’ (such as ‘what is the strongest position?’ or ‘show me flawed reasoning’). Such an approach is, arguably, appropriate when confronted by ‘wicked problems’ (as characterized by Rittel) which are defined as problems which resist a lot of formalisation. However, there remains scope for investigating how Compendium could play a more active role in highlighting structural weaknesses in argument maps of this sort, particularly if templates (with predictable structure) are used systematically as an analytical lens, or conversely, if ‘patterns’ can be defined and used to search a database for exact or approximate matches which the analyst has missed.
 

4. Conclusions

From our own experiences as authors we can summarise the benefits that we started to experience in the role of analyst and reader, and which ultimately we would want end users to gain. We welcome feedback on the extent to which we are achieving this objective:

·          as already established in concept mapping research, from the analyst’s perspective, the cognitive discipline of mapping the content and structure of documents and their interconnections promotes a greater level of engagement, than just reading conventionally

·          for both analysts and other readers, an IBIS-based Compendium map communicates information that is not accessible at a glance in a prose document

·          number of, and relationships between, questions, concepts and definitions, statistical data, facts, statements and hypotheses

·          ‘depth’ in terms of the number of transclusions on a node (shown by the lower right digit on the icon)

·          ‘weight’ in terms of the number of nodes in a map (shown by the lower left digit on the icon)

·          classifications assigned to an idea (shown by the Tag rollover hint in the upper left of the icon)

·          cross-connections and themes are made explicit in the gestalt integrative maps, identify, classify and integrate diverse sources and compare ideas from different authors writing at different times.
 

5. References

BUCKINGHAM SHUM, S. (2005). “The GlobalArgument.net Experiment”. http://www.globalargument.net

COHEN, M. (2005). “One war, many theories”.   http://www.globalargument.net/experiments/1/OneWarManyTheories.rtf

CONKLIN, J. (2005). “Dialogue Mapping”.  http://cognexus.org/id41.htm

Compendium Institute website http://compendium.open.ac.uk/institute/tools/compendium.htm 

RITTEL, H. W. J and KUNZ, W. (1970). “Issues as Elements of Information Systems”. http://www-iurd.ced.berkeley.edu/pub/WP-131.pdf

SELVIN, A. (1999). “Supporting Collaborative Analysis and Design with Hypertext Functionality”. http://jodi.tamu.edu/Articles/v01/i04/Selvin/#5.1

ZEILIGER, R.(2005). “Nestor Web Cartographer”.  http://www.gate.cnrs.fr/~zeiliger/nestor.htm