Enriching Representations of Work to Support Organisational Learning

Tamara Sumner, Knowledge Media Institute, The Open University, UK

John Domingue, Knowledge Media Institute, The Open University, UK

Zdenek Zdrahal, Knowledge Media Institute, The Open University, UK

Marek Hatala, Technical University of Kosice, Slovakia

Alan Millican, BAe Virtual University, UK

Jayne Murray, BAe Virtual University, UK

Knut Hinkelmann, DFKI, DE

Ansgar Bernardi, DFKI, DE

Stefan Wess, TecInno, DE

Ralph Traphöner, TecInno, DE

1. INTRODUCTION

"Businesses spend up to $100 billion each year to train workers. Yet estimates are that less than 10% of this training transfers to the job. So business wastes $90 billion each year...." [Review on US training effectiveness (Detterman 1993)]

Current theories of learning reveal why this is so: the process of acquiring knowledge cannot be separated from the process of applying it. Integrating working and learning is not a desirable luxury - it is a fundamental requirement for businesses to remain competitive. In this document, we describe our research approach for investigating this challenge and our expected outcomes. This project has been recently funded by ESPRIT in the IT for Learning and Training in Industry programme. This document is based on the proposal and we welcome comments on our approach from the workshop participants.

In this project, we are building on our existing technologies supporting web-based collaboration, learning-on-demand, and knowledge modelling, to develop tools and methodologies for integrating working and learning within knowledge-intensive organisations. Specifically, our tools and methodologies will support organisational learning within operational areas relevant to the industrial partners. The key business objectives that our approach will address are:

* Supporting individuals and groups to continually reflect on and improve their work practices, particularly in operational areas of planning and bid winning.

* Supporting distributed groups to share `best practices' and improve their coordination efforts.

* Promoting the establishment of `virtual centres of excellence' that serve to identify core competencies and nurture their further development by bringing people together (across time and geography) with relevant expertise.

These business objectives can be realised by viewing organisational learning as a process by which knowledge that is created during working is incrementally captured, structured, and maintained so that this knowledge can be accessed or delivered when needed to inform individual and group work tasks. Our learning approach integrates the full spectrum of learning needs - individual, group, and organisational - and is informed by the extensive experience of The Open University in distance education and open learning. We will enable organisations to realise these business objectives by providing tools and methodologies facilitating the cost-effective construction, and sustained use, of knowledge-enriched intranets.

The project's core technical objective is to integrate existing tools and technologies to develop The CEDAR Toolkit. This toolkit will enable developers to provide customers with Contextually-Enriched Document ARchive systems containing:

* a document-centred discourse space for structuring discussions around representations of work;

* client tools enabling users to articulate and refine domain concepts by incrementally enriching their representations of work with related discussions and underlying knowledge models;

* knowledge delivery and access mechanisms supporting individual and group learning;

* an intranet-based organisational memory server for knowledge capture, structuring and maintenance.

Technology alone cannot ensure that learning takes place. The project's methodological objective will be the articulation of new work practices and roles needed to realise sustainable organisational learning within specific industrial settings. This objective will be assessed and refined by deploying and evaluating our technologies in naturalistic settings throughout the project within selected industrial groups, namely TecInno, British Aerospace (BAe), DFKI (the German National AI Centre), JOLA and Saarbergwerke AG. Specifically, to ensure that the tools and methodologies delivered by the project are both useful and usable, three pilot projects will provide the context for all implementation and evaluation activities:

(1) The Team Workbook - Fostering best practices through the intertwined teaching and use of planning methodologies such as Total Quality Management at the BAe Virtual University within BAe,

(2) The Experience Archive - Assisting the sales force and engineers to share expertise through enriched product documentation archives at TecInno with JOLA,

(3) ProGroup Electronic Manual - Supporting wide area organisational learning using a proactive electronic group manual to integrate multiple group memories at DFKI with Saarbergwerke AG.

We begin by enumerating key learning processes that an integrated organisational learning approach must address. Next, we analyse the shortcomings of the current state-of-the-art from technical and cognitive perspectives. We then describe our technical and methodological objectives and conclude with our project strategy, describing the activities and expected outcomes from each of the three cases.

2. NATURE OF WORKPLACE LEARNING

"Learning is the new form of labour" (Zuboff 1988) in a knowledge economy - it is absolutely vital that learning be effective and efficient. Yet, many industries rely on traditional `school models' for most of their educational needs even though workplace learning is fundamentally different from traditional school learning (Table 1). Empirical studies of professional practice, by ourselves and others (Lave 1991; Sachs 1995; Sumner 1995) , show that while the focus is primarily on getting the job done, learning is inextricably intertwined with working. In order to do their job, professionals must continually learn to apply existing knowledge to routine or innovative situations and to construct new knowledge in response to changing workplace situations. Thus, learning is fundamentally embedded in ongoing work activities and these work activities, in turn, give rise to the problems driving the learning that must take place.

Table 1. School Learning versus Workplace Learning

Our research is concerned with building computational environments integrating both working and learning. Our previous work focused on supporting individual learning (Sumner and Stolze 1996; Sumner, Bonnardel et al. 1997) , and knowledge modelling (Domingue, Motta et al. 1993; Motta and Zdrahal 1996; Motta 1998) in centralised work settings. In this project, we extend these previous efforts to the organisational level, and provide support for the integrated spectrum of learning activities (individual, group, and organisational) in distributed work settings. Before looking at the state-of-the-art, we will examine current theoretical perspectives and empirical findings of individual, group, and organisational learning to enumerate key challenges which an integrated approach must address.

2.1. Individual Learning: Becoming Reflective Practitioners

In his seminal book, "The Reflective Practitioner," Schön (Schön 1983) describes an action-breakdown-reflection cycle that underlies professional practice. In this cycle, practitioners engage in situated action until their expectations are not met and they experience a breakdown in the current work situation. At that moment, practitioners stop and reflect on how to overcome the breakdown before proceeding. These breakdowns in situated action present opportunities for learning because there is an opportunity to construct new contextualized knowledge while solving a personally relevant problem (Fischer 1994).

However, detecting and overcoming breakdowns requires much skill and domain-specific knowledge. People newly hired into a workplace will often lack this necessary knowledge. Even "old-timers" can be challenged. In complex domains, no one person can possess all the knowledge necessary to complete a task (Rittel and Webber 1984). In dynamic domains, what constitutes "necessary" knowledge is continually changing. Practitioners, both old and new, need to continually learn and relearn how to: (1) recognise potential breakdowns, (2) identify knowledge relevant to the breakdown, and (3) apply this knowledge or construct new knowledge to overcome the breakdown.

- Key challenges for supporting individual workplace learning are to support reflection-in-action processes by helping practitioners analyse their work products and detect potential problems or opportunities. Systems need to deliver relevant information to practitioners that they may be unaware of to assist their reflections.

2.2. Group Learning: Becoming Communities of Practice

These days, teams and groups form the core work units in many industries. However, while teams themselves are widespread, examples of effective group practices are less so. The literature is filled with stories of dysfunctional group working practices (Curtis, Krasner et al. 1988; Grudin 1991). Studies reveal that effective groups are good at "perspective making" (Boland and Tenkasi 1995) ; i.e., they share customs, conventions and standard practices that help to get the job done more effectively (Brown, Collins et al. 1989; Lave 1991; Sachs 1995). It is precisely these often tacit customs and conventions that form "best practices."

Our empirical studies of groups suggest that supporting communities to evolve their own vocabularies and to elaborate them to create a shared domain model is a critical step towards creating a common perspective (Sumner 1995) and an effective "community of practice." Shared vocabularies improve group communications; shared practices improve group coordination during complex tasks. Over time, as communities engage in negotiation and reflection about how to do their job better, their vocabularies and domain models become more elaborate and formal; i.e., their tacit understandings are articulated and refined towards more explicit knowledge forms. In many cases, they enrich their tools and work products with formal representations of their domain models in order to have better tool support for their work practices. The articulation and use of explicit domain models reinforce best practices by aiding the consistent reproduction and interpretation of work products. We refer to these processes of elaborating vocabularies, negotiating, and enriching as "domain construction" .

- A key challenge for supporting group learning is to support domain construction processes. Systems need to enable practitioners to articulate their informal understandings, to view and discuss their emerging ideas and domain models and, importantly, to incrementally modify domain models as their understandings change.

2.3. Becoming a Learning Organisation

Typically, organisations are composed of multiple interacting communities, each with highly specialised knowledge, skills, and technologies. Important tasks like product design and innovation in knowledge-intensive firms require these diverse communities to bridge their differences and integrate their knowledge and skills to create a new, shared perspective (Boland and Tenkasi 1995). Some researchers argue that this social sharing is the crucial first step towards knowledge creation (Nonaka and Takeucji 1995).

This bridging process is not so much one of passive `sharing' as one of active `perspective taking' and it is often complicated by the fact that a community's shared vocabulary or domain model is often tacit, making it uninspectable and difficult for another community to understand. Communities may share similar words and concepts at the surface level, but may actually be using them in entirely different ways (a phenomena dubbed `ontological drift' (Robinson and Bannon 1991)) resulting in communication and coordination problems. Sometimes this bridging is further complicated by time - the critical experts are no longer with the company or are otherwise unavailable for collaboration. Supporting such long-term, asynchronous collaboration is particularly important in industries needing specialised expertise that also rely on mobile and flexible workforces. Ultimately for perspective-taking to be successful, shared objects and work products must be re-interpreted and assigned a shared meaning - a process that usually requires much debate and negotiation.

- A key challenge for organisational learning is to support perspective taking. Systems need to support knowledge sharing across workplace communities and across time. However, sharing knowledge is different from simply sharing information - people need support for interpreting each others' perspective and for negotiating a new, shared perspective.

2.4. Integrating Individual, Group, and Organisational Learning

Building on Argyris and Schön's definition (Argyris and Schön 1978), we define organisational learning as: (1) a process that takes place through the agency of the individual members, (2) where individual and group learning experiences become incrementally embedded in organisational memory. An integrated approach is necessary because one cannot support organisational learning without supporting individual agency. However, we extend their definition to take into account the different types of learning processes at each of the three levels; i.e. individual, group, and organisational. Table 2 summarises requirements for our integrated approach.

Table 2. Requirements for an Integrated Organisational Learning Approach

3. STATE OF THE ART

Here, we examine the state-of-the-art with respect to the requirements outlined in Table 2. We begin by looking at an important type of structured discussion space, design rationale systems, which our approach builds on. Then we analyse related work in organisational memories, learning-on-demand systems, intranet-based document management systemsand proposed extensions to the Web that CEDAR builds on.

Design Rationale. Design rationale systems provide structured discussion spaces based on various notations such as issues, pros, and cons (Conklin and Begeman 1988) , and questions, options, and criteria (Buckingham Shum 1996) . These systems assume that as practitioners work, they also add their reasoning and justifications to the system using the provided notation and thus create an `organisational memory' as they work. Experiences indicate that design rationale systems can be very useful for supporting long-term asynchronous negotiation and collaboration across time (Fischer, Grudin et al. 1992) . However, one weakness of this approach is the lack of support for context: often there is little integration between work products and discussions about the products. This separation results in a loss of necessary context for understanding and interpreting both the design rationale and the products (Ruhleder 1994) . Integrating work products with their related discussion is also important for sustainability of the rationale repository; integration makes the repository easier to access during actual work, which in turn promotes making further additions to it (Fischer, Lemke et al. 1991) .

Organisational Memories. Organisational memories are motivated by the desire to preserve and share the knowledge and experiences that reside in an organisation. As such, most systems focus on capturing the knowledge, storing it, and making it accessible, rather than explicitly supporting the creation of new knowledge. By themselves, organisational memories are a necessary but insufficient step towards organisational learning. Several analyses based on case studies >(Ackerman 1993) and critiques from social and psychological perspectives (Bannon and Kutti 1996) conclude that maintenance of contextuality is of crucial importance for supporting learning or `active remembering'. Likewise, the generic nature of memories that attempt to serve all needs across large organisations often inhibits successful location and interpretation of relevant information. Recent efforts are instead targeting smaller, more focused approaches such as task-based memories (Ackerman and Mandel 1997) , methodology-based memories (Hidding 1997) and community memories that support the incremental evolution of both structure and content (Marshall, Shipman et al. 1994) . These approaches are a positive step towards enabling organisational memories to support the learning needs of individuals and groups.

Learning on Demand. Many approaches to supporting learning-on-demand or just-in-time learning are divorced from actual work contexts and, instead, are embedded within traditional curriculum-driven educational models. Their emphasis is on using the Internet to support distance learning. Other approaches intertwining working and learning in design domains, such as critiquing systems, have proven effective in supporting workplace learning for both newly hired and experienced designers (Sumner, Bonnardel et al. 1997). The challenge for this type of learning-on-demand is to have a rich shared context between the user and the system in order to determine the user's potential information needs (Fischer, Nakakoji et al. 1993) . Another crucial feature, particularly with regard to supporting reflection-in-action, is proactivity: systems need to point out potential problems at the `right time' when practitioners are best able to take advantage of the provided information (Lemke 1990) .

Intranet-based Document Management Systems. Intranets are the fastest growing segment of the Internet market. Intranets use World Wide Web technologies to support an organisation's internal information needs, which are often document-centred since documents permeate much of organisational practice (Brown and Duguid 1996) . Such uses were heralded by pre-Web groupware products such as Lotus Notes(TM) (Orlikowski 1992) . However, as recently noted by Xerox(TM) (with more than 200 servers and 20,000 users), "this internal Web, as an environment for supporting organisational work, is falling short of our expectations and hopes in significant ways" (pg, 81) (Rein, McCue et al. 1997) . They found off-the-shelf intranet software to be most successful when deployed to support simple, well-understood work practices where the information was factual and did not involve interpretation. They advocated creating a new style of intranet, with support for document management roles. Similarly, we view systems such as Intranets and Lotus Notes as important generic enabling technologies that we can build on to create organisational learning systems, rather than end-points in themselves.

Extensions to the Web. A number of extensions for placing semantic representations within Web documents have recently been put forward. The Resource Description Framework (RDF) (W3C, 1998b) is a draft proposal for representing metadata - data about the data - within a document. Encoding metadata into Web pages would have benefits in a number of areas. For example, semantic search engines and site catalogues could be created. At the core of RDF is a model for representing named properties and their values. The properties can be used to encode the attributes of and relationships between resources within a document. Because RDF has been designed to represent data and not knowledge its representational power is relatively weak when compared with standard knowledge modelling languages (see for example, section 5.3). At a syntactic level RDF models would be represented in XML (W3C, 1998a) a proposed extension to HTML which is at the recommendation stage.

Analysis of these Approaches. Common shortcomings across many of the above approaches are passivity, lack of extensibility, and lack of context. The passivity of many approaches requiring users to search for relevant information in large memory stores limits their ability to support reflection-in-action. Several approaches lack extensibility which inhibits their evolution in the workplace to take into account new knowledge or work practices; i.e., organisational memories that are difficult for practitioners to extend. Such non-extensible systems cannot support domain construction processes underlying group learning. Finally, common among many approaches was lack of support for context, either to assist human interpretation and learning (i.e., the separation of design rationale from work product) or to support intelligent system interpretation (e.g., the necessity of shared context for learning-on-demand mechanisms). In the next section, we will describe how our approach towards supporting organisational learning addresses these shortcomings.

4. OUR APPROACH: CONTEXTUALLY-ENRICHING DOCUMENTS

The core of our approach centres on enabling practitioners to progressively enrich their representations of work with important contextual cues and information arising from social discourse processes. A key contribution of our `enriching' approach is to capture contextual cues and information in such a way as to tightly couple them with representations of work.

When discussing the critical role of learning in the new knowledge economies, Zuboff described `smart machines' that could assist in actively `informating' practitioners as they work (Zuboff 1988) . In our view, it is not smart machines per se that serve to informate, but contextually-enriched documents since documents form the core of many business practices and are the objects that many practitioners work with daily.

In a recent study looking at document use by knowledge workers, Kidd found that the process of articulating and refining work products was often more important for informing practitioners than the products themselves (Kidd 1994) . In a broader context, Brown and Duguid contrast two basic models of document use: `documents as darts' where documents serve as a means of objectively transmitting knowledge and `documents as a means for supporting social processes' (Brown and Duguid 1996) . They argue that a key role of documents is to support negotiation and interpretation as communities struggle to reach a shared understanding. Likewise, several leaders in collaborative working and learning have argued for a re-thinking of the status of `representations of work' (Bannon 1995; Suchman 1995) . They argue that rather than regarding these work products as static accounts or descriptions of the way things are, representations should be regarded as starting points for discussion about the way things ought to be.

In our view, the key to integrating working and learning is to support these socially-based, process-oriented views of representations of work. As indeed, these processes derived from analyses of document use are the same as the processes underlying individual, group, and organisational learning: reflection, articulation, elaboration, negotiation, interpretation, and sharing.

The core of our approach centres on enabling practitioners to progressively enrich their `representations of work' with important contextual cues and information arising from these social processes. Representations of work take on many forms, including project specifications, design solutions, project bids, planning documents, etc. In most organisations, these representations take the form of documents, usually in paper form and increasingly in digital form as they are published on organisational intranets. Contextual cues and information takes on many forms including discussions surrounding the document, shared vocabularies or practices underlying the document, relationships to organisational competencies, and other related or dependent work products.

While this contextual information is vital for supporting key processes such as interpretation, we argue that simply capturing this information is insufficient. A key contribution of our `enriching' approach is to capture these important contextual cues and information in such a way as to tightly couple them with the representations of work. This enriching approach has three direct benefits:

* The richer context supports improved human-human communication and collaboration by keeping the context for interpreting a document coupled to the document itself.

* Enriching is a user-centred form of extensibility, encouraging practitioners to articulate their tacit understandings and incrementally refine them towards more explicit knowledge representations by reifying the context and its interconnections.

* The richer context makes possible active forms of computer support (i.e., learning-on-demand) by providing a richer machine-interpretable context.

5. CEDAR TOOLKIT AND METHODOLOGY

Earlier, we discussed how organisational memory approaches based on the `one-size-fits-all model' were problematic, resulting in memories that were too general to be useful or understood. Following our belief in the centrality of interpretation, we will focus on more specific forms: a large part of this project will be the creation of three task-specific instances of organisational memories based on our contextually-enriched document approach. Two key challenges for organisational memories are cost effectiveness and their sustainable integration into work practices. To meet these challenges, this project will develop a toolkit supporting the cost effective construction and customisation of task-specific memories, and a methodology guiding others in incorporating the memories into work practices to achieve sustained use and growth. We will develop an intranet-based architecture and an integrated toolkit (CEDAR) based on a central Contextually-Enriched Document ARchive server (the organisational memory) and distributed clients using our existing web-based collaboration, publishing, learning-on-demand, and knowledge modelling technologies.

CEDAR will enable (1) developers (information providers and knowledge engineers) to construct an initial organisational memory `seed', (2) end-users to view and extend the memory contents, and (3) agents to deliver critical information at the right time to support reflection-in-action.

Cedar will support two kinds of developers - knowledge engineers and information providers. It is envisioned that knowledge engineers will use CEDAR to create initial knowledge base `seeds' by re-using and refining existing model libraries. However, the key contribution of the CEDAR approach is to enable small teams of information providers to efficiently publish web-based documents linked to integrated discussion spaces and knowledge models without requiring extensive specialised technical knowledge. We expect the CEDAR toolkit to prove cost effective in two ways. Firstly, the toolkit will make it possible for information providers, such as corporate trainers from BAe's Virtual University, to construct organisational memories with only minimal assistance from knowledge engineers. Secondly, the toolkit will automate large parts of the publishing process which reduces both the time to create a site and the time required to verify a site's correctness. Experiences with Digital Document Discourse Environment (see section 5.2.) indicate that time savings of two orders of magnitude (from weeks to hours) are achievable (Sumner and Buckingham Shum 1998) .

A key issue in organisational memory research is sustainability: it is relatively easy to create a memory but it is more difficult to successfully integrate it into work practices to ensure use and growth. In this area, social issues are as important as technical issues. Successful efforts in creating sustainable memories recognise that using and contributing to the memory must be harmonised with both work practices and organisational practices (Terveen, Selfridge et al. 1993) . Surveys of our own experiences and the experiences of others highlight three lessons:

Lesson 1. A comprehensive memory lifecycle is needed to guide deployment, use, and maintenance. One lifecycle we will build on is the seeding, evolutionary growth, and reseeding model (Fischer, McCall et al. 1994) . According to this model, knowledge engineers work with end-users to create an initial memory `seed.' End-users extend the seed during use, contributing to evolutionary growth. At times after periods of extensive growth, knowledge engineers return to restructure and re-organise the memory, an activity called `re-seeding.'

Lesson 2. In recent times, there has been a realisation that there are no simple categories describing people's skills such as end-users versus developers. Rather a broad spectrum exists of people with varied skills and motivations with respect to technology. In many cases, effective and sustained use of technology has been accompanied by the emergence, and organisational acknowledgement of, a middle ground of technically sophisticated end-users, referred to as local developers (Gantt and Nardi 1992) or collection managers (Rein, McCue et al. 1997) . These people assume responsibility for maintaining and modifying the system to support changing work practices and organisational needs. A benefit of local developers is that they are part of the work community and have a deep understanding of existing work practices and needs.

Lesson 3. It is insufficient to disseminate technology by simply `throwing it over the wall' and assume that it will revolutionise work practices (Orlikowski 1992) . Studies have found that a proactive approach is needed to help practitioners foster the new `frames of reference' necessary for understanding the possibilities and appropriate uses of the new technologies (Orlikowski and Gash 1994) .

Building on these ideas, we will adopt an initial methodology based on participatory evolutionary development (PED) (Sumner and Stolze 1997) . In the PED model, individuals in the workplace, with participatory design and tool adaptation skills, help practitioners in co-evolving their tools and practices in appropriate ways. According to PED, the following roles and lifecycle processes should be supported:

Seeding. Knowledge engineers and information providers (such as corporate trainers) will work with local developers to create an initial archive seed using the CEDAR toolkit. We will use a variant of the VITAL Knowledge Based Systems (KBS) methodology (Jonker, Kontio et al. 1991) to create the initial seed. The VITAL project was a four and a half year research and development enterprise which aimed to provide both methodological and software support for developing large, industrial, embedded KBS applications. The VITAL knowledge engineering methodology is centred around the notion of a process product, which is an "essential and permanent deliverable produced in the course of a KBS project" (Jonker, Kontio et al. 1991).

Continuous Growth and Maintenance. Using CEDAR clients, practitioners (end-users) and local developers will contribute towards the growth of the archive through use. Local developers will assist practitioners in making more extensive changes to the archive structure and knowledge models, and will take responsibility for document collection management. Practitioners will enrich work documents by annotating their content using existing domain ontologies. An ontology (Gruber, 1993) is an explicit conceptualisation of a universe of discourse, that is the set of objects and relationships which specify a portion of the world under consideration. The methodology will give guidance on mapping work representations to ontological representations. In addition to mapping between existing ontologies and work representations enrichment may entail extensions to an ontology to cover idiosyncratic cases or innovative design solutions. Work representations will also be enriched with design rationale.

5.1. The CEDAR ToolKit

The CEDAR toolkit will be based on a server/client architecture communicating over a corporate intranet. At the heart of the system will be the CEDAR archive supported by a set of CEDAR client tools. The CEDAR client tools will allow the archive to be accessed via the Web. The clients will share a common infrastructure which will implement generic facilities such as synchronous communication.

5.1.1. The CEDAR Archive

CEDAR relies on an Organisational Memory (OM) consisting of contextually-enriched documents. These hypermedia documents are work products enriched with related communications (e.g., structured on-line discussions), community coordination data (models of competencies), task or domain-specific models, and broader models of corporate values. We support publishing of hypermedia work products based on standard web-protocols and formats. In addition to the OM the archive will consist of organisational memory maintenance tools and agents associated with organisational memory. The OM maintenance tools will allow the knowledge engineer to edit, browse and inspect the OM, initialise, create and update domain ontologies, and backup and archive the contents of OM.

5.1.2. The CEDAR Clients

To support learning while working, the CEDAR archive will be supported by client tools enabling:

Reflection-in-Action. Underlying knowledge models will be used to determine which parts of the memory are relevant to the current work context. Reasoning mechanisms will interpret and tailor selected knowledge to relate to the current task. This contextualized knowledge will be delivered to the user and presented in a way to enhance his learning processes. Consequently, the user can learn (a) from other members of the organisation or (b) from the store of previous organisational experiences. The user's results based upon learning while working are, in turn, fed back into the memory and preserved for future re-use.

Domain Construction Client tools will enable practitioners to enrich their hypermedia documents with links to knowledge models and communication traces stored in the CEDAR server as they work. The tools will support users to reify their emerging best practices by extending the organisational memory in two ways: (1) through on-line discourse coupled to work products and knowledge models, and (2) by extending knowledge models as their work progresses.

Perspective-Taking Practitioners will be able to share their expertise by exchanging contextually richer versions of their work products linked to underlying knowledge models and communication traces. This will assist interpretation and negotiation, enabling distributed work groups to comprehend each other's work and coordinate their activities.

From the perspectives of end-users, task-specific memories created using the CEDAR toolkit will have several common characteristics, including:

* a document-centred virtual discussion space enabling distributed communities to engage in debate and negotiation about their shared work products and their shared knowledge models. These discussions are tightly integrated with the work products, thus enriching the document with greater context.

* enriching mechanisms enabling them to extend knowledge models and interconnect models with specific documents and discussion threads, contributing to a richer document context.

* visible representations, such as concept maps, showing the current view of the structure of knowledge pertinent to their work group and organisation.

* mechanisms enabling end-users to publish and share their enriched work products with others.

* user interface agents (Lieberman 1997) that point out potential problems or missed opportunities in work products, point out possible links and dependencies to the work of other individuals or groups in the organisation, and inform user activities by providing relevant cases stored in the memory.

The bulk of the CEDAR toolkit will be based on the integration of existing toolkits and modelling libraries developed at the Knowledge Media Institute (KMI) - we will now briefly describe these systems and libraries.

5.2. Digital Document Discourse Environment

The Digital Document Discourse Environment (D3E) supports the publication of web-based documents with integrated discourse facilities and interactive components. D3E is based on extensive research into how hypertext systems can support critical reflection and the analysis of arguments in writing and software design (Buckingham Shum and Hammond 1994; Buckingham Shum 1996; Kolb 1997; Sumner, Bonnardel et al. 1997) . The first D3E project was to create an electronic journal on the Web (e-journal) supporting dialogue and debate between geographically distributed readers and authors. It became clear that production tools were needed to make the publishing of the journal tractable by a small team. It also became clear that there were many contexts where documents need to be discussed in different ways by different communities. This motivated the requirements for a tailorable publishing toolkit which could be used to generate a variety of sites.

D3E consists of tools for generating and managing a site, and tools supporting the document interface (Figure 1). The Publisher's Toolkit is a Java(TM) application that enables practitioners to publish HTML files with special forms of navigation and integrated discourse facilities without requiring them to have detailed knowledge of HTML. To support interpretation and negotiation within discourse communities, D3E generates a discussion space that is tightly coupled to the document (Figure 1) .

Figure 1: Output of the D3E toolkit given a source HTML article. On the left is the Article Window, on the right the Commentaries Window showing the top level outline view of discussion about the document. Key: [1] Comment icon embedded in each section heading: displays section-specific comments; [2] active contents list; [3] icon to display top level discussion outline, as shown on right; [4] icon to download Acrobat version; [5] citation automatically linked to reference in footnote window; [6] reverse link to citation; [7] links back into article; [8] general discussion heading defined in toolkit; [9] headings for section-specific comments.

To date, we have used D3E in numerous contexts, including the publication of several e-journals (JIME 1996; Buckingham Shum and McKnight 1997) , a national debate on government policy recommendations (KMI 1997a) , and a `new form' of academic conference (KMI 1997b) . Using D3E, the time required to publish an article with linked discourse facilities was significantly reduced, from approximately one week of person effort to about half an hour! What often starts as fairly conventional document forms (e.g., a text document from a word processor converted to HTML, or a slide presentation) are progressively enriched as surrounding discourse is reified and embedded back into the documents themselves. In the cases where D3E has been deployed, the processes of domain construction were facilitated by a combination of tool support and the redefining of participant roles (Sumner and Buckingham Shum 1998) . In this project, we will use D3E as a starting point providing web-based publishing and document-centred discourse. We will extend D3E using our knowledge modelling and knowledge sharing technologies, and context based retrieval agents (described below), enabling it to support richer forms of domain construction and reflection-in-action.

5.3. Knowledge Modelling Languages

Several knowledge modelling languages have been proposed in recent years. However, very few have been used beyond simple experiments with toy problems. Our current library of OCML (Operational Conceptual Modelling Language) (Motta 1998) models comprises several thousand definitions and has been used to model several application domains, such as geology, engineering design, and healthcare, and is used in Open University knowledge modelling courses (Open University 1997). OCML's property of operationality, together with the provision of several mechanisms for integrating OCML models with other software components, facilitates the rapid development of application systems. The language is available on a number of platforms, including Macintosh, PCs and Unix. We will use our extensive library of models to form a re-usable model substrate within the CEDAR toolkit. This library will assist users of the CEDAR toolkit in quickly constructing an initial knowledge model.

5.4. Knowledge Sharing Tools

Knowledge modelling is often a collaborative effort since the knowledge contained within a model should represent a common view shared within a community. The collaborative construction and maintenance of a model by a geographically dispersed community is a difficult task. Recently the World Wide Web (WWW) has been seen to offer a way of easing these problems. The Ontolingua group at Stanford (Farquhar, Fikes et al. 1995) under the Knowledge Sharing Effort (Neches, Fikes et al. 1991), and more recently the Ontosaurus project (Swartout, Patil et al., 1996), have used WWW technology to enable collaborative browsing and editing knowledge models. KMI has developed similar web-based tools such as WebOnto (Domingue 1998) which supports the collaborative construction of OCML models over the web (Figure 2).

Figure 2. A screen snapshot showing an ontology being edited using WebOnto. Within the large window the left panel shows items of a specified type within the current ontology (currently the instances within the kmi-planet ontology), the middle panel shows a well from which new ontological elements can be dragged to the right panel - a graphical browsing/editing area. The two small windows shows the textual representations of two class definitions.

5.5. Context-Based Retrieval Agents

The techniques for forming complex knowledge-enriched queries will capitalise on our experience of integrating case based reasoning systems with our web based ontology tools within the World-Wide Design Lab (WWDL) (Zdrahal and Domingue 1997). In WWDL we consider the generic case-based computational model to consist of a case-library and a reasoning engine. The reasoning engine performs two major tasks: case retrieval, and adaptation. The case retrieval task involves finding the case with the best match to the problem specification. This case then becomes the basis for the solution. In the adaptation step the case is evaluated and potential discrepancies are fixed. When this generic computation model is instantiated in a distributed collaborative environment efficiency problems arise. When a case library is organised in this fashion it is desirable that the number of Internet accesses is minimised since a high number of Internet accesses could make the tool too unresponsive to be of practical use. WWDL achieves this using an algorithm that takes only four steps no matter how large the case library.

Figure 3. The architecture of WWDL showing how we integrated a cased-based reasoning tool with the Web and our ontologies. The abbreviations in the figure are: BKA - book-keeping agent; CMA - case-matching agent; RMA - request manager agent, CL - case library.

Building on WWDL we view project teams as being composed of individuals working on overlapping pieces of work. In our framework work representations are underpinned by ontological descriptions. This means that overlapping work will be represented by common parts of the ontology. A Team Coordination Agent will keep track of which workers are responsible for each work product component. Whenever a shared part of a work product is changed (even if indirectly) the agent will notify the interested parties. In addition the agent will be able to deliver information on the source of the change including any stored rationale.

A Book-Keeping Agent will provide easy access to the documents in the OM. It will scan the published works of collaborating partners, find the latest versions and update its indexes to these documents. Ambiguities will be resolved by means of shared domain ontologies. This agent will operate either in the autonomous mode or on a user's request, for example, when the user wishes to increase the priority of publishing his or her work.

A Retrieval Agent will be responsible for finding episodes which match queries posed by workers. The matching will make use of the index library maintained by the book-keeping agent and the underlying ontological descriptions, thus dramatically reducing search time.

To facilitate the integration of our tools in a fashion which ensures that they will meet the needs of our users we will adopt an iterative and participatory (Greenbaum and Kyung 1991) development strategy where design and implementation activities are driven by use experiences in our three case studies, described in the following section.

6. RESEARCH STRATEGY

We will pay special attention to the process of integrating CEDAR tools into work practices. Rather than separating phases of development followed by deployment of pilot applications, the process of methodology and tool development will be intertwined with deployment from the very beginning. Early deployment will provide important feedback and will drive their further development. Specifically, we will use a case approach: three pilot projects will provide the context for all implementation and evaluation activities. Each case involves both types of CEDAR users - developers and end-users - but in different domains and business contexts (Table 3).

Table 3. Overview of the three pilot projects.

Our case approach enables users to immediately profit from the ENRICH approach by constructing a memory system based on their immediate business needs or the needs of a customer. We'll now describe each case, particularly the problem being addressed, the end-users, and the expected outcomes.

6.1. BAe Case: The Team Workbook

British Aerospace has in place a change programme called `BenchmarkBAe' designed to help the Company achieve its goal to become a benchmark company through the pursuit of excellence. This change process is being led by the CEO and the senior management team, who have identified five values - Customers, People, Performance, Innovation & Technology, and Partners - as keys to achieving this goal. Value Teams comprising members from the senior management team have established and defined practices and behaviours that drive and sustain the Company Values (published in 'Our Value Plan' issue 2 1998). This work is being consolidated into a series of educational programmes that will eventually embrace all employees. As part of this larger programme, a Team Workbook has been developed to help promote the understanding and application of these values in the operational and business environment.

The Team Workbook is a planning tool designed to stimulate more effective working of both team leaders and their teams. The Workbook is based around a three step process: (1) preparing the values-based plan, (2) declaring and delivering the plan, (3) reviewing and improving the plan and work practices. These iterative steps provide a learning framework enabling teams to secure continuous improvements in their sphere of responsibility. The current Workbook is paper based and is in the process of being deployed across the entire company (44,000 employees). Using the Workbook, local teams identify their internal customers and suppliers, map their key processes, measure their performance, and collectively work to secure higher levels of performance and greater customer satisfaction. The Workbook includes Total Quality Management (TQM) tools and is designed to be suitable for teams with little or no prior TQM experience. A support structure of trained facilitators within an overall change management framework is also being implemented.

During this project we will construct a digital Value-Enhanced Team Workbook underpinned by a corporate knowledge base representing the five values. The Value-Enhanced Team Workbook will support teams to articulate their plans directly within the Workbook and to discuss and review their plans on-line. It will enable teams (e.g., customers and suppliers) to share experiences and best practices by linking shared plans and values. It will also enable teams to feed into and augment the corporate knowledge base.

The Team Workbook is an ideal testbed for this project for several reasons. First, it provides a natural fit between business needs and our technical and methodological approach. While the paper-based workbook helps teams to become communities of practice, it does not directly contribute to organisational learning because the results of the planning activities are not captured, re-used or shared. Second, in parallel with the deployment of the Workbook, British Aerospace has established an internal browser-based intranet enabling us to use the intranet coupled with knowledge modelling tools to deliver the Workbook concept. Finally, we will leverage the support structure of the trained facilitators to test our proposed methodology.

In the Team Workbook case, we will:

* Formulate an initial ontology containing two interrelated layers. One layer will represent the corporate values and the other layer will represent key processes from the Workbook.

* Use CEDAR to construct the Value-Enhanced Team Workbook with embedded discourse facilities and underlying knowledge models.

* Integrate push (learning-on-demand) and pull (case-based retrieval) agents supporting the appropriate use of Workbook methodologies and the identification and delivery of related plans.

* Deploy the Value-Enhanced Workbook in two trials. The first trial will consist of one `shop floor' site in BAe experienced with using the paper-based Workbook. This site will use the Enhanced Workbook prototype to construct an initial plan. The second trial will involve two `shop floor' sites in BAe.

* Test and refine the CEDAR methodology, particularly in the areas of role articulation and training element requirements. This case will enable us to test our information provider and local developer roles in particular. Information providers (from BAe's Virtual University, the Workbook authors) will use CEDAR to construct the Value-Enhanced Workbook. We will examine how trained facilitators could play the role of local developers.

Expected Outcomes. In this case, we will focus on examining the usefulness and usability of the Value-Enhanced Team Workbook, and the relative contribution of linking groups via the underlying corporate knowledge model, according to the following working hypotheses:

Individual learning and reflection will be supported by providing practitioners with active support for constructing and reviewing plans using TQM methodologies. We will consider reflection to have occurred if individuals discuss through the workbook their plans or the methodologies. We will consider learning to have occurred if individuals change their plans based on agent intervention.

Group learning and domain construction will be supported by linking the plans to the underlying knowledge bases and enabling practitioners to enrich both their own plans and the knowledge base. We will consider domain construction to have occurred if practitioners enrich their own plans with links to the knowledge bases and if practitioners extend or discuss the underlying knowledge bases.

Organisational learning and perspective-taking will be supported by linking groups using the underlying corporate values model. We will consider perspective-taking to have occurred if practitioners engage in debate or negotiation about another team's plan with respect to their own planning objectives.

6.2 TecInno/JOLA Case: The Experience Archive

JOLA is a SME with 67 employees, that produces electro-mechanical and electronic devices. The company is a supplier of exception-detecting equipment. It has a diverse portfolio of products (e.g. level controls and leakage detectors) and operates in a complex market. JOLA is active in all countries of the EU, in many countries of Eastern Europe and in Israel, Korea and Malaysia. It has subsidiaries in France and in the United Kingdom and co-operates with distributors in other EU countries. Employees from the subsidiaries and sales distributors work together at the headquarters in Lambrecht, Rhineland-Palatinate and form a multicultural and multilingual team.

The customers of JOLA are mostly other SMEs from the area of construction and building as well as engineering. In its specific markets JOLA has to act very flexibly to meet diverse customer demands. The orders come in spontaneously and are often urgent due to the nature of the product line, e.g. leakage detectors, which often have to be installed in a very late stage of a construction project due to neglect during the planning phase. Sales people as well as engineers need access to numerous technical documents and best practices to be able to solve a customer`s problem quickly. Since JOLA has a small sales force with particular areas of expertise distributed over Europe, it needs efficient means to share knowledge and to train the sales force on the job. CEDAR in conjunction with an intranet offers the opportunity to meet this need.

In contrast to BAe, JOLA is a SME with very different organisational structures and resources. Consequently, we expect to learn different lessons than in the BAe case. The contribution of the JOLA case is to deliver a showcase of how to implement ENRICH technology in a SME environment. The BAe Team Workbook case, on the other hand, is of particular importance for the dissemination and the uptake of the results with end-users in a large corporate environment.

In the JOLA case, we will:

* Formulate an initial knowledge model (concept map) on the utilisation of JOLA products to allow for the capture and structuring of product know-how.

* Apply CEDAR to the technical product documentation to enrich its representation in order to enable further development and discussion.

* Integrate the approach into the organisation during the trial phases, to test possible new ways of collaboration and learning for the sales force and the engineering department.

Expected Outcomes. The enrichment of technical documents and the introduction of explicit knowledge models will enable JOLA engineers and sales people to share their expertise. Learning will take place when a sales person solves a new customer problem by applying JOLA products. This requires individual learning to tackle the new situation based on reflection on known problem solutions supported by case-based retrieval. The documented new product application will be shared among the sales people as well as with engineers to allow for group learning and the furtherance of domain construction. Such learning will in particular encourage the product improvement process at JOLA based on the discourse between sales and engineering. Organisational learning in this context is a long term perspective involving the strategic targeting of the product portfolio, and thus will be difficult to prove within the short term of the envisaged project. However, we will consider the important first step of perspective-taking across work communities to have occurred if sales people and engineers discuss and debate their respective viewpoints using the Experience Archive.

6.3 DFKI/Saarbergwerke Case: The ProGroup Electronic Manual

Saarbergwerke AG is an international energy and technology company with over 18,000 employees and annual sales of more than DM 4 billion. The corporate group is in the midst of change. Within the past few decades, its two main areas of operation, mining and power generation, have become the source of numerous new activities in the fields of energy, environment, trade and services, and rubber. Its range of services covers everything from providing foreign coal mines with state-of-the art technology, exporting innovative power station know-how, constructing regional long-distance heating networks, to working on combined solutions regarding waste disposal and environmental protection. This highly innovative environment continuously challenges the company's ability to build up, evaluate, and disseminate knowledge in crucial technical areas.

In previous projects with DFKI GmbH, Saarbergwerke AG developed and deployed a system supporting the systematic recording, structuring, and dissemination of knowledge and experiences relevant to the maintenance of complex mining machines. During machine operation, events, observations, and actions are documented and classified into ontological structures describing the machine configuration, diagnostic process, and observation/fault identifiers. These structures ease access to large archives of relevant technical documentation.

The systematic revision of knowledge entries supports the continuous evolution of the accumulated knowledge. Model-oriented, heuristic retrieval mechanisms realise a goal-oriented knowledge dissemination as an effective communication support. The results clearly demonstrated the high possible benefit which can be gained from IT support for organisational learning. In the face of the growing importance of global technology dissemination as a central business area for Saarbergwerke AG, the extension of this approach towards an encompassing memory for a global community of practice is an important business objective. The global community of practice emerges from the integration of various local task forces which are empowered to share their particular experiences.

In the ProGroup case, we will:

* Integrate multiple group memories at various locations into a common discourse environment, using web-based techniques provided by the CEDAR toolkit. This satisfies a key technical prerequisite for wide area organisational learning.

* Develop and realise a methodology for the collaborative creation of ontological structures shared between the local groups. This will be done with the CEDAR toolkit and the ENRICH methodology.

* Extend and complement the model-based pull mechanism of the recently deployed electronic fault recording system (ESB) by an active, context-sensitive push mechanism based on the retrieval machinery of the CEDAR case retrieval components.

* Develop push/pull learning-on-demand agents utilising the CEDAR text analysis components. These agents will analyse the accumulated experience database in order to automatically generate: suggestions for classifying experience logs and technical documentation, suggestions for new domain concepts, and explanations of faulty machine behaviour.

* Apply and test the approach by deploying the results in several coal mines equipped by Saarberg technology and continuously evaluating their use.

Expected outcomes. The results of the ProGroup study will extend the profitable use of recorded maintenance experience from a local to an organisation-wide, global scale. More specifically:

Individual learning and reflection is supported by the active presentation of relevant experiences, based on the experiences' organisation and structuring by explicit knowledge models. The presented information will immediately influence the ongoing diagnosis and repair process which in turn leads to immediate comments and extensions of the knowledge base. Technical documentation, as an, up to now, virtually not used knowledge resource, will be considerably better exploited.

Group learning and domain construction is supported by the extension of the explicit knowledge models on an as-needed basis as required for classifying completely new maintenance experiences. This domain model extension is done in cooperation between the machine operator on duty and the supervisor who maintains the knowledge-base off-line. Moreover, integration of the knowledge models of several groups (and of their experience databases as well) leverages and multiplies the usefulness of single experiences and stimulates discussions about different knowledge structures constructed by different groups.

Organisational learning and perspective-taking can be supported if knowledge structures (with respect to fault categories, critical operations, and standard repair procedures) emerging from the maintenance activities are discussed between people from engineering, planning, and training departments, thus influencing the construction of new machines, the planning of mining logistics, and the training of machine operators.

As in the BAe case, the ProGroup study shares the overall goal of continuously improving ongoing work processes and quality management. As in the JOLA case, improved exploitation of technical documentation plays a crucial role. Thus we expect valuable synergy among the three case studies. Additionally, the fine-grained technical knowledge structures and the focus on geographically distributed cooperation are distinguishing characteristics of the ProGroup pilot.

7. EVALUATION

Consistent with recommended case study approaches, we will collect several types of qualitative data in order to attain convergence (see Table 4) in our final analysis (Yin 1984) .

Table 4. Evaluation techniques for assessing support for integrated learning processes and business objectives.

Evaluation methods that will help assess the ENRICH approach include:

Server Log Analysis. The CEDAR memory server will maintain logs tracking usage patterns and changes to the contents and structure of the memory. This analysis method is useful for looking at longitudinal patterns and for looking at the actions of geographically distributed communities.

Organisational Memory Content Analysis. This involves looking at specific cases or products stored in the memory, specific parts of the knowledge model, and discussion analysis. These focused examinations complement the broader-brush log analyses by providing finer-grained accounts.

Workplace Observations. We will also conduct workplace observations at each case sites. The purpose of these observations are to gain an understanding of the tools in use and the impact of the tools work and organisational practices. These observations will be particularly helpful for refining our methodology.

Surveys. In cases involving geographically distributed users, we will use survey techniques rather than workplace observations.

Metrics. Metrics will be collected in order to estimate the time, cost, and effort of constructing a memory seed using CEDAR. These metrics will help to assess the toolkit's cost effectiveness.

8. SUMMARY

We believe the ENRICH project is innovative with respect to its technological basis, its integrated learning approach, and our case-oriented project strategy. We envision making important contributions in several areas:

- Our integrated learning approach is soundly based on theories of working and learning and on extensive empirical evidence.

- The CEDAR Toolkit will demonstrate cost effectiveness and practicality. A unique contribution will be our focus on making sophisticated technologies accessible to relatively non-technical information providers and cost-conscious SMEs.

- Three pilot projects in a wide spectrum of application domains (aerospace, engineering design, power industry) will serve to refine and assess the ENRICH approach in naturalistic settings. Each project will result in a readily exploitable application in its particular business domain, By comparing across the pilots and generalising from our common experiences, we will be in a better position to develop tools and methodologies with broad applicability across numerous business domains.

We are at the beginning of this project and the ideas and techniques presented here remain to be proved. However, we believe that our conceptual approach, which is soundly based on theories of learning and empirical studies, offers some insights into the nature of organisational learning which can usefully inform system design. We look forward to discussing these ideas at the workshop.

ACKNOWLEDGEMENTS

This research is funded by the ESPRIT programme for IT for learning and training in industry. We are grateful to Andreas Abecker, Marc Eisenstadt, Simon Buckingham Shum, and Markus Stolze for their comments on earlier drafts of this document.

REFERENCES

Ackerman, M. (1993). Definitional and Contextual Issues in Organizational and Group Memories., Hawaiian International Conference on System Sciences (HICSS 27), Hawaii (Jan), IEEE Computer Press.

Ackerman, M. and E. Mandel, "Memory in the Small: An Application to Provide Task-Based Organizational Memory for a Scientific Community," Technical Report http://www.ics.uci.edu/CORPS/ackerman.html, UC Irvine, 1997.

Argyris, C. and C. Schön (1978). Organizational Learning: A Theory of Action Perspective. Reading, Addison Wesley.

Bannon, L. (1995). "The Politics of Design: Representing Work." Communications of the ACM, 38(9): 66-68.

Bannon, L. and K. Kutti (1996). Shifting Perspectives on Organizational Memory: From Storage to Active Remembering. Hawaiian International Conference on System Sciences (HICCS 29), IEEE Computer Press.

Boland, R. and R. Tenkasi (1995). "Perspective Making and Perspective Taking in Communities of Knowing." Organization Science ,6(4): 350-372.

Brown, J. S., A. Collins and P. Duguid (1989). "Situated Cognition and the Culture of Learning." Educational Researcher ,Jan-Feb: 32-42.

Brown, J. S. and P. Duguid (1996). "The Social Life of Documents." First Monday, 1. http://www.firstmonday.dk/issues/issue1/documents/

Buckingham Shum, S. (1996). Analyzing the Usability of a Design Rationale Notation. Design Rationale: Concepts, Techniques, and Use. Hillsdale, NJ, Lawrence Erlbaum Associates. 185-215.

Buckingham Shum, S. and N. Hammond (1994). "Argumentation-Based Design Rationale: What Use at What Cost?" International Journal of Human-Computer Studies, 40(4): 603-652.

Buckingham Shum, S. and C. McKnight (1997). "(Eds.) World Wide Web Usability: Special Issue, Int. J. Human-Computer Studies." ,47 (1): 1-222. http://www.hbuk.co.uk/ap/ijhcs/webusability.

Conklin, J. and M. Begeman (1988). "gIBIS: A Hypertext Tool for Exploratory Policy Discussion." Transactions of Office Information Systems, 6(4): 303-331.

Curtis, B., H. Krasner and N. Iscoe (1988). "A Field Study of the Software Design Process for Large Systems." Communications of the ACM, 31(11): 1268-1287.

Detterman, D. K. (1993). The Case for the Prosecution: Transfer as an Epiphenomenon. Transfer on Trial: Intelligence, Cognition, and Instruction. Norwood, New Jersey, Ablex. 1-24.

Domingue, J. (1998). Tadzebao and WebOnto: Discussing, Browsing, and Editing Ontologies on the Web., Proceedings of the 11th Banff Knowledge Acquisition Workshop (KAW'98), Banff, Canada, (April 18-23).

Domingue, J., E. Motta and S. Watt (1993). The Emerging Vital Workbench., Knowledge Acquisition for Knowledge-Based Systems 7th European Workshop (EKAW '93), Toulouse and Caylus, France (September), Springer-Verlag.

Farquhar, A., Fikes, R., Pratt, W., and Rice, J. Collaborative Ontology Construction for Information Integration. Knowledge Systems Laboratory Department of Computer Science, KSL-95-63, August 1995.

Fischer, G. (1994). "Turning Breakdowns into Opportunities for Creativity." Knowledge-Based Systems Journal ,7(4): 221-232.

Fischer, G., J. Grudin, A. C. Lemke, R. McCall, J. Ostwald, B. N. Reeves and F. Shipman (1992). "Supporting Indirect, Collaborative Design with Integrated Knowledge-Based Design Environments." HCI, 7 (Special Issue on Computer Supported Cooperative Work) (3).

Fischer, G., A. C. Lemke, R. McCall and A. Morch (1991). "Making Argumentation Serve Design." Human Computer Interaction ,6(3-4): 393-419.

Fischer, G., R. McCall, Ostwald, B. Reeves and F. Shipman (1994). Seeding, Evolutionary Growth and Reseeding: Supporting the Incremental Development of Design Environments., Human Factors in Computing Systems (CHI '94), Boston, MA,

Fischer, G., K. Nakakoji, J. Ostwald, G. Stahl and T. Sumner (1993). Embedding Computer-Based Critics in the Contexts of Design., Human Factors in Computing Systems (Interact '93 and CHI '93), Amsterdam (24-29 April), ACM Press.

Gantt, M. and B. Nardi (1992). Gardeners and Gurus: Patterns of Cooperation Among CAD Users., Human Factors in Computing Systems (CHI '92), Monterey, CA (May 3-7), ACM Press.

Greenbaum, J. and M. Kyung (1991). Design at Work: Cooperative Design of Computer Systems. Hillsdale, NJ, Lawrence Erlbaum Associates.

Gruber, T. R. (1993). A Translation Approach to Portable Ontology Specifications. Knowledge Acquisition, 5(2), 1993.

Grudin, J. (1991). "Systematic Sources of Suboptimal Interface Design in Large Product Development Organizations." HCI, 6: pp. 147-196.

Hidding, G. J. (1997). "Reinventing Methodology: Who Reads it and Why?" Communications of the ACM ,40(11): 102-109.

JIME (1996). Journal of Interactive Media in Education, Knowledge Media Institute, The Open University, http://www-jime.open.ac.uk/.

Jonker, W., Kontio, J., and Motta, E. (1991) Definition and Positioning of the VITAL Project. VITAL Project Report ID732.1, PTT Research, Groningen, November 1991.

Kidd, A. (1994). The Marks are on the Knowledge Worker., Human Factors in Computing Systems (CHI '94), Boston, MA (April 24-28), ACM Press.

KMi (1997a). Dearing Report Discussion Site. Knowledge Media Institute, The Open University, U.K. http://kmi.open.ac.uk/Dearing/.

KMi (1997b). learning.org. Knowledge Media Institute, The Open University, U.K. http://kmi.open.ac.uk/learning.org.

Kolb, D. (1997). Scholarly Hypertext: Self-Represented Complexity, Proceedings of The Eighth ACM Conference on Hypertext, Southampton.

Lave, J. (1991). Situated Learning in Communities of Practice. Perspectives on Socially Shared Cognition. Washington, DC, American Psychological Association. 63-82.

Lemke, A. C. (1990). Framer-Hypertext: An Active Issue-Based Hypertext System. Proceedings of the Workshop on Intelligent Access to Information Systems (Darmstadt, Germany). Darmstadt, Germany, GMD-IPSI. 34-38.

Lieberman, H. (1997). Autonomous Interface Agents., Human Factors in Computing Systems (CHI '97), Atlanta, Georgia (March 22-27), ACM Press.

Marshall, C., F. Shipman and R. McCall (1994). Putting Digital Libraries to Work: Issues from Experiences with Community Memories., Digital Libraries'94, College Station, Texas (June 19-21).

Motta, E. (1998). Reusable Components for Knowledge Models. PhD Thesis. Knowledge Media Institute, The Open University.

Motta, E. and Z. Zdrahal (1996). Parametric Design Problem Solving., 10th Banff Knowledge Acquisition for Knowledge-Based System Workshop (KAW'96), Banff, Canada, (November 9th-14th).

Neches, R., Fikes, R., Finin, T., Gruber, T., Patil, R., Senator, T., and Swartout, W. R. (1991) Enabling Technology for Knowledge Sharing. AI Magazine, 12(3) pp. 16-36.

Nonaka, I. and H. Takeucji (1995). The Knowledge-Creating Company. Oxford, University Press.

Open University (1997) DM871 Intelligent Systems: Analysis and Design. Open University Master's Course in the Computers for Commerce and Industry Programme.

Orlikowski, W. (1992). Learning from Notes: Organizational Issues in Groupware Implementation., CSCW '92, Toronto, Canada, ACM Press.

Orlikowski, W. and D. Gash (1994). "Technological Frames: Making Sense of Information Technology in Organizations." ACM Transactions on Information Systems ,12(2): 174-207.

Rein, G., D. McCue and J. Slein (1997). "A Case for Document Management Functions on the Web." Communications of the ACM ,40(9): 81-89.

Rittel, H. and M. M. Webber (1984). Planning Problems are Wicked Problems. Developments in Design Methodology. New York, John Wiley & Sons. 135-144.

Robinson, M. and L. Bannon (1991). Questioning Representations., Second European Conference on Computer-Supported Cooperative Work (ECSCW'91), Amsterdam, The Netherlands (Sept. 25-27).

Ruhleder, K. (1994). "Rich and Lean Representations of Information for Knowledge Work: The Role of Computing Packages in the Work of Classical Scholars." ACM Transactions on Information Systems, 12(2): 208-230.

Sachs, P. (1995). "Transforming Work: Collaboration, Learning, and Design." Communications of the ACM (Special Issue on Representations of Work), 38(9): 36-44.

Schön, D. A. (1983). The Reflective Practitioner: How Professionals Think in Action. New York, Basic Books.

Suchman, L. (1995). "Making Work Visible." Communications of the ACM ,38(9): 56-64.

Sumner, T. (1995). The High-Tech Toolbelt: A Study of Designers in the Workplace., Human Factors in Computing Systems (CHI '95), Denver, CO (May 7-11), ACM Press.

Sumner, T. and S. Buckingham Shum (1998). From Documents to Discourse: Shifting Conceptions of Scholarly Publishing., Human Factors in Computing Systems (CHI '98), Los Angeles (April 18-23).

Sumner, T. and M. Stolze (1996). "Integrating Working and Learning: Two Models of Computer Support." AI & Society ,10(Special Issue on Hypermedia and Workplace Learning): 70-78.

Sumner, T. and M. Stolze (1997). Evolution, Not Revolution: PD in the Toolbelt Era. Computers and Design in Context. Cambridge, MIT Press. 1-26.

Sumner, T. R., N. Bonnardel and B. Kallak Harstad (1997). The Cognitive Ergonomics of Knowledge-Based Design Support Systems., Human Factors in Computing Systems (CHI '97), Atlanta, Georgia (March 22-27), ACM Press.

Swartout, B., Patil, R., Knight, K. and Russ, T. (1996) Toward Distributed Use of Large-Scale Ontologies, USA. Proceedings of the 10th Banff Knowledge Acquisition Workshop, Banff, Alberta, Canada, November 9-14, 1996.

Terveen, L., P. Selfridge and M. D. Long (1993). From "Folklore" to "Living Design Memory"., Conference on Human Factors in Computing (Interact '93 and CHI '93), Amsterdam (24-29 April), ACM Press.

W3C (1998a) Extensible Markup Language (XML) 1.0 W3C Recommendation 10th February 1998. http://www.w3.org/TR/REC-xml.

W3C (1998b) Resource Description Framework (RDF) Model and Syntax W3C Working Draft 16th February 1998 http://www.w3.org/TR/WD-rdf-syntax/.

Yin, R. (1984). Case Study Research: Design and Methods. Newbury Park, CA, SAGE Publications.

Zdrahal, Z. and Domingue, J. (1997) The World Wide Design Lab: An Environment for Distributed Collaborative Design. 11th International Conference on Engineering Design, August, 19-21 1997, Tampere, Finland.

Zuboff, S. (1988). In the age of the Smart Machine. New York, Basic Books, Inc.