About KMi

Human and Machine Annotation for Collective Sensemaking

Knowledge Media Institute, The Open University, Milton Keynes, UK
3 year fully-funded PhD (Oct. 2012-Sept.2015)
Stipend: £40,770 (£13,590/year)

Recent studies suggest that social annotation is key to several sensemaking tasks such as reflection, self-analysis and examination of changes, assessment and learning [1]. Social annotation is a means to reflect on personal and collective interpretation of documents. This activity is particularly important when dealing with information-intensive intellectual tasks, which require collaboration of several stakeholders and powerful scaffolding of thinking and reflection [2]. Nonetheless human annotation of documents is a highly cognitive and time-consuming task.

How can we effectively support social annotation of documents in a way which reduces reading time and supports the identification and reflection on key contents and the sensemaking of key issues?

Automatic annotation of texts aims at helping users to rapidly and reliably find pre-defined relevant information in large document collections. When a large document collection or even an open set of documents from the entire web is the basis for collective reflexion, the limited possibilities of human annotation in terms of quantity could be complemented by machine annotation.

Various computer platforms have been developed to support human annotation by slightly formalizing it for further processing, like allowing making links among documents and annotations, indexing and retrieving them. Nonetheless, at present automatic annotation systems and systems for supporting human annotation are used separately.

The aim of this PhD reaseach is to investigate the design, implementation and testing of an integrated system of human and machine annotation to support collective sensemaking.

The PhD candidate will build on cutting edge KMi research and technologies for annotating documents in collective sensemaking tasks, in what we have defined as Contested Collective Intelligence (CCI) settings [3]. In particular, the Cohere platform (http://cohere.open.ac.uk/) [4] allows social annotation of Web documents and enhances the possibilities of collective reflection by pointing out main questions, crucial inconsistency and contrasting ideas. This allows members of organizations to collectively seize the complexity of the issues at hand.

The PhD candidate will be invoved in the study, improvement and integration of technolgies such as Cohere with state-of-the-art discourse analysis software like the discourse module of the Xerox Incremental Parser (XIP)[5]. XIP detects sentences in which the authors indicate contrasting ideas, gaps in knowledge, open questions etc, which are central elements in Contented Collective Intelligence processes [6].

The integration of advanced social annotation tools such as Cohere with Natural Language Processing and discourse analysis software such as Xerox's XIP will lay down the first experimental foundations of an integrated human and machine annotation approach for collective sense-making, with the goal of assisting the collective annotation, summarization and evaluation of a large collection of argumentative documents.

PhD Challenge

This PhD will fund a candidate to design, implement and test an integrated system of human and machine annotation for collective sensemaking.

You will contribute to the future of Collective Intelligence (CI) and CSCW systems which combine human annotation to harness machine analysis and reasoning power. To do so, you will work with mixed-initiative approaches and investigate how to couple automated services with direct human manipulation, in an attempt to take advantage of the power of human intelligence and valuable automated reasoning.

A number of theoretical and practical questions will be the focus of this PhD study:

  • How can human and machine annotation be combined in a unique sensemaking process?
  • How can results of human and machine annotation be visualized, explored and searched?
  • How to cope with the complexity of the resulting network of human and machine annotations?
  • How to evaluate collective intelligence systems which are built from the combination of human and machine annotation?
  • How to design, implement and test an integrated system that takes optimal advantage both of the rapidity and rigour of the machine and of the unique human insight?
Seize the Day

This is a fantastic opportunity if you're passionate about the future of Computer Supported Cooperative Work (CSCW) and want to engage with next generation NLP and Sensemaking Technologies which base on Discourse theory, analysis and practice.

If you're already a great web and database developer, and now looking to develop a research career in Collective Intelligence technologies, this is your chance to contribute to one of the fastest growing topics in Social Web science.

Your supervisors will be Simon Buckingham Shum and Anna De Liddo, who are active contributors to the field of CSCW and Collective Sensemaking. The PhD will be in collaboration with scientists at XEROX Research Center, who are well placed to provide guidance and input on NLP and Machine Annotation research. In particular, this work will be done in partnership with Ágnes Sándor, project leader at Xerox Research Centre Europe (XRCE) (Meylan, France), whose research into Natural Language Processing and Discourse Analysis provides a foundational element for this project.

Open University is the place to do research in Hypermedia and Sensemaking technologies for the Social Web. The Knowledge Media Institute is an 80-strong state of the art research lab, prototyping the future for the Open University and the many other organisations with whom KMi partners. KMi is renowned for its creative, can-do culture, and its high impact on the OU's strategic thinking and technical capacity [Locate KMi]. The Xerox Research Centre Europe (XRCE) is a multidisciplinary research centre with scientists in computer science, human-computer interaction, natural language processing, machine learning applied to information, data and documents. Its mission is to be a centre of excellence for the understanding and design of technology that enables innovative business processes and empowering IT services for knowledge workers in industry. The differentiating approach of the centre is the design new technology and services based on the knowledge gathered by XRCE's own work practice specialists. XRCE is part of the global Xerox Innovation Group made up of 800 researchers and engineers in four world-renowned research and technology centres.


For further information on this PhD project please contact:

Dr Simon Buckingham Shum
Senior Lecturer & Associate Director
Email | Dr Simon Buckingham Shum Website | Simon Buckingham Shum
+44 (0)1908 655723

Dr Anna De Liddo
Research Associate
Email | Dr Anna De Liddo Website | Dr Anna De Liddo
+44 (0)1908 653591


[1] Kalnikaité, Vaiva and Steve Whittaker (2008): Social summarization: does social feedback improve access to speech data? In Proceedings of CSCW 2008, ACM Press, New York, pp 9-12.

[2] Lin, Xiaodong, Cindy Hmelo, Charles K. Kinzer and Teresa J. Secules (1999): Designing technology to support reflection. Educational Technology Research and Development, 47, (3), 43-62.

[3] De Liddo, Anna and Buckingham Shum, Simon (2010). Cohere: A prototype for contested collective intelligence. In: ACM Computer Supported Cooperative Work (CSCW 2010) - Workshop: Collective Intelligence In Organizations - Toward a Research Agenda, February 6-10, 2010, Savannah, Georgia, USA. http://oro.open.ac.uk/19554/.

[4] Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. http://oro.open.ac.uk/10421/.

[5] Sándor, Á. (2006). Using the author's comments for knowledge discovery. Semaine de la connaissance, Atelier texte et connaissance, Nantes, June 29.2006.

[6] De Liddo, Anna; Sándor, Ágnes and Buckingham Shum, Simon (2012, In Press). Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work Journal (CSCW). Eprint: http://oro.open.ac.uk/31052.


The relevant application form can be found at http://www.open.ac.uk/research/research-degrees/overview.php.

It is essential that you include both a proposal and a CV with your application. These are central to our judging of applications, both at shortlisting time and afterwards.

Application submissions can be directed to KMi Recruitment Coordinator at the Knowledge Media Institute, Open University, Milton Keynes, MK7 6AA, UK, Tel. +44 (0)1908 654774, Fax +44 (0)1908 653169.

Application Deadline

7th June 2012

The Open University
KMi 2013 - A review of the year

Download the KMi 2013 Review of the year iBook to your iOS device or alternatively as a PDF.

Journal | 25 years of knowledge acquisition