Training Software Engineers in a Novel Usability Evaluation Technique

Novel approaches to designing or analysing systems only become useful when they are usable by practitioners in the field, and not just by their originators. Design techniques often fail to make the transition from research to practice because insufficient attention is paid to understanding and communicating the skills required to use them. This paper reports on work to train software engineering students to use a user-centred language for describing and analysing interface designs called the "Programmable User Model Instruction Language", or IL. Various types of data, including video, studentsÕ IL descriptions and brief usability reports were collected during training, and subsequently analysed. These show that after 6 hoursÕ training, students have a good grasp of the syntax of the notation, and are starting to use notational affordances to support their reasoning, but that their reasoning is still limited by a poor grasp of the underlying cognitive theory. A comparison of the analyses of trainees with those of experts provides a means of developing a better understanding of the nature of expertise in this area -- as comprising an understanding of the syntax and the surface semantics of the notation, the underlying cognitive theory, the method of conducting an analysis and the implications of the analysis for design.

1. School of Computing Science, Middlesex University, Bounds Green Road, London, N11 2NQ, UK.

2. Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, UK.

3. MRC Applied Psychology Unit, 15 Chaucer Road, Cambridge, CB2 2EF, UK.

Publication(s)

International Journal of Human-Computer Studies (1998, in press)