Full Seminar Details

Susan Elliott Sim

 Susan Elliott Sim
Better Science Through Benchmarking: Lessons for Software Engineering
This event took place on Friday 04 June 2004 at 14:30

Benchmarking has been used to compare the performance of a variety of technologies, including computer systems, information retrieval systems, and database management systems. In these and other research areas, benchmarking has caused the science to leap forward. Until now, research disciplines have enjoyed these benefits without a good understanding of how they were achieved. In this talk, I present a process model and a theory of benchmarking to account for these effects. These were developed by examining case histories of existing benchmarks and my own experience with community-wide tool evaluations in software reverse engineering. According to the theory, the tight relationship between a benchmark and the scientific paradigm of a discipline is responsible for the leap forward. A benchmark operationalizes a scientific paradigm; it takes an abstract concept and turns it into a guide for research. Application of this theory will be illustrated using an example from reverse engineering: the C++ Extractor Test Suite (CppETS), a benchmark for comparing fact extractors for the C++ programming language. This talk will conclude with a discussion of how insights from studying benchmarking can improve the science in software engineering and collaboration in scientific communities more broadly.

(No replay available due to a shortage of technical staff to record event on the day)

Watch the webcast replay >>

Jobs

Research Assistant / Associate

Knowledge Media Institute (KMi)
"£29,301 - £38,183"
Based in Milton Keynes
Temporary contract - September 2017 until December 2018

The Knowledge Media Institute (KMi) is looking for a Research Assistant or a Research Associate to work jointly on two EU funded projects. The Hub4NGI project will strengthen and coordinate the work done by ongoing and upcoming projects, focusing...

Research Assistant / Associate

Knowledge Media Institute (KMi)
"£29,301 - £38,183"
Based in Milton Keynes
Temporary 15 months contract, commencing Octoberr 2017

The Knowledge Media Institute (KMi) is looking for a Research Assistant or a Research Associate to work on an EU funded project. The Hub4NGI project will strengthen and coordinate the work done by ongoing and upcoming projects, focusing on Next...

ERDF Project Secretary

Knowledge Media Institute (KMi)
"£21,843 - £24,565"
Based in Milton Keynes
Temporary contrat until 30th June 2019

The Open University’s Knowledge Media Institute (KMi) is a distinct research unit within the Faculty of Science, Technology, Engineering and Mathematics (STEM). The unit seeks an experienced Project Secretary to support the delivery management of...

CONTACT US

Knowledge Media Institute
The Open University
Walton Hall
Milton Keynes
MK7 6AA
United Kingdom

Tel: +44 (0)1908 653800

Fax: +44 (0)1908 653169

Email: KMi Support

COMMENT

If you have any comments, suggestions or general feedback regarding our website, please email us at the address below.

Email: KMi Development Team