Tech Report

Feature Reduction for Document Clustering and Classification

Often users receive search results which contain a wide range of documents, only some of which are relevant to their information needs. To address this problem, ever more systems not only locate information for users, but also organise that information on their behalf. We look at two main automatic approaches to information organisation: interactive clustering of search results and pre-categorising documents to provide hierarchical browsing structures. To be feasible in real world applications, both of these approaches require accurate yet efficient algorithms. Yet, both suffer from the curse of dimensionality - documents are typically represented by hundreds or thousands of words (features) which must be analysed and processed during clustering or classification. In this paper, we discuss feature reduction techniques and their application to document clustering and classification, showing that feature reduction improves efficiency as well as accuracy. We validate these algorithms using human relevance assignments and categorisation.

Publication(s)

DTR 2000/8, Department of Computing, Imperial College London

ID: kmi-00-14

Date: 2000

Author(s): Stefan Rüger and Susan Gauch

Resources:
Download PDF

View By

Other Publications

Jobs

Front-end Web Developer and UX Designer (GR7)

Knowledge Media Institute (KMi)
£33,199 - £39,609
Based in Milton Keynes
Temporary contract up to 20 months

The team at the OU runs the world's largest aggregator of research papers called CORE (core.ac.uk), with about 1.5 million monthly active users, and a set of projects promoting principles of Open Science, such as FOSTER (fosteropenscience.eu). CORE provides free access to millions of Open Access research papers as well as a number of information services for researchers, organisations and businesses. These include services enabling text & data mining, recommender systems, content management...

Front-end Web Developer and UX Designer (GR6)

Knowledge Media Institute (KMi)
£27,025 - £32,236
Based in Milton Keynes
Temporary contract up to 20 months

The team at The Open University runs the world's largest aggregator of research papers called CORE (core.ac.uk), with about 1.5 million monthly active users, and a set of projects promoting principles of Open Science, such as FOSTER (fosteropenscience.eu). CORE provides free access to millions of Open Access research papers as well as a number of information services for researchers, organisations and businesses. These include services enabling text & data mining, recommender systems, content...

Research Asst / Assoc / Fellow for Blockchain Learning Verification

Knowledge Media Institute (KMi)
£30,395 - £44,559 (Grade AC1 / AC2 / AC3)
Based in Milton Keynes
Temporary contract until 31st December 2021

The Knowledge Media Institute (KMi) is a distinct research unit within the Faculty of Science, Technology, Engineering and Mathematics (STEM) at the Open University. KMi is looking for a Research Assistant, a Research Associate or a Research Fellow to work on the IoC project - how blockchains can be used to store student accreditation and lifelong learning records in order to enhance employability. The Open University leads the IoC's first theme on university learning, which aims to...

Mobile Applications Developer (GR6)

Knowledge Media Institute (KMi)
£27,025 - £32,236
Based in Milton Keynes
Permanent Appointment

The Research and Innovation Software team develop software for a range of research projects and University courses. With this role we are looking for someone who can be a mobile applications developer for the team. This would involve both maintaining our existing set of mobile applications and undertaking the development of new mobile applications using the Unity development environment. You will also be required to extend your capabilities over time to include Augmented Reality (AR) and...

Research Asst / Assoc / Fellow for Mainstreaming Learning Analytics

Knowledge Media Institute (KMi)
£30,395 - £44,559 (Grade AC1 / AC2 / AC3)
Based in Milton Keynes
Temporary contract until 31st March 2021

The Knowledge Media Institute (KMi) is a distinct research unit within the Faculty of Science, Technology, Engineering and Mathematics (STEM) at the Open University. KMi is looking for a Research Assistant, a Research Associate or a Research Fellow to work on the IoC project - how we can adapt, deploy and mainstream our learning analytics tool across a number of educational establishments nationally. The Open University leads the IoC's first theme on university learning, which aims to...

Research Assistant / Associate

Knowledge Media Institute (KMi)
£30,395 - £39,609 (Grade AC1 / AC2)
Based in Milton Keynes
Temporary contract until 30th April 2020

The Knowledge Media Institute (KMi) is looking for a Research Assistant or a Research Associate (depending on qualification), to work on EU funded project - Up2U. The project will be focusing on the context of secondary schools, often referred to as high schools, which provide secondary education between the ages of 11 and 19 depending on the country, after primary school and before higher education. The learning context from the perspective of the students is the intersection of formal...

CONTACT US

Knowledge Media Institute
The Open University
Walton Hall
Milton Keynes
MK7 6AA
United Kingdom

Tel: +44 (0)1908 653800

Fax: +44 (0)1908 653169

Email: KMi Support

COMMENT

If you have any comments, suggestions or general feedback regarding our website, please email us at the address below.

Email: KMi Development Team