Tech Report

Feature Reduction for Document Clustering and Classification

Often users receive search results which contain a wide range of documents, only some of which are relevant to their information needs. To address this problem, ever more systems not only locate information for users, but also organise that information on their behalf. We look at two main automatic approaches to information organisation: interactive clustering of search results and pre-categorising documents to provide hierarchical browsing structures. To be feasible in real world applications, both of these approaches require accurate yet efficient algorithms. Yet, both suffer from the curse of dimensionality - documents are typically represented by hundreds or thousands of words (features) which must be analysed and processed during clustering or classification. In this paper, we discuss feature reduction techniques and their application to document clustering and classification, showing that feature reduction improves efficiency as well as accuracy. We validate these algorithms using human relevance assignments and categorisation.

Publication(s)

DTR 2000/8, Department of Computing, Imperial College London

ID: kmi-00-14

Date: 2000

Author(s): Stefan Rüger and Susan Gauch

Resources:
Download PDF

View By

Other Publications

Jobs

Research Asst / Assoc for Blockchain Learning Verification

Knowledge Media Institute (KMi)
£29,799 - £38,833 (Grade AC1 / AC2)
Based in Milton Keynes
Temporary contract for 18 months

The Knowledge Media Institute (KMi) is a distinct research unit within the Faculty of Science, Technology, Engineering and Mathematics (STEM) at the Open University. KMi is looking for a Research Assistant or a Research Associate to work on the IoC project & how blockchains can be used to store student accreditation and lifelong learning records in order to enhance employability. The Open University leads the IoC's first theme on university learning, which aims to influence computer science...

Research Asst / Assoc for Mainstreaming Learning Analytics

Knowledge Media Institute (KMi)
£29,799 - £38,833 (Grade AC1 / AC2)
Based in Milton Keynes
Temporary contract until 31st March 2021

The Knowledge Media Institute (KMi) is a distinct research unit within the Faculty of Science, Technology, Engineering and Mathematics (STEM) at the Open University. KMi is looking for a Research Assistant or a Research Associate to work on the IoC project & how we can adapt, deploy and mainstream our learning analytics tool across a number of educational establishments nationally. The Open University leads the IoC's first theme on university learning, which aims to influence computer...

CONTACT US

Knowledge Media Institute
The Open University
Walton Hall
Milton Keynes
MK7 6AA
United Kingdom

Tel: +44 (0)1908 653800

Fax: +44 (0)1908 653169

Email: KMi Support

COMMENT

If you have any comments, suggestions or general feedback regarding our website, please email us at the address below.

Email: KMi Development Team