Many language processing applications could benefit from knowing the intended meaning of each word in a piece of text. For example, one would not expect a question answering system when faced with the question 'Which plants thrive in chalky soil?' even to consider returning information about factories (the industrial rather than floral sense of 'plant').

Currently, the most successful approach to disambiguating word meaning involves training machine learning algorithms on text in which each word has been tagged by hand with its intended meaning. Unfortunately, manual tagging is extremely expensive, and there is only one large hand-tagged resource currently in existence, for English and containing text in a mixture of domains. Because even this resource contains sufficient information only for commonly occuring words and since word sense distributions are often highly skewed, systems have to fall back to always guessing the first, or predominant sense for many words.

Predominant sense information could be derived from hand-tagged resources, but this is only practical for English, and even then the predominant sense of a word can depend on the domain or source of a document. (The first sense of 'star' for example would be different in the popular press and scientific journals).

In this talk, I will describe a method for determining the predominant sense of a word in any given domain automatically from raw text, and report some experiments which show that the automatically inferred sense information can in some cases be more accurate than similar information derived from hand-annotated text.

Download powerpoint presentation (464kb ZIP file)

Watch the webcast replay >>