Search Engine-Crawler Symbiosis: Adapting to Community Interests
This event took place on Monday 15 December 2003 at 11:30
Dr Shannon Bradshaw
Web crawlers have been used for nearly a decade as a search engine component to create and update large collections of documents in order to serve the widely varying needs of web users. Typically the crawler and the rest of the search engine are not closely integrated, because the objective of the crawler is simply to gather as broad a sample of the Web as possible.
This approach to search engines while effective for simple information needs such as the take away menu for a local restaurant is less effective for searches requiring a range of documents in a focused topic space. Such information needs are regularly encountered in research and business.
In this talk I will present an approach to building search engines targeted to specific communities with shared focused interests. This work differs from previous approaches to focused crawling in that the focus of the system automatically changes as the needs and interests of its community evolve. Our approach is based on a tightly coupled system in which a crawler and a search engine engage in a symbiotic relationship. The crawler feeds the search engine and the search engine in turn helps the crawler to better its performance.
We show that symbiosis can help the system learn about a community's interests and serve that community's web search needs with better focus.
This event took place on Monday 15 December 2003 at 11:30
Dr Shannon Bradshaw
Web crawlers have been used for nearly a decade as a search engine component to create and update large collections of documents in order to serve the widely varying needs of web users. Typically the crawler and the rest of the search engine are not closely integrated, because the objective of the crawler is simply to gather as broad a sample of the Web as possible.
This approach to search engines while effective for simple information needs such as the take away menu for a local restaurant is less effective for searches requiring a range of documents in a focused topic space. Such information needs are regularly encountered in research and business.
In this talk I will present an approach to building search engines targeted to specific communities with shared focused interests. This work differs from previous approaches to focused crawling in that the focus of the system automatically changes as the needs and interests of its community evolve. Our approach is based on a tightly coupled system in which a crawler and a search engine engage in a symbiotic relationship. The crawler feeds the search engine and the search engine in turn helps the crawler to better its performance.
We show that symbiosis can help the system learn about a community's interests and serve that community's web search needs with better focus.
Future Internet
KnowledgeManagementMultimedia &
Information SystemsNarrative
HypermediaNew Media SystemsSemantic Web &
Knowledge ServicesSocial Software
Social Software is...
Interacting with other people not only forms the core of human social and psychological experience, but also lies at the centre of what makes the internet such a rich, powerful and exciting collection of knowledge media. We are especially interested in what happens when such interactions take place on a very large scale -- not only because we work regularly with tens of thousands of distance learners at the Open University, but also because it is evident that being part of a crowd in real life possesses a certain 'buzz' of its own, and poses a natural challenge. Different nuances emerge in different user contexts, so we choose to investigate the contexts of work, learning and play to better understand the trade-offs involved in designing effective large-scale social software for multiple purposes.
List all Social Software Projects
Check out these Hot Social Software Technologies:
List all Social Software Technologies
