Narrative, Multimodality and Multimedia Content Analysis: computing words, pictures and stories
This event took place on Tuesday 07 November 2006 at 11:30
Dr. Andrew Salway
Semantic technologies for the production, dissemination, retrieval and browsing of multimedia information all require the generation of machine-processable descriptions of the meanings conveyed by the multimedia information, i.e. its content. Typically, metadata is generated by the automatic analysis of raw multimedia data and is then used to structure, index, summarise and browse multimedia data collections. Two fundamental questions arise. What forms should the descriptions of multimedia content take? And how can the descriptions be generated automatically? In this seminar I will argue that recent studies of narrative and multimodality have important contributions to make to the specification and design of data structures and algorithms for multimedia content analysis. After reviewing relevant theories and analytic frameworks from the fields of narratology, semiotics and multimedia discourse analysis, I will focus on findings from recent research into the extraction of narrative structures in feature films, and the modeling of image-text relations in web pages. Potential applications of this work include film retrieval based on story similarity and hypervideo based on story structures, and enhanced web search engines that fuse meanings from the image and text components of web pages.
By way of background… Narrative is an important concept because much multimedia data exists to tell a story, be it a feature film, a news story, or somebody’s life story realized in their personal media collection. To date however, most techniques for multimedia content analysis describe the topic, or aboutness, of media items in terms of the entities and events referred to / depicted, without describing the connections between them that make them into a story. That multimodality is an important concept for multimedia content analysis should go without saying – a defining characteristic of multimedia is the fact that the multimedia whole conveys more than the sum of its parts. However, much research has concentrated on the analysis of individual media types (image, video, text, audio) in isolation - their integration, if addressed at all, is ad-hoc. Recent advances in multimedia discourse analysis have produced insights into how different media types, e.g. image and text, combine to convey meaning in print, film, and new media forms.
Download presentation slides
This event took place on Tuesday 07 November 2006 at 11:30
Semantic technologies for the production, dissemination, retrieval and browsing of multimedia information all require the generation of machine-processable descriptions of the meanings conveyed by the multimedia information, i.e. its content. Typically, metadata is generated by the automatic analysis of raw multimedia data and is then used to structure, index, summarise and browse multimedia data collections. Two fundamental questions arise. What forms should the descriptions of multimedia content take? And how can the descriptions be generated automatically? In this seminar I will argue that recent studies of narrative and multimodality have important contributions to make to the specification and design of data structures and algorithms for multimedia content analysis. After reviewing relevant theories and analytic frameworks from the fields of narratology, semiotics and multimedia discourse analysis, I will focus on findings from recent research into the extraction of narrative structures in feature films, and the modeling of image-text relations in web pages. Potential applications of this work include film retrieval based on story similarity and hypervideo based on story structures, and enhanced web search engines that fuse meanings from the image and text components of web pages.
By way of background… Narrative is an important concept because much multimedia data exists to tell a story, be it a feature film, a news story, or somebody’s life story realized in their personal media collection. To date however, most techniques for multimedia content analysis describe the topic, or aboutness, of media items in terms of the entities and events referred to / depicted, without describing the connections between them that make them into a story. That multimodality is an important concept for multimedia content analysis should go without saying – a defining characteristic of multimedia is the fact that the multimedia whole conveys more than the sum of its parts. However, much research has concentrated on the analysis of individual media types (image, video, text, audio) in isolation - their integration, if addressed at all, is ad-hoc. Recent advances in multimedia discourse analysis have produced insights into how different media types, e.g. image and text, combine to convey meaning in print, film, and new media forms.
Download presentation slides
Future Internet
KnowledgeManagementMultimedia &
Information SystemsNarrative
HypermediaNew Media SystemsSemantic Web &
Knowledge ServicesSocial Software
Semantic Web and Knowledge Services is...
Our research in the Semantic Web area looks at the potentials of fusing together advances in a range of disciplines, and applying them in a systemic way to simplify the development of intelligent, knowledge-based web services and to facilitate human access and use of knowledge available on the web. For instance, we are exploring ways in which tnatural language interfaces can be used to facilitate access to data distributed over different repositories. We are also developing infrastructures to support rapid development and deployment of semantic web services, which can be used to create web applications on-the-fly. We are also investigating ways in which semantic technology can support learning on the web, through a combination of knowledge representation support, pedagogical theories and intelligent content aggregation mechanisms. Finally, we are also investigating the Semantic Web itself as a domain of analysis and performing large scale empirical studies to uncover data about the concrete epistemologies which can be found on the Semantic Web. This exciting new area of research gives us concrete insights on the different conceptualizations that are present on the Semantic Web by giving us the possibility to discover which are the most common viewpoints, which viewpoints are mutually inconsistent, to what extent different models agree or disagree, etc...
Our aim is to be at the forefront of both theoretical and practical developments on the Semantic Web not only by developing theories and models, but also by building concrete applications, for a variety of domains and user communities, including KMi and the Open University itself.
List all Semantic Web and Knowledge Services Projects
Check out these Hot Semantic Web and Knowledge Services Technologies:
List all Semantic Web and Knowledge Services Technologies
