The extraction of semantic information from multimedia content is a research area that faces multiple challenges: scalability; data scarcity; multiple statistical models for each modality; computational limitations when processing large-scale training datasets; incorrect ground truth...

To address some of the issues hindering multimedia retrieval applications we propose a novel learning framework to extract semantic multimedia information. The framework combines both knowledge and statistical data, and it is divided in three parts: (1) multimedia mining, (2) multi-modal information fusion, and (3) semantic information extraction.

We will discuss several aspects of the framework, such as, scalability, its solid statistical foundation (borrowed from Generalized Linear Models and Bayesian Theory), how it is able to elegantly cope with different modalities, and its performance on semantic image retrieval and large-scale semantic video retrieval.

Watch the webcast replay >>