In dynamic environments, such as web-based cultural heritage sites, where neither the individual user requirements nor the requested material can be predicted in advance, an automated presentation generation process is required. For that, however, the system relies on semantic, episodic, and technical representation structures that provide full experience to the user by means of montage. Such constructivistic needs of new media require more than characterizing audio-visual information on a perceptual level using objective measurements, such as those based on image or sound processing or pattern recognition. These retrospective technologies understand media rather from the point of view of automatically index multimedia information.

Though machine-generated metadata is cheap to produce, it is insufficient because it s exclusively organized around the media structures? sensory surfaces that is, the physical features of an image, or video, or audio stream. There is lots of evidence, however, that a great deal of required annotation can be provided by manual labour. In this talk we investigate the notion of individual and purpose-driven audio-visual media annotation during the production process. The aim is to make use of human activity to extract the significant syntactic, semantic and semiotic aspects of the media content as well as the related process, which then can be transformed into formal descriptions. The resulting descriptions form a conceptual information space, in which authors are semantically supported in their creative efforts to generate, manipulate, and exchange information. The approach is demonstrated with examples mainly taken from encyclopaedic spaces from domains such as of theory, history, and anthropology of film or cultural heritage, as provided by museums for the fine arts.

Watch the webcast replay >>