Multimodal Analysis For Bridging Semantic Gap with Biologically Inspired Algorithms

The amount and complexity of digital media being generated, stored, transmitted, analysed and accessed has exponentially increased as a result of advances in computer and Web technologies. Much of this information combines digital images, video, audio, graphics and textual data. Large-scale online video repositories enable users to creatively share material along a wide audience. Consequently, there is an increasing interest in associating media items with free-text annotations, ranging from simple titles and detailed descriptions of the video content. In an effort to reduce the complexity of the annotation task, this talk will outline some of the techniques developed for indexing large-scale multimedia repositories by exploiting multi-modality of information space. One such approach combines the use of semantic expansion and visual analysis for predicting user tags for online videos. The framework is designed to exploit visual features using biologically inspired algorith ms and associated textual metadata, which is semantically, expanded using complementary textual resources. The experimental results indicate the usefulness of the proposed approach for analysing large-scale media items.