Multimodal Scene Analysis

Recognizing objects in scenes is one of the most fundamental and challenging problems in computer vision. The main objective of our core research area Multimodal Scene Analysis is the detection, recognition and analysis of 2D and 3D objects in static and natural scenes.

We tackle several major challenges that arise in scene analysis:
How to detect and recognize objects from different viewpoints while under varying illumination and being occluded by other objects, or objects with clutter backgrounds. Moreover, we focus on finding a proper paradigm that is able to close the semantic gap in the broad field of object recognition and scene analysis.

 

As an expert group, we approach these challenges in several ways. Firstly, we work on improving the performances of traditional object representation and matching methods by employing additional novel features and matching strategies. Secondly, we incorporate several modalities which can be easily acquired by nowadays commercially available devices, i.e. depth, multispectral and colour data. Thirdly, we fuse other modalities like sound and haptic feedback information. The acquisition of such a large amount of data requires extremely efficient and robust algorithms in order for them to be analysed.

Therefore, mechanisms such as feature extraction and selection have to be adapted according to our demands. At the same time, we are working on developing a generic, adaptive and easily extensible model that is capable of handling this mass of information. The adaptive component continuously monitors the results which are rated by the response of an external supervisor. This feedback is used to re-adjust the model parameters and new relations between low-level features can be established in order to form mid and high level features.