Suche
News
You are here
Title: Object Shape Generation, Representation and Matching
Author: Cong Yang
Supervisor: Prof. Dr. Marcin Grzegorzek
Reviewers: Prof. Dr. Marcin Grzegorzek and Prof. Dr. Dietrich Paulus
Date of defence: 26.09.2016
Abstract:
Shape matching and recognition is a fundamental aspect of many problems in computer vision, including object or scene recognition, moving tracking and object detection, etc. While rigid shape matching is relatively well understood, matching shapes undergoing nonrigid deformations remains challenging. Moreover, shape generation is also a challenging task since nearly all approaches face the same diffculty: background clutter. The aim of the research in this thesis is to present novel approaches to generate, represent and match object shapes. To achieve this, the following three problems are addressed in particular:
Shape Generation: Shape generation is generally applied based on shape contour detection which is a process to locate an object, identify its contour parts, and segment out its contour. After that, an object shape can be generated by properly filling hollow spaces. In order to detect shape contour parts (open curves), a Contour Segment (CS)-based open curve matching approach is introduced. CS is the basic element for representing and matching open curves. However, there exist various CS descriptors and the discrimination power of each descriptor is unclear. Furthermore, deep insights are needed regarding the suitable combination between CS descriptors and their correlated matching methods. Therefore, this thesis studies and evaluates 26 CS descriptors in a structured way. Formed on evaluations, the recommended CS descriptors and their correlated matching algorithms are employed for open curve matching.
Shape Representation: Shape-based object representation looks for effective and perceptually important shape features based on either shape boundary or region information. However, there are three main challenges. The first one is how to extract efficient descriptors that are invariant to shape rotation, translation and scaling. The second one is how to extract shape descriptors that are robust to noise and distortions. The third challenge is how to generate descriptors with low computational complexity.
In order to solve these problems, in this thesis, one way is to propose simple geometry descriptors (coarse-grained descriptors) for capturing coarse-grained shape features and then fuse them with existing descriptors to improve the description power of individuals. Another way is to propose rich descriptors (fine-grained descriptors) which have a higher description power since the “detailed” shape features are captured. Experiments show that the proposed coarse-grained shape descriptors can significantly improve the description power of existing descriptors without adding too much computational complexity. Furthermore, the proposed fine-grained descriptors achieves the state-of-the-art performances in several standard datasets.
Shape Matching: Based on descriptors, shape matching aims to calculate the overall similarity (or dissimilarity) between two object shapes. There is no general model for shape matching since different descriptors may have different feature vector structures. In addition, even with the same structure, a matching method is required by considering both accuracy and speed. Specifically, for coarse-grained shape descriptors, the matching between shapes is usually conducted by using simple vector distance functions. Thus, the matching accuracy could be improved by optimising distance functions without increasing the computational complexity. In contrast, fine-grained shape descriptors have complex structures and the matching process is normally applied based on correspondences and statistics. Therefore, the matching accuracy could be improved by exploring extra information from descriptors to enhance the correspondences and statistics.
With these observations, several matching algorithms are proposed in this thesis. Specifically, for coarse-grained descriptors, a supervised optimisation strategy is proposed to improve the flexibility and adaptability of their feature vectors. For fine-grained descriptors, inspired by the idea of high-order graph matching, the singleton, pairwise and third-order potential functions are proposed to solve the interesting point and hierarchical skeleton matching problems. In addition to the coarse- and fine-grained shape descriptors, three integrated descriptors are also delivered and evaluated.
Lastly, two shape-based applications are introduced and assessed using the proposed descriptors: (1) With a coarse-grained descriptor, an automatic system for classifying Environmental Microorganism (EM) is introduced. Their classification is a very important indicator for biological treatment processes and environmental quality evaluations. Experimental results certify the effectiveness and practicability of the proposed EM classification system as well as the shape-based method. (2) With a fine-grained descriptor, a novel skeleton-based approach is proposed for audio envelope analysis. To effectively describe the envelope of audio signals, a skeleton-based descriptor and its matching method are introduced. The application of violin sound matching proves the usability of these approaches.
In order to solve these problems, in this thesis, one way is to propose simple geometry descriptors (coarse-grained descriptors) for capturing coarse-grained shape features and then fuse them with existing descriptors to improve the description power of individuals. Another way is to propose rich descriptors (fine-grained descriptors) which have a higher description power since the “detailed” shape features are captured. Experiments show that the proposed coarse-grained shape descriptors can significantly improve the description power of existing descriptors without adding too much computational complexity. Furthermore, the proposed fine-grained descriptors achieves the state-of-the-art performances in several standard datasets.
Shape Matching: Based on descriptors, shape matching aims to calculate the overall similarity (or dissimilarity) between two object shapes. There is no general model for shape matching since different descriptors may have different feature vector structures. In addition, even with the same structure, a matching method is required by considering both accuracy and speed. Specifically, for coarse-grained shape descriptors, the matching between shapes is usually conducted by using simple vector distance functions. Thus, the matching accuracy could be improved by optimising distance functions without increasing the computational complexity. In contrast, fine-grained shape descriptors have complex structures and the matching process is normally applied based on correspondences and statistics. Therefore, the matching accuracy could be improved by exploring extra information from descriptors to enhance the correspondences and statistics.
With these observations, several matching algorithms are proposed in this thesis. Specifically, for coarse-grained descriptors, a supervised optimisation strategy is proposed to improve the flexibility and adaptability of their feature vectors. For fine-grained descriptors, inspired by the idea of high-order graph matching, the singleton, pairwise and third-order potential functions are proposed to solve the interesting point and hierarchical skeleton matching problems. In addition to the coarse- and fine-grained shape descriptors, three integrated descriptors are also delivered and evaluated.
Lastly, two shape-based applications are introduced and assessed using the proposed descriptors: (1) With a coarse-grained descriptor, an automatic system for classifying Environmental Microorganism (EM) is introduced. Their classification is a very important indicator for biological treatment processes and environmental quality evaluations. Experimental results certify the effectiveness and practicability of the proposed EM classification system as well as the shape-based method. (2) With a fine-grained descriptor, a novel skeleton-based approach is proposed for audio envelope analysis. To effectively describe the envelope of audio signals, a skeleton-based descriptor and its matching method are introduced. The application of violin sound matching proves the usability of these approaches.