ITEM METADATA RECORD
Title: Large-scale Classification and Retrieval of 3D Shapes
Other Titles: Opzoeking en classificatie van 3D objecten op grote schaal
Authors: Knopp, Jan
Issue Date: 27-May-2015
Abstract: This thesis focuses on three main topics in the area of 3D recognition: shape classification, retrieval, and scene segmentation.
For classification, in contrast to previous methods that focused on the overall shape appearance only, we extended the popular SURF features into a third dimension and presented a novel method to learn the efficient 3D Implicit Shape Model (3D ISM), which is a class-specific star model that combines the appearance and the relative position of local patches from 3D shapes.
The proposed shape retrieval approach is based on our previous classification model. Thus, 3D ISM introduces additional spatial constraints to improve the basic bag-of-visual-words ordering. This ISM-based verification step allows us to use shape expansion that gains even better results. In addition, we show how to usenbsp;retrieval pipeline to improve shape matching, classification, and text-based 3D shape search. The method was also practically used in the 3D Coform project in a search for artifacts in a museum’s repository.
The previously introduced method for classification was only applied to recognize classes of shapes that are isolated, clean, and without holes. Instead, segmenting large 3D scenes is a more realistic scenario. Objects need to be firstly found and then segmented. We propose two methods. The first, combines our previous findings in classification with CRF optimization and the second is a more sophisticated framework that solves ISM and graph optimization jointly. Once the object is found and segmented from its background, we use our method (3D ISM) or introduced new Boltzmann Machines-based approach to fill-in holes and to correct the wrong parts of incomplete shape.
Each task is evaluated on several benchmarks against a variety of state-of-the-art methods.
Description: Knopp J., ''Large-scale classification and retrieval of 3D shapes'', Proefschrift voorgedragen tot het behalen van het doctoraat in de ingenieurswetenschappen, KU Leuven, May 2015, Leuven, Belgium.
Table of Contents: Abstract v
Glossary ix
Contents xi
List of Figures xvii
List of Tables xxi
1 Introduction 1
1.1 Motivation and objectives . . . . . . . . . . . . . . . . . . . . . 2
1.2 Why is 3D recognition challenging? . . . . . . . . . . . . . . . . 3
1.3 Contributions & Overview . . . . . . . . . . . . . . . . . . . . . 4
2 Background 7
2.1 Representing real-life objects as 3D shapes . . . . . . . . . . . . 7
2.1.1 What are objects and their categories? . . . . . . . . . . 7
2.1.2 Former 3D recognition . . . . . . . . . . . . . . . . . . . 8
2.1.3 3D to represent the real world . . . . . . . . . . . . . . 9
2.1.4 Description of 3D shapes . . . . . . . . . . . . . . . . . . 11
2.2 Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.5 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3 3D SURF & Three-dimensional ISM for shape classification 21
3.1 Introduction and related work . . . . . . . . . . . . . . . . . . . 22
3.2 Shape representation as the set of 3D SURF features . . . . . . 23
3.2.1 Feature detector . . . . . . . . . . . . . . . . . . . . . . 24
3.2.2 Feature descriptor . . . . . . . . . . . . . . . . . . . . . 25
3.3 Implicit Shape Model for 3D classification . . . . . . . . . . . . 27
3.3.1 Visual Words Construction . . . . . . . . . . . . . . . . 27
3.3.2 Learning and Weighting Votes . . . . . . . . . . . . . . 28
3.3.3 Determining a Query Shape’s Class . . . . . . . . . . . . 31
3.4 Evaluation of 3D SURF features . . . . . . . . . . . . . . . . . 32
3.4.1 Sensitivity to parameters . . . . . . . . . . . . . . . . . 32
3.4.2 Performance of 3D SURF vs. state-of-the-art detectors/descriptors. . 34
3.5 3D SURF and ISM for 3D classification . . . . . . . . . . . . . 36
3.5.1 3D shape classification of reconstructed real life scenes . 38
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4 Rotation invariant ISM 41
4.1 Introduction & Related Work . . . . . . . . . . . . . . . . . . . . 41
4.2 Hough transform based classification . . . . . . . . . . . . . . . 43
4.3 Rotation invariant voting . . . . . . . . . . . . . . . . . . . . . 43
4.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.4.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.4.2 Experimental Results . . . . . . . . . . . . . . . . . . . 48
4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5 Verification and expansion to improve shape search, classification,
matching, and text-based shape search 51
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.3 Shape search using verification . . . . . . . . . . . . . . . . . . 55
5.3.1 Initial Bag-of-Words search . . . . . . . . . . . . . . . . 55
5.3.2 Structural verification step . . . . . . . . . . . . . . . . 56
5.3.3 Query expansion . . . . . . . . . . . . . . . . . . . . . . 58
5.4 Evaluation of verification and expansion . . . . . . . . . . . . . 59
5.4.1 Competitors . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.4.2 Evaluation of shape retrieval . . . . . . . . . . . . . . . 62
5.4.3 Expansion for classification . . . . . . . . . . . . . . . . 64
5.4.4 Expansion for denser shape matching . . . . . . . . . . 65
5.5 Text-based shape search . . . . . . . . . . . . . . . . . . . . . . 65
5.5.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.5.2 Improving text-based shape search . . . . . . . . . . . . 66
5.5.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . 68
5.6 Shape search on archaeological data . . . . . . . . . . . . . . . 68
5.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6 Segmentation of 3D scenes 73
6.1 Introduction and Related work . . . . . . . . . . . . . . . . . . 73
6.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . 75
6.3 The method: ISM and MinCut for object segmentation . . . . 77
6.3.1 MinCut for efficient object segmentation . . . . . . . . . 77
6.3.2 Definition of the recognition based unary term . . . . . 78
6.3.3 Definition of the pairwise term . . . . . . . . . . . . . . 78
6.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.4.1 Unary term competitors . . . . . . . . . . . . . . . . . . 80
6.4.2 Parameters setup . . . . . . . . . . . . . . . . . . . . . . . 81
6.4.3 Results & discussion on synthetic data . . . . . . . . . . 82
6.4.4 Application on SfM data . . . . . . . . . . . . . . . . . . 83
6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7 Joint object detection and segmentation 87
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
7.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
7.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.3.1 Initialization . . . . . . . . . . . . . . . . . . . . . . . . 92
7.3.2 Voting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
7.3.3 Evaluating Hypotheses . . . . . . . . . . . . . . . . . . . 93
7.3.4 Back-Projection . . . . . . . . . . . . . . . . . . . . . . 94
7.3.5 Handling the Background . . . . . . . . . . . . . . . . . 95
7.4 Implementation Details . . . . . . . . . . . . . . . . . . . . . . 95
7.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
8 Shape Completion 101
8.1 Introduction and related work . . . . . . . . . . . . . . . . . . . 102
8.2 The method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
8.2.1 Energy-based models . . . . . . . . . . . . . . . . . . . . 104
8.2.2 The basic RBM . . . . . . . . . . . . . . . . . . . . . . . 104
8.2.3 Training the RBM . . . . . . . . . . . . . . . . . . . . . 107
8.2.4 Completion using the RBM . . . . . . . . . . . . . . . . 109
8.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
8.3.1 The competing techniques . . . . . . . . . . . . . . . . . . 111
8.3.2 Evaluating . . . . . . . . . . . . . . . . . . . . . . . . . 112
8.3.3 Relations between hidden and visible nodes . . . . . . . 116
8.4 Improving scene’s objects . . . . . . . . . . . . . . . . . . . . . 116
8.4.1 The method & results . . . . . . . . . . . . . . . . . . . 117
8.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
9 Conclusion & Future Work 121
9.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Bibliography 123
Publication status: published
KU Leuven publication type: TH
Appears in Collections:ESAT - PSI, Processing Speech and Images

Files in This Item:
File Status SizeFormat
thesis.pdf Published 19159KbAdobe PDFView/Open

 


All items in Lirias are protected by copyright, with all rights reserved.