Author:

Abstract:

Multimedia applications play a cutting-edge role in the revolution towards 3-dimensional visual processing and communication, e.g. 3DTV, natural human-computer interface, and augmented/virtual-reality. Multi-view technologies are essential to the exciting progress, which are capable of extracting and rendering the 3-dimensional real world from 2-dimensional images. Besides attractive functionalities, multi-view algorithms are usually computationally intensive. Moreover, real-life multimedia applications have multiple requirements, e.g. high visual quality, real-time performance and low cost. Given such demanding requirements, one critical research issue is how to reach the optimal designs in the ever-expanding design space. In the thesis, we study the approach of algorithm-architecture co-design, by developing complexity-aware algorithms and leveraging on parallel computing platforms. The research objective of the thesis is two-fold. First, we develop efficient designs of critical multi-view modules on emerging parallel computing platforms. These designs should have both competing algorithmic quality and real-time execution speed, which can be adopted by practical systems. Second, based on the designs, we distill a set of design techniques and principles for efficient visual computing, including development of efficient algorithms and architecture-oriented designs. These principles can be applied to approach the optimal designs in a systematic way.In the thesis, efficient algorithms for sparse feature detection and dense stereo matching have been developed. To reach real-time designs, we have explored three representative parallel computing platforms, i.e. multi-core central processing units (CPUs), graphic processing units (GPUs), and field programmable gate arrays (FPGAs). Aiming at real-time multi-view processing, the thesis has made several contributions: 1) an efficient parallel detection algorithm for MSER features, 2) an adaptive-weight approach for local stereo matching, 3) an adaptive-shape approach for local stereo matching, 4) two real-time high-quality stereo matching designs on GPUs and FPGAs respectively, 5) a set of concrete design principles for efficient visual computing. The proposed algorithms and designs have achieved leading performance among state-of-the-art methods.