Author:

Keywords:

PSI_VISICS

Abstract:

This doctoral thesis deals with the enhancement of digital images by increasing their resolution, a field commonly referred to as super-resolution. Classically this problem was solved by acquiring multiple images of the same scene and computing a sub-pixel accurate registration, thereby gaining a combined insight into intra-pixel areas that wasn't available in the source images. More recently techniques were suggested that make single-image super-resolution possible through the use of well-chosen image priors and low- and high-resolution example image pairs. Typically these techniques use a database of small low- and high-resolution image patch pairs to learn the relationship between local features in low-resolution and high-resolution versions of the same image. The goal of this thesis is to introduce novel single-image super-resolution methods that push the boundaries of current methods in terms of execution speed and output quality. The first contribution we make is a generalization of the patch-matching problem. So far, the patches used for single-image super-resolution have typically been assumed to be small patches of a fixed size and aspect ratio. However, there is no reason why these patches shouldn't have a more general definition which allows for many different sizes and ratios depending on their content. We propose an efficient method to find the optimal shape and size for each matching patch pair and we test it for two popular patch-matching applications: single-image super-resolution and nonlocal means denoising. Our second contribution is the introduction of a sparsity-based super-resolution method that focusses on improving execution speed by moving a large part of the algorithm to the training phase, thereby improving the test-time execution speed dramatically. We refer to this method as Anchored Neighborhood Regression (ANR) because we use local neighborhoods that are anchored to dictionary atoms to perform offline regression. We then extend this method to A+: an adjusted formulation that significantly outperforms other state-of-the-art methods in terms of quality by making use of a potentially near-infinite pool of exemplar patches. It keeps the computational efficiency of ANR by keeping the extra time constraints associated with large databases in the offline phase. Our third contribution is the exploration of semantic priors for single-image super-resolution. We show that using semantic priors such as image segmentations and labels can significantly improve output quality, and even in cases where the quantitative improvement is small we can still see significant local qualitative improvement. We propose a semantic framework based on our A+ method and investigate the theoretical limits and practical benefits of using semantic priors. Finally, we suggest a systematic evaluation method on large standard image datasets to compare the output quality of different super-resolution algorithms. Our proposed method is inspired by human visual perception, as opposed to commonly used evaluation metrics like Peak-Signal-to-Noise-Ratio (PSNR) or Structural Similarity (SSIM). Aside from these contributions, we perform a case study of two specific applications of super-resolution: Minimally invasive surgery and license plate enhancement.