Home > Talks

Adaptive Kernel Regression for Image Processing and Reconstruction

We have developed a class of robust nonparametric estimation methods which are ideally suited for the reconstruction of signals and images from noise-corrupted and sparse or irregularly sampled data. The framework results in locally adapted kernels which take into account both the spatial density of the available samples, and the actual values of those samples. As such, they are automatically steered and adapted to both the given sampling "geometry", and the samples' "radiometry". As the framework we propose does not rely upon strong assumptions about noise or sampling distributions, it is applicable to a wide variety of problems, including image upscaling, high quality interpolation from irregular, sparse and noisy samples, state of the art denoising, and deblurring.

Multidimensional Kernel Regression for Video Processing and Reconstruction

We introduce a novel framework for adaptive enhancement and upscaling of videos containing complex activities based on multidimensional kernel regression. In this framework, each pixel in the video sequence is approximated with a 3-D local (Taylor) series, capturing the essential local behavior of its spatio-temporal neighborhood. The coefficients of this series are estimated by solving a local weighted least-squares problem, where the weights are a function of the 3-D space-time orientation in the neighborhood. As this framework is fundamentally based upon the comparison of neighboring pixels in both space and time, it implicitly contains information about the local motion of the pixels across time. The proposed approach not only significantly widens the applicability of super-resolution methods to a broad variety of video sequences containing complex motions, but also yields improved overall performance.

Precise Multi-Frame Motion Estimation and Its Applications

In addressing the problem of estimating the relative motion between the frames of a video sequence. In comparison with the commonly applied pairwise image registration methods, the proposed method considers global consistency conditions for the overall multi frame motion estimation problem, and is more accurate. We review the recent work on this subject and propose an optimal framework, which directly applies the consistency conditions as both hard constraints in the estimation problem, or as soft constraints in the form of stochastic (Bayesian) priors. The framework is applicable to virtually any motion model and enables us to develop a robust approach, which is resilient against the effects of outliers and noise, and therefore useful for demanding applications such as superresolution.

(Super) Resolution: Statistical Definition, Computation, and Fundamental Limits

In this talk, I present an overview of our work in the statistical analysis of resolution, its computational enhancement in imaging, and its inherent fundamental limits. On the computational image reconstruction front, we have developed efficient, and statistically optimal algorithms for superresolution from video, spanning a wide range of problems in this field, including robust multi-frame image fusion, simultaneous demosaicing and super-resolution, and color video-to-video super-resolution.On the more fundamental analysis, having developed a proper statistical definition of resolution, we have been able to analyze the resolution/SNR tradeoff for general optical imaging systems, deriving a practically useful scaling law relating them. Furthermore, we have performed related analysis demonstrating and quantifying the deep interdependence between the motion estimation and image reconstruction problems, thereby yielding insight into fundamental limits of each, and how the knowledge of one set of parameters affects the other. I touch on most of the above inter-related topics, and describe some of them at higher resolution.

Mask Design for Resolution Enhancement: An Inverse Problem in Optical Microlithography

Optical microlithography, a technique similar to photographic printing, is used for transferring circuit patterns onto silicon wafers. The above process introduces distortions arising from optical limits and non-linear resist effects, leading to poor pattern fidelity and yield loss. The input to the above system is a photo-mask (or reticle), which can be controlled (or engineered) such that it cancels out (or compensates for) the process losses to come. This forms the basis of optical proximity correction (OPC) and phase shift masks (PSM), two commonly employed resolution enhancement techniques for patterning very small features (close to the optical limit). In this talk, we discuss a novel inverse lithography technology (ILT) framework to synthesize OPC and PSM for high-fidelity patterning of random logic 2-D features. ILT attempts to synthesize the input mask which leads to the desired output wafer pattern by inverting the mathematical forward model from mask to wafer. Our framework employs a pixel-based mask parameterization, continuous function formulation, and analytic gradient-based optimization techniques to synthesize the masks. We also introduce a regularization framework to control the tone and complexity of the synthesized masks, and inculcate other user-defined properties. The results indicate automatic generation and placement of assist bars, which are very popular in the semiconductor industry. We conclude by briefly discussing ILT-based mask design for double exposure lithography systems, which are deemed as key technology enablers for the future.

Reconstructing Shapes From Brightness Functions

In this talk we address the problem of reconstructing the shape of a convex object from measurements of the areas of its shadows in several directions. This type of very weak measurement is sometime referred to as the brightness function of the object, and may be observed in an imaging scenario by recording the total number of pixels where the object's image appears. Related measurements, collected as a function of viewing angle, are also referred to as "lightcurves" in the astrophysics community, and are employed in estimating the shape of atmosphereless rotating bodies (e.g. asteroids). We address the problem of shape reconstruction from brightness functions by constructing a least-squares optimization framework for approximating the underlying shapes with polygons in two dimensions, or polyhedra in three dimensions, from noisy, and possibly sparse measurements of the brightness values.