Icassp2003.com

Yonina Eldar, Technion, Israel

Abstract

The problem of estimating a deterministic parameter vector from noisy measurements has countless applications in science and engineering. The overwhelmingly most popular approach to estimation in such scenarios is the maximum likelihood (ML) principle. One of the main justifications for the ML method is that under suitable regularity conditions it asymptotically achieves the Cramer- Rao lower bound on unbiased estimators. However, there are many examples in which the ML strategy leads to unreasonable estimators. Furthermore, it is well known that biased methods can result in a smaller mean-squared error (MSE), which is a direct measure of the estimator’s ability to predict parameter values. Thus, instead of utilizing data driven methods, such as the ML approach, we discuss a framework for designing estimators with good MSE behavior.

An important special case of this general model is the Gaussian linear regression problem in which the goal is to estimate a deterministic vector x from noisy measurements y = Hx + w where H is a known model matrix and w is a Gaussian noise vector. The ML estimator for this problem coincides with the well-known least-squares (LS) method. In 1956, Stein shocked the statistics community by showing that estimators exist which dominate LS, meaning that their MSE is smaller than that resulting from the LS approach, for all values of x. This surprising phenomenon led to a surge of research into biased methods that dominate the LS strategy for the regression model.

In the first part of the tutorial we survey results on LS domination and introduce the recent blind minimax framework for generating a wide class of low-complexity, LS-dominating estimators which are constructed from a simple intuitive principle. We also treat the problem of evaluating the performance of different strategies based on the concepts of admissibility and dominance. In the second part of the tutorial we demonstrate how these ideas can be extended to more general, nonlinear models by developing performance benchmarks on the MSE that are smaller than the traditional Cramer-Rao lower bound for all values of the unknowns. The derivation suggests explicit constructions of estimators that achieve these bounds in cases where an efficient ML or minimum variance unbiased (MVU) estimator exist, by performing a simple linear transformation of the ML or MVU solution. This leads to estimators that result in a smaller MSE than the ML (MVU) method for all possible values of the unknowns.

• Introduction
  • Data driven criteria vs. MSE-based methods
  • The maximum likelihood criterion: Merits and drawbacks
  • Background on convex analysis
• Linear Gaussian model
  • Stein’s phenomenon
  • Dominance and admissibility
  • Least-squares dominating estimators
  • Minimax and blind minimax estimation
• Estimation in nonlinear problems
  • Uniform CRLB
  • Improving the CRLB and maximum-likelihood estimation

Background Requirements

The tutorial is intended for a broad audience with basic knowledge in statistical signal processing.

Speaker Biographies

Yonina C. Eldar received the B.Sc. degree in Physics in 1995 and the B.Sc. degree in Electrical Engineering in 1996, both from Tel-Aviv University (TAU), Tel-Aviv, Israel, and the Ph.D. degree in Electrical Engineering and Computer Science in 2001 from the Massachusetts Institute of Technology (MIT), Cambridge.

From January 2002 to July 2002 she was a Postdoctoral Fellow at the Digital Signal Processing Group at MIT. She is currently an Associate Professor in the Department of Electrical Engineering at the Technion - Israel Institute of Technology, Haifa, Israel. She is also a Research Affiliate with the Research Laboratory of Electronics at MIT.

From 1992 through 1996 she was in the program for outstanding students in TAU. In 1998 she held the Rosenblith Fellowship for study in Electrical Engineering at MIT, and in 2000 she held an IBM Research Fellowship. She was a Horev Fellow in the Leaders in Science and Technology program at the Technion, and an Alon Fellow. In 2004, she was awarded the Wolf Foundation Krill Prize for Excellence in Scientific Research, and in 2005 the Andre and Bella Meyer Lectureship. She is a member of the IEEE Signal Processing Theory and Methods technical committee, an Associate Editor for the IEEE Transactions on Signal Processing, and the EURASIP Journal on Signal Processing, and on the Editorial Board of Foundations and Trends in Signal Processing.