# ECE 7251: Signal Detection and Estimation

Georgia Institute of Technology

Spring 2002

**Instructor:** Prof. Aaron Lanterman

**Office:** GCATT 334B

**Phone:** 404-385-2548

**E-mail:** lanterma@ece.gatech.edu

**Course website**: users.ece.gatech.uiuc.edu/~lanterma/ece7251

**When and where:** MWF, 12:05-12:55,

212 Engineering Science and Mechanics Building

**Prerequisites:** Knowledge of probability and random processes

at the level of the text by Papoulis, Stark and Woods,

or Leon-Garcia, i.e. ECE6601 or equivalent.

Some questions on the take-home

portion of the exams will require some basic programming skills;

although you are welcome to use whatever programming language you like,

competency in MATLAB will be extremely helpful.

**Note that ECE6250 is NOT a prerequisite for this offering of ECE7251!**

# Textbooks

**Required texts:**

- A.O. Hero, “Statistical Methods for Signal Processing,” 1998-2002.

This is a set of course notes written by Prof. Al Hero (ECE, Univ. of

Michigan-Ann Arbor), which he plans to eventually turn into a textbook.

A link to the

PDF file will be provided, and copies will be made available for purchase

in the campus bookstore. - H.V. Poor, “An Introduction to Signal Detection and Estimation,”

2nd Edition, Springer, 1994.

Prof. Hero’s notes will provide the main structure

of the course and the inspiration for my lectures. However, the notes are

a bit lacking on textual descriptions at the moment, so I thought it best

that everyone have a copy of Vince Poor’s book too, in order to have a

solid reference readily available. Prof. Hero welcomes comments on his notes;

send your comments to me, and I’ll compile them together into a coherent

report.

**Texts on reserve in the library:**

- H.L. Van Trees, “Detection, Estimation, and Modulation Theory,”

H.L. Van Trees, Vol. I, 1968, recently reprinted in paperback (Sept. 2001)

by John

Wiley & Sons. (Amazon.com sells this for $66.) - L.L. Scharf, “Statistical signal processing :

detection, estimation, and time

series analysis,” Addison-Wesley, 1991. - S.M. Kay, “Fundamentals of statistical

signal processing. Volume I:

estimation theory,” Prentice-Hall, 1998 - S.M. Kay, “Fundamentals of statistical

signal processing. Volume II:

detection theory,” Prentice-Hall, 1998 - M.D. Srinath, P.K.Rajasekaran, R. Viswanathan, “Introduction to

Statistical Signal Processing with Applications,” Prentice-Hall, 1996.

# Grades, Exams, and Other Necessities of Life

**Grade breakdown:**

Exam 1 (Feb. 6): 20%

Exam 2 (March 13): 20%

Exam 3 (April 3): 20%

Final Exam: 40%

(If you have a conflict with these any of these times, please let me know

ASAP.)

Each exam will have an in-class part and a take-home part.

The in-class part will be open book and open notes, although this is primary

so

you won’t panic; the in-class section will emphasize your “intuitive”

understanding of the material. If you find yourself spending

most of your time on the in-class portion frantically flipping through your

notes trying to find the answer, you will run out of time.

The take-home portion will consist of more

in-depth problems which couldn’t possibly completed in an in-class

exam, although they are not intended to be exceptionally time consuming.

You will be given a generous amount of time to complete it.

The take-home portion may involve some highly instructive computational

experiments. You will be allowed to consult with any resources in the

library, posted on web sites, etc., as long as these are properly

cited in your solution. You will not, however, be

allowed to discuss the take-home

portion in any way with anyone besides myself.

I’m giving three exams (which is more than usual for a graduate-level

class) since it

helps reduce the *bad day effect.* The bad day effect hits

you when you happen to be having a bad day (you know, one of those days

where nothing is going right and you’ve run out of coffee

and you can’t seem to get your brain working)

on the one day you’re taking

the single midterm exam. When there are several midterm exams, you’re not

likely to have a bad day on all the exam days, so if you do have a

bad day, it can get averaged out.

**A note on homework, or the lack thereof:**

Notice that no homework will be collected and graded. In lecture, various

problems will be suggested for you to try at your leisure; working through

as many problems as you can will be the best way to prepare for the exams,

particularly the in-class portion.

You are strongly encouraged to work together in groups, preferably with

a lot of coffee. It’s also best

to tackle a problem or two per day, perhaps soon after the lecture while

the material is still fresh in your head,

rather than sit down and try a whole

bunch at once in a marathon session right before the exam.

**Office hours:**

If you have an office in GCATT, just go ahead and drop by; I will be there

most afternoons and evenings, although never in the morning. If you are

stationed outside of GCATT (or are otherwise having trouble getting a hold

of me in person),

send me an e-mail, and I’ll set up a time to meet with you in Van Leer

(or wherever is most convenient for you.)

I will

generally go to lunch after class; people are welcome to join me for lunch

if they have questions or generally want to chat. I will have more formal

office hours before the exams.

**A note on e-mail:**

As anyone who’s taken a course from me before will attest, I tend to send

out a lot of course-related e-mail. Make sure your e-mail account is working

(and not over quota or something like that) so you don’t miss anything good.

# Tentative schedule

- General Structure
- Sufficient Statistics; Exponential Families

- Parameter Estimation
- Bayesian Estimation (MAP and MMSE)
- Orthogonality Principle of MMSE
- Linear Minimum Mean Squared Error Estimation
- Maximum-Likelihood
- Method of Moments
- Cramer-Rao Bounds (both random and nonrandom)

- Computational Techniques
- EM Algorithm: Theory
- EM Algorithm: Examples
- Markov Chain Monte Carlo algorithms

- Filtering for Discrete-Time Processes
- Kalman Filter
- Wiener Filter

- Simple Hypothesis Testing
- Bayesian Detection
- Minmax Detection
- Neyman-Pearson Lemma; ROC Curves
- Chernoff Bounds

- Composite Hypothesis Testing
- Uniformly Most Powerful (UMP) Tests
- Locally Most Powerful (LMP) Tests
- Generalized Likelihood Ratio Tests (GLRT)
- Detector Structures for Discrete-Time Data with Gaussian, Laplacian,

and Cauchy Noise

- Model Order Estimation
- Schwarz’s Bayesian Information Criterion
- Minimum Description Length Criterion
- Stochastic Complexity

- Continuous-Time Extensions
- Karhunen-Loeve Expansions
- Grenander’s Theorem
- Detection with Continuous Data
- Parameter Estimation with Continuous Data