# ECE 7251: Signal Detection and Estimation

Georgia Institute of Technology

Spring 2002

**Instructor:** Prof. Aaron Lanterman

**Office:** GCATT 334B

**Phone:** 404-385-2548

**E-mail:** lanterma@ece.gatech.edu

**Course website**: users.ece.gatech.edu/~lanterma/ece7251

**When and where:** MWF, 12:05-12:55,

212 Engineering Science and Mechanics Building

# Syllabus

# News

- The first quiz has been postponed to Monday, Feb. 11.
- The second quiz has been postponed to Monday, March 18.

# Lectures, Suggested Readings, and Suggested Problems

- Lecture 2: Sufficient Statistics and Exponential Families (1/7/01)

(ppt)

(pdf) (Poor, pp. 158-160, 164; Hero, pp. 24-30; some lecture

examples taken from pp. 30-33 of Srinath)

**Suggested problems:**Hero, Sec. 3.5, #5, #6 (parts a and b only) - Lecture 3: Introduction to Bayesian Estimation (1/9/01)

(ppt)

(pdf) (Poor, pp. 142-147)

(Poor, pp. 142-147; Hero pp.

32-38) - Lecture 4: Examples of Bayesian Estimation (1/11/01)

(whiteboard)

(Poor, pp. 147-152,**pay particular attention to Example IV.B.2**;

Hero, pp. 38-42)

**Suggested problems:**Poor, Sec. IV.F: #1, #7 - Lecture 5: More Examples and Properties

of Bayesian Estimation

(1/14/01)

(whiteboard)

(Hero, pp. 42-46)

**Suggested problem:**Poor, Sec. IV. F: #25 (parts a and b only)

**Highly suggested problem:**Try to derive the CME,

CmE, and MAP estimators

on pp. 43-44 of Hero (good practice with erf functions; you may need to do

integration by parts) - Lecture 6: The Orthogonality Principle in MMSE Estimation (1/16/01)

(ppt)

(pdf)

(Poor, pp. 221-229; Hero, pp. 82-96)

**Suggested problems:**

Three interesting MMSE problems with solutions

(PDF), Scan of an old UIUC exam problem

(This one is interesting since it shows how the orthogonality principle

is useful for things beyond computing linear MMSE. Here, you compute a

quadratic MMSE!) - Lecture 7: Examples of Linear and Nonlinear MMSE Estimation (1/18/01)
- Lecture 8: Nonrandom Parameter Estimation (1/23/01)

(ppt)

(pdf)

(Poor, pp. 173-185; the discussion on p. 179 and continuing on to the

top of p. 180 is particularly enlightening; Hero, pp. 51-60, pp. 70-76) - Lecture 9: The Cramer-Rao Bound (1/25/01)

(ppt)

(pdf)

(Poor, pp. 167-173, pp. 185-186; Hero, pp. 60-70)

**Suggested problems:**Poor, Sec. IV. F: #15, #25 (now try parts c and d) - Lecture 10: Estimation Under Additive Gaussian Noise (a.k.a. Least

Squares Solutions) (1/28/01)

(ppt, revised 2/08)

(pdf, revised 2/08)

(Poor, pp. 155-157) - Lecture 11: Examples with Non-Gaussian Data, Part I
- Lecture 12: Examples with Non-Gaussian Data, Part II
- Lecture 13:”The” Expectation-Maximization Algorithm

(Basic Formulation and Simple Example) (2/4/02)

(ppt)

(pdf) - Lecture 14:”The” Expectation-Maximization Algorithm

(Theory) (2/6/02)

(ppt)

(pdf) - Lecture 15: EM Algorithm for Gaussian Mixture
- Lecture 16: The Kalman Filter

(ppt)

(pdf) - Lecture 17: Variations of the Kalman Filter

(ppt)

(pdf) - Lecture 18: Wiener Filtering

(ppt)

(pdf) - Lecture 19: Introduction to Detection Theory

(including Bayesian and MinMax Tests)

(ppt)

(pdf)

(read Poor, pp. 5-18, but try not to get too bogged down in the proofs, and

*skim*the parts on randomization; Hero, p. 140-160)**Suggested problems:**Poor, Sec. II.F, #2 (parts a and b), #4 (parts

a and b), #6 (parts a and b) - Lecture 20: Gaussian Detection Example (Equal Variances)

(Poor, Example II.C.1) - Lecture 21: Gaussian Detection Example (Equal Means)
- Lecture 22: Neyman-Pearson Tests and ROC Curves

(ppt)

(pdf)

(Poor, pp. 22-29; Hero, pp. 160-174)

)

**Suggested problems:**Poor, Sec. II.F, #2 (part c), #4 (part c),

Hero #7.3, #7.4 - Lecture 23: Chernoff Bounds (Theory)

(ppt, revised 3/18)

(pdf, revised 3/18)**Suggested problems:**See exercises on lecture slides - Lecture 24: Chernoff Bounds (Gaussian Examples)

(ppt, revised 3/18)

(pdf, revised 3/18)**Suggested problems:**See exercises on lecture slides - Lecture 25: The Role of Information Theory in Detection Theory

(ppt)

(pdf) - Lecture 26: Uniformly Most Powerful Tests

(ppt)

(pdf)

(Hero, pp. 177-192)**Suggested problems:**Hero, #7.2 - Lecture 28: Locally Most Powerful Tests

(ppt)

(pdf) (look over solution to Poor,

Sec. II.6, #15; Hero, pp. 196-208)**Suggested problems:**Hero, #8.1 - Lecture 29: Generalized Likelihood Ratio Tests and

Model Order Selection Criteria

(ppt)

(pdf)

(Hero, pp. 209-211;*skim*Hero, Chapter 9)**Suggested problems:**Hero, #8.2, #8.4, #8.5 - Lecture 30: Karhunen-Loeve Expansions, Part I
- Lecture 31: Karhunen-Loeve Expansions, Part II
- Lecutre 32: Karhunen-Loeve Expansions, Part III (Hero, pp. 325-327)
**Suggested problems:**

A Hard K-L Expansion Problem - Lecture 33: Detecting Deterministic Signals in Noise

(ppt)

(Poor, pp. 45-63;

read the solution to Poor, Sec. III.F, #1)**Suggested problems:**Poor, Sec. III.F, #6, #13 - Lecture 34: General Multivariate Gaussian Detection Problems

(ppt)

(Hero, pp. 249-261, pp. 269-279) - Lecture 35: Continous-Time Detection of Deterministic Signals

in White Gaussian Noise

(ppt) - Lecture 36: A Peek at Channel Capacity

(ppt) (Hero, pp. 339-343)**Suggested problems:**Poor, Sec. III.F, #3 - Lecture 37: Continous-Time Detection of Deterministic Signals

in Colored Gaussian Noise

(ppt)

(Hero, pp. 327-339;**pay particular attention to pp. 327-330**; notice

that what Van Trees calls Q, Hero calls r^{-1}) - Lecture 38: Incoherent Detection

(ppt)

(Poor, pp. 65-72; skim Hero, pp. 356-362)**Suggested problem:**Poor, Sec III.F, #15 - Lecture 39: Parameter Estimation with Continuous-Time Data, Part I

(Poor, pp. 327-331) - Lecture 40: Parameter Estimation with Continuous-Time Data, Part II

(Poor, pp. 331-333)# Data Files

# Exam Solutions

- In-class Quiz I Solutions

(pdf)

# Other Goodies

- Kalman’s

original paper on Kalman filtering,

recently re-typeset - Farshid Delgosha’s solutions to problems 2-5 starting on p. 29 of

Hero’s book (MS Word doc)

(pdf) and problems 1-7

(MS Word doc) (pdf)

and 25(ab)

(MS Word doc)

(pdf)

in Chapter 4 of Poor’s

book - Brief

Comments on the EM Algorithm by Prof.

Donald

L. Snyder. This contains

very nice description of the “classic” EM algorithm, and a review of

research on the EM algorithm at Washington University circa 1986. - Volkan gave a presentation on estimation theory in Jim McClellan’s

research group. He let me post his

slides and a related

short

paper

with related proofs.

Volkan did a great job; it’s

a nice distillation of some of the material in Ch. 4 of Poor. Enjoy!

- In-class Quiz I Solutions