ECE 7251: Signal Detection and Estimation
Georgia Institute of Technology
Spring 2002
Instructor: Prof. Aaron Lanterman
Office: GCATT 334B
Phone: 404-385-2548
E-mail: lanterma@ece.gatech.edu
Course website: users.ece.gatech.edu/~lanterma/ece7251
When and where: MWF, 12:05-12:55,
212 Engineering Science and Mechanics Building
Syllabus
News
- The first quiz has been postponed to Monday, Feb. 11.
- The second quiz has been postponed to Monday, March 18.
Lectures, Suggested Readings, and Suggested Problems
- Lecture 1: Introduction (1/4/01)
(ppt)
(pdf) - Lecture 2: Sufficient Statistics and Exponential Families (1/7/01)
(ppt)
(pdf) (Poor, pp. 158-160, 164; Hero, pp. 24-30; some lecture
examples taken from pp. 30-33 of Srinath)
Suggested problems: Hero, Sec. 3.5, #5, #6 (parts a and b only) - Lecture 3: Introduction to Bayesian Estimation (1/9/01)
(ppt)
(pdf) (Poor, pp. 142-147)
(Poor, pp. 142-147; Hero pp.
32-38) - Lecture 4: Examples of Bayesian Estimation (1/11/01)
(whiteboard)
(Poor, pp. 147-152, pay particular attention to Example IV.B.2;
Hero, pp. 38-42)
Suggested problems: Poor, Sec. IV.F: #1, #7 - Lecture 5: More Examples and Properties
of Bayesian Estimation
(1/14/01)
(whiteboard)
(Hero, pp. 42-46)
Suggested problem: Poor, Sec. IV. F: #25 (parts a and b only)
Highly suggested problem: Try to derive the CME,
CmE, and MAP estimators
on pp. 43-44 of Hero (good practice with erf functions; you may need to do
integration by parts) - Lecture 6: The Orthogonality Principle in MMSE Estimation (1/16/01)
(ppt)
(pdf)
(Poor, pp. 221-229; Hero, pp. 82-96)
Suggested problems:
Three interesting MMSE problems with solutions
(PDF), Scan of an old UIUC exam problem
(This one is interesting since it shows how the orthogonality principle
is useful for things beyond computing linear MMSE. Here, you compute a
quadratic MMSE!) - Lecture 7: Examples of Linear and Nonlinear MMSE Estimation (1/18/01)
- Lecture 8: Nonrandom Parameter Estimation (1/23/01)
(ppt)
(pdf)
(Poor, pp. 173-185; the discussion on p. 179 and continuing on to the
top of p. 180 is particularly enlightening; Hero, pp. 51-60, pp. 70-76) - Lecture 9: The Cramer-Rao Bound (1/25/01)
(ppt)
(pdf)
(Poor, pp. 167-173, pp. 185-186; Hero, pp. 60-70)
Suggested problems: Poor, Sec. IV. F: #15, #25 (now try parts c and d) - Lecture 10: Estimation Under Additive Gaussian Noise (a.k.a. Least
Squares Solutions) (1/28/01)
(ppt, revised 2/08)
(pdf, revised 2/08)
(Poor, pp. 155-157) - Lecture 11: Examples with Non-Gaussian Data, Part I
- Lecture 12: Examples with Non-Gaussian Data, Part II
- Lecture 13:”The” Expectation-Maximization Algorithm
(Basic Formulation and Simple Example) (2/4/02)
(ppt)
(pdf) - Lecture 14:”The” Expectation-Maximization Algorithm
(Theory) (2/6/02)
(ppt)
(pdf) - Lecture 15: EM Algorithm for Gaussian Mixture
- Lecture 16: The Kalman Filter
(ppt)
(pdf) - Lecture 17: Variations of the Kalman Filter
(ppt)
(pdf) - Lecture 18: Wiener Filtering
(ppt)
(pdf) - Lecture 19: Introduction to Detection Theory
(including Bayesian and MinMax Tests)
(ppt)
(pdf)
(read Poor, pp. 5-18, but try not to get too bogged down in the proofs, and
skim the parts on randomization; Hero, p. 140-160)
Suggested problems: Poor, Sec. II.F, #2 (parts a and b), #4 (parts
a and b), #6 (parts a and b) - Lecture 20: Gaussian Detection Example (Equal Variances)
(Poor, Example II.C.1) - Lecture 21: Gaussian Detection Example (Equal Means)
- Lecture 22: Neyman-Pearson Tests and ROC Curves
(ppt)
(pdf)
(Poor, pp. 22-29; Hero, pp. 160-174)
)
Suggested problems: Poor, Sec. II.F, #2 (part c), #4 (part c),
Hero #7.3, #7.4 - Lecture 23: Chernoff Bounds (Theory)
(ppt, revised 3/18)
(pdf, revised 3/18)
Suggested problems: See exercises on lecture slides - Lecture 24: Chernoff Bounds (Gaussian Examples)
(ppt, revised 3/18)
(pdf, revised 3/18)
Suggested problems: See exercises on lecture slides - Lecture 25: The Role of Information Theory in Detection Theory
(ppt)
(pdf) - Lecture 26: Uniformly Most Powerful Tests
(ppt)
(pdf)
(Hero, pp. 177-192)
Suggested problems: Hero, #7.2 - Lecture 28: Locally Most Powerful Tests
(ppt)
(pdf) (look over solution to Poor,
Sec. II.6, #15; Hero, pp. 196-208)
Suggested problems: Hero, #8.1 - Lecture 29: Generalized Likelihood Ratio Tests and
Model Order Selection Criteria
(ppt)
(pdf)
(Hero, pp. 209-211; skim Hero, Chapter 9)
Suggested problems: Hero, #8.2, #8.4, #8.5 - Lecture 30: Karhunen-Loeve Expansions, Part I
- Lecture 31: Karhunen-Loeve Expansions, Part II
- Lecutre 32: Karhunen-Loeve Expansions, Part III (Hero, pp. 325-327)
Suggested problems:
A Hard K-L Expansion Problem - Lecture 33: Detecting Deterministic Signals in Noise
(ppt)
(Poor, pp. 45-63;
read the solution to Poor, Sec. III.F, #1)
Suggested problems: Poor, Sec. III.F, #6, #13 - Lecture 34: General Multivariate Gaussian Detection Problems
(ppt)
(Hero, pp. 249-261, pp. 269-279) - Lecture 35: Continous-Time Detection of Deterministic Signals
in White Gaussian Noise
(ppt) - Lecture 36: A Peek at Channel Capacity
(ppt) (Hero, pp. 339-343)
Suggested problems: Poor, Sec. III.F, #3 - Lecture 37: Continous-Time Detection of Deterministic Signals
in Colored Gaussian Noise
(ppt)
(Hero, pp. 327-339; pay particular attention to pp. 327-330; notice
that what Van Trees calls Q, Hero calls r^{-1}) - Lecture 38: Incoherent Detection
(ppt)
(Poor, pp. 65-72; skim Hero, pp. 356-362)
Suggested problem: Poor, Sec III.F, #15 - Lecture 39: Parameter Estimation with Continuous-Time Data, Part I
(Poor, pp. 327-331) - Lecture 40: Parameter Estimation with Continuous-Time Data, Part II
(Poor, pp. 331-333)
Data Files
Exam Solutions
-
In-class Quiz I Solutions
(pdf)
Other Goodies
- Kalman’s
original paper on Kalman filtering,
recently re-typeset - Farshid Delgosha’s solutions to problems 2-5 starting on p. 29 of
Hero’s book (MS Word doc)
(pdf) and problems 1-7
(MS Word doc) (pdf)
and 25(ab)
(MS Word doc)
(pdf)
in Chapter 4 of Poor’s
book - Brief
Comments on the EM Algorithm by Prof.
Donald
L. Snyder. This contains
very nice description of the “classic” EM algorithm, and a review of
research on the EM algorithm at Washington University circa 1986. - Volkan gave a presentation on estimation theory in Jim McClellan’s
research group. He let me post his
slides and a related
short
paper
with related proofs.
Volkan did a great job; it’s
a nice distillation of some of the material in Ch. 4 of Poor. Enjoy!