Topics to be covered (tentative list)

• Descriptive statistics
• Discrete and continuous probability
• Random variables and expectation
• Special random variables: Gaussian, Bernoulli, Beta, Gamma, Uniform, Poisson, Exponential, Binomial, etc.
• Hypothesis testing
• Parameter estimation
• Regression
• Probability density estimation

Intended Audience

Learning Materials and Textbooks

• Lecture slides that will be regularly posted. I may occasionally post links to applets or videos, or additional material such as problem sets.
• We will use moodle for posting assignments and grades
• Course textbook: Introduction to Probability and Statistics for Engineers and Scientists: Fourth Edition

Computational Resources

Matlab
• Download MATLAB from here.
• Matlab tutorial 1
• Matlab tutorial 2
• Matlab tutorial 3
• The MathWorks - MATLAB Tutorial
• Matlab Primer
• On-line Matlab Help
• Writing Fast Matlab Code (pdf)
• One more tutorial for writing fast matlab code
• Code Vectorization Guide
• Matlab Programmin Style Guidelines (pdf)
• Matlab array manipulation

Grading Policy (tenative)

• Mid-sem exam: 25%
• Final exam (cumulative): 25%
• Programming and written assignments (about five): 35% - all to be done in groups of 2 students.
• Two pre-announced quizzes: 15% total

Other Policies

• Attendance is mandatory. Students with less than 80% attendance may be given a DX grade.
• Assignments will be given out (typically) once every two or three weeks. They must be submitted on or before the deadline. No late assignments will be accepted. The programming components of the assignments will typically involve MATLAB, so you must be willing to learn it quickly.
• We will adopt a zero-tolerance policy against any forms of plagiarism or any other form of cheating. Just don't do it! In cases of plagiarism, givers and takers will both be considered equally responsible.
• This course is (inherently) cumulative. The syllabus for the final exam will include everything taught during the semester.

Tutorials

• See moodle

Quizzes

• See moodle

Lecture Schedule:

Date	Content of the Lecture	Assignments/Readings/Notes
30/07 (Tue)	Introduction, course overview and course policies Descriptive Statistics Descriptive statistics: key terminology Methods to represent data: frequency tables, bar/line graphs, frequency polygon, pie-chart Concept of frequency and relative frequency Cumulative frequency plots Interesting examples of histograms of intensity values in an image Data summarization: mean and median "Proof" that median minimizes the sum of absolute deviations - using calculus	Slides: Course Overview Slides: Descriptive statistics Readings: section 2.1, 2.2 from the textbook by Sheldon Ross
02/08 (Fri)	Proofs that median minimizes the sum of absolute deviations without using calculus Concept of quantile/percentile Standard deviation and variance, some applications Two-sided Chebyshev inequality with proof; One-side Chebyshev inequality (Chebyshev-Cantelli inequality) Concept of correlation coefficient and formula for it; proof that its value lies from -1 to +1	Slides: Descriptive statistics Readings: section 2.1, 2.2, 2.3, 2.4, 2.6 from the textbook by Sheldon Ross Proof that Median minimizes the sum of absolute deviations from the mean
03/08 (Sat)	Correlation coefficient: properties; uncentered correlation coefficient; limitations of correlation coefficient and Anscombe's quartet Correlation and causation Proof of one-sided Chebyshev's inequality MATLAB Demo Code	Slides: Descriptive statistics Readings: section 2.1, 2.2, 2.3, 2.4, 2.6 from the textbook by Sheldon Ross The correlation versus causation debtate: Link 1, Link 2, Link 3.
06/08 (Tue)	Discrete Probability Discrete probability: sample space, event, composition of events: union, intersection, complement, exclusive or, De Morgan's laws Boole's and Bonferroni's inequalities Conditional probability, Bayes rule, False Positive Paradox Birthday paradox Independent and mutually exclusive events	Slides: Discrete Probability Readings: Chapter 3 from the textbook by Sheldon Ross
09/08 (Fri)	Random variables Random variable: concept, discrete and continuous random variables Probability mass function (pmf), cumulative distribution function (cdf) and probability density function (pdf) Expected value for discrete and continuous random variables Law of the Unconscious Statistician (LOTUS): Expected value of a function of a random variable The mean and the median as minimizers of squared and absolute losses respectively (with proof for the former) Variance and standard deviation, with alternate expressions	Slides: Random variables Readings: Chapter 4 from the textbook by Sheldon Ross
13/08 (Tue)	Concept of joint PMF, PDF, CDF Concept of covariance, concept of mutual independence and pairwise independence Properties of covariance Covariance: properties, correlation versus independence Concept of moment generating function, two different proofs of uniqueness of moment generating function for discrete random variables, properties of moment generating functions PDF/PMF of sum of random variables Proof that median minimizes total absolute loss	Slides: Random variables Readings: Chapter 4 from the textbook by Sheldon Ross
16/08 (Fri)	Conditional CDF, PMF, PDF; verification of definition Families of random variables Concept of families of random variables Bernoulli PMF: mean, median, mode, variance, MGF Binomial PMF: relation to Bernoulli PMF, mean, median, mode, variance, plots, MGF, difference between binomial and geometric distribution Gaussian (normal) PDF: motivation from the central limit theorem Illustration of central limit theorem, statement of central limit theorem	Slides: Random variables Slides: Families of Random variables Readings: Section 4.3.2 from the textbook by Sheldon Ross Readings: Section 5.1,5.2,5.5,6.1,6.2 from the textbook by Sheldon Ross
17/08 (Sat)	Derivation of mean, variance, MGF, median, mode CDF of a Gaussian and its relations to error functions; probability of a Gaussian random variable to have values between mu +/- k sigma. Statement of central limit theorem and its extensions; proof of CLT using MGF application of CLT and its relation to the binomial distribution - de Moivre-Laplace theorem (without proof); one application of the CLT; relation between CLT and the law of large numbers	Slides: Families of Random variables Readings: Section 5.1,5.2,5.5,6.1,6.2 from the textbook by Sheldon Ross
20/08 (Tue)	Gaussian tail bounds Distribution of the sample mean and the sample variance, Bessel's correction; Chi-squared distribution - definition, genesis, MGF, properties; use of a chi-square distribution toward defining the PDF of the sample variance Uniform distribution: mean, variance, median, MGF; applications in sampling from a pre-specified PMF; application in generating a random permutation of a given set	Slides: Families of Random variables Readings: Section 5.1,5.2,5.5,6.1,6.2,6.3,6.4 from the textbook by Sheldon Ross
23/08 (Fri)	Poisson distribution: mean, variance, MGF, mode, addition of Poisson random variables, examples; derivation of Poisson from binomial Relation between Poisson and Gaussian distributions, examples Multinomial PMF - generalization of the binomial, mean vector and covariance matrix for a multinomial random variable, MGF for multinomial	Slides: Families of Random variables Readings: Section 5.2,5.6 from the textbook by Sheldon Ross
24/08 (Sat)	Exponential distribution: mean, median, MGF, variance, property of memorylessness, minimum of exponential random variables Parameter Estimation Parameter Estimation Concept of parameter estimation (or parametric PDF/PMF estimation) Maximum likelihood estimation (MLE) MLE for parameters of Bernoulli, Poisson, Gaussian and uniform distributions Least squares line fitting as an MLE problem	Slides: Families of Random variables Slides: Parameter Estimation Derivations for MLE Readings: Section 5.6 from the textbook by Sheldon Ross Readings: Sections 7.1, 7.2, 7.5, 7.7, 9.2 (for least squares line fitting) of the textbook by Sheldon Ross
27/08 (Tue)	Least squares line fitting as an MLE problem Concept of estimator bias, mean squared error, variance Estimators for interval of uniform distribution: example of bias Concept of two-sided confidence interval and one-sided confidence interval Confidence interval for mean of a Gaussian with known standard deviation Confidence interval for variance of a Gaussian	Slides: Parameter Estimation Derivations for MLE Readings: Sections 7.1, 7.2, 7.5, 7.7, 9.2 (for least squares line fitting) of the textbook by Sheldon Ross
30/08 (Fri)	Concept of nonparametric density estimation Concept of histogram as a probability density estimator Bias, variance and MSE for a histogram estimator for a smooth density (with bounded first derivatives) which is non-zero on a finite-sized interval; derivation of optimal number of bins (equivalently, optimal binwidth) and optimal MSE O(n^{-2/3})	Slides: Parameter Estimation Derivations for MLE Derivation of MSE, bias, variance for a histogram (Read section 6.1 only. These notes are by Prof. Yen-Chi Chen from the Univ. of Washington, Seattle. A local copy of the pdf is here Readings: Sections 7.1, 7.2, 7.5, 7.7, 9.2 (for least squares line fitting) of the textbook by Sheldon Ross
13/09 (Fri)	Hypergeometric distribution: genesis, mean, variance Applications of the hypergeometric distribution in counting of animals via the capture-recapture method Concept of kernel density estimator Bias, variance and MSE for a kernel density estimator for a smooth density (with bounded second derivatives); derivation of optimal bandwidth and optimal MSE O(n^{-4/5})	Slides: Parameter Estimation Derivations for MLE Derivation of MSE, bias, variance for a kernel density estimator (Read sections 6.2, 6.3, 6.4. These notes are by Prof. Yen-Chi Chen from the Univ. of Washington, Seattle. A local copy of the pdf is here Readings: Sections 7.1, 7.2, 7.5, 7.7, 9.2 (for least squares line fitting) of the textbook by Sheldon Ross