Overview of the course

• Introduction to the course.

Session on Scilab

• Session notes, scripts, datasets. Introduction to Scilab, Plotting, Matrix operations
• Some additional notes.

The Method of Collecting Data, Variables, Sampling, Integrity, Coverage etc, Descriptive and Summary Statistics

• Video including slides and audio.
• Slides.

Descriptive Statistics: Bar plot, value plot, scatter plot, histogram

• Video including slides and audio.
• Slides.
• Scilab code.
• Scilab code.

Descriptive Statistics: Introduction to summary statistics, mean, median, mode, percentile and intro to measures of spread, and measures of shape

• Video including slides and audio.
• Slides.
• Practice Problem Set 1.
• Scilab code.
• Scilab code.
• Scilab code.
• Scilab code.
• Scilab code.

Measure of spread, reading measures of location from histograms, Effect of noise/outliers

• Video including slides and audio.
• Slides.
• Scilab code.
• Scilab code.

Percentile, Chebyshev's inequalities and their proofs

• Video including slides and audio.
• Slides.
• Scilab code.
• Scilab code.
• Scilab code.
• Scilab code.
• Scilab code.

Chebyshev's inequalities and their proofs concluded, correlation coefficient and introduction to linear regression

• Video including slides and audio.
• Slides.
• Scilab code.
• Question Bank 1.

Properties of correlation coefficient, Linear regression

• Slides.
• Some additional notes.
• Scilab code.

Derivation of coefficients of Linear regression

• Slides.
• Some additional notes.
• Scilab code.
• Practice Problem Set 2.
• Question Bank 2. Attempt questions 1,2,3,4,5,6,7,36,37,38,43,44,45,54,55.

Polynomial and Multiple variable Regression

• Video, including slides and audio.
• Slides.
• Some additional notes.
• Scilab code.

Overfitting, Ridge Regression, epsilon insensitive loss

• Video, including slides and audio.
• Slides.
• Some additional notes.
• Some additional notes.
• Scilab code.

Epsilon insensitive loss and Support Vector Regression, Introduction to Probability

• Video, including slides and audio.
• Slides.

Probability as a modular function (and a basic idea of submodular and supermodular functions), Chain rule, Independence of events, Bayes rule. Chapter 3 of Sheldon Ross

• Video, including slides and audio.
• Slides.
• Some additional notes.
• Scilab code.

Bayes Rule, Random variables (discrete and continuous), Expectation, Standard deviation, Empirical and actual risk for standard deviations as examples of sample and population means. Concluding chapter 3 and beginning chapter 4 of Sheldon Ross.

• Slides.
• Some additional notes.
• Question Bank 3. Attempt all questions. In particular, following questions MUST be attempted: Question Nos. 4, 5, 6, 10, 11, 17, 18, 19, 21, 23, 25, 26, 27, 30, 34, 35, 36, 38, 42, 43 .
• Scilab code.

Further properties between mean, variance, covariance for functions of (independent) random variables, introduction to Markov's and Chebyshev's inequalities. Sections 4.4, 4.5, 4.6, 4.7 of Sheldon Ross

• Slides.
• Some additional notes.
• Some additional notes.
• Scilab code.

Continuous Random Variables, Probablity Density Functions for single variables, joint distributions and conditional distributions, Expected value, Variance and Covariance and their properties for sums of random variables. See chapter 4 of Sheldon Ross

• Video, including slides and audio.
• Slides.
• Some additional notes.
• Practice Problem Set 3.
• Scilab code.

Markov's inequality, Chebyshev's inequality, Weak law of large numbers (Section 4.9 of Sheldon Ross), some problem solving to illustrate expectation and variance of sums of random variables.

• Slides.
• Some additional notes.
• Scilab code.

Cumulative distribution function for continuous random variables and connection with probability density function (See Chapter 4 of Sheldon Ross)

• Video, including slides and audio.
• Slides.
• Some additional notes.
• Question Bank 4. Following questions MUST be attempted: Question Nos. 1, 4, 5, 6, 7, 9, 10, 11, 12, 14, 16, 17, 20, 23, 24, 25, 28, 2, 3, 8, 13, 15, 18, 19, 21, 22, 26, 27, 29 until 57..
• Scilab code.

Equivalent defintions of independence, moment generating function of a random variable and functions of random variables (See Chapter 4, section 8 of Sheldon Ross)

• Video, including slides and audio.
• Slides.
• Slides.
• Some additional notes.
• Scilab code.

Moment generating function of weighted sum of independent random variables (Chapter 4 of Sheldon Ross), illustration with Gaussian random variable, two equivalent ways of deriving expected value of function of multiple random variables (Homework: prove the equivalence and conditions under which the equivalence holds), Bernoulli, Multivariate bernoulli and binomial random variables

• Video, including slides and audio.
• Slides.
• Slides (pdf).
• Some additional notes.
• Some additional notes.
• Link 1 to help solve homework problem<.
• Link 2 to help solve homework problem.

Proof of the simple expression of expected value of a function of multiple random variables, motivation for the poisson random variable from the bernoulli and binomial random variables

• Video, including slides and audio.
• Slides.
• Slides (pdf).

Derivation of the poisson distribution from the binomial distribution: the poisson theorem, mgf, mean, variance of poisson, distribution of sum of independent poissons

• Video, including slides and audio.
• Slides.
• Slides (pdf).

Stochastic proces, Bernoulli & binomial process, geometric and hypergeometric distributions. Homework: how would you extend the bernoulli process to a poisson process, the way we extended the corresponding distribution?

• Video, including slides and audio.
• Slides.
• Slides (pdf).

Concluding bernoulli process and motivating the Geometric and Pascal distributions of (inter)arrival times, Hypergeometric distributions, general definition of stochastic process, classification of stochastic processes based on discrete/continuous state space and discrete/continuous time, the Chinese Restraunt Process, the Poisson Process and motivating the exponential and Erlang distributions

• Video, including slides and audio.
• Slides.
• Slides (pdf).
• Slides.
• Slides (pdf).
• Practice Problem Set 4.
• Question Bank 4. Following questions MUST be attempted if not already: Question Nos. 2, 3, 8, 13, 15, 18, 19, 21, 22, 26, 27, 29 until 57
• Question Bank 5. Following questions MUST be attempted: Question Nos. 1-48

Derivation of exponential distribution as distribution of interarrival times in the limiting case of a bernoulli process, properties of exponential distribution, derivation of the erlang distribution, the gamma distribution, Merged multiple poisson process streams

• Video, including slides and audio.
• Slides.
• Slides (pdf).

More practice problems for poisson processes and exponential distributions and merged poisson process streams

• Video, including slides and audio.
• Slides.
• Slides (pdf).

Discussion of Uniform distribution and its properties, sampling from other distributions based on sampling from uniform distribution using transformation functions (refer to Question 3 of Quiz 2 for illustration for exponential distribution), derivation of pdf of normal (Gaussian) distribution form the Binomial distribution in the limiting case as n tends to infinity and np tends to a constant, properties of Gaussian distribution

• Slides.
• Slides (pdf).

Moment Generating Function for the Normal Distribution via the Standard Normal Distribution, Motivation for Standard Normal Distribution - computing probabilities for normal distributions via looking up the table for standard normal distribution. Example Standard Normal Table can be found here

• Slides.
• Slides (pdf).

Random sample: definition and sampling strategies, Random statistic as random variable, sample mean as a random statistic and discussion of a case study to obtain the distribution of the sample mean. Note that the problem for illustrating concepts that we discussed in class is just problem number 10 of problem set 4

• Slides.
• Slides (pdf).

Maximum entropy probability distributions: the Normal and Uniform distributions, Motivating example for central limit theorem (CLT), formal statement and proof of CLT, Motivation for and description of Chi-squared and t-distributions

• Slides.
• Slides (pdf).
• Question Bank 6. Following questions MUST be attempted: Question Nos. 1-17

Estimators, Point estimator, unbiased estimators, Minimum variance unbiased estimators, illustration for the Gaussian distribution. For lectures spanning 27-10-2014 to 3-11-2014, refer to Chapters 6 and 7 of Sheldon Ross. You can exclude sections 6.5.2, 7.4, 7.6 and 7.8 of Sheldon Ross. Also you can refer to Sections 6.1 and 6.2 of Hogg and Craig.

• Slides.
• Slides (pdf).

Gaussian Distribution of sample mean when data sample is from Gaussian distribution with unknown mean, t-distribution for sample mean normalised using unbiased sample variance and chi-squared distribution for (rescaled) unbiased sample variance when random sample is from Gaussian distribution with unknown mean and unknown variance. For lectures spanning 27-10-2014 to 3-11-2014, refer to Chapters 6 and 7 of Sheldon Ross. You can exclude sections 6.5.2, 7.4, 7.6 and 7.8 of Sheldon Ross. Also you can refer to Sections 6.1 and 6.2 of Hogg and Craig.

• Slides.
• Slides (pdf).

MVUE, Cramer-Rao Lower bound, definition of likelihood function, Moment estimators, Least squares estimator, Bias-variance decomposition, Maximum likelihood estimator. For lectures spanning 27-10-2014 to 3-11-2014, refer to Chapters 6 and 7 of Sheldon Ross. You can exclude sections 6.5.2, 7.4, 7.6 and 7.8 of Sheldon Ross. Also you can refer to Sections 6.1 and 6.2 of Hogg and Craig.

• Slides.
• Slides (pdf).

Maximum likelihood estimation with examples: bernoulli, exponential, Gaussian. Confidence interval estimation. For lectures spanning 27-10-2014 to 3-11-2014, refer to Chapters 6 and 7 of Sheldon Ross. You can exclude sections 6.5.2, 7.4, 7,6 and 7.8 of Sheldon Ross. Also you can refer to Sections 6.1 and 6.2 of Hogg and Craig.

• Slides.
• Slides (pdf).
• Some additional notes (1).
• Some additional notes (2).
• Some additional notes (3).
• Some additional notes (4).
• Practice Problem Set 5 and the solutions.

Confidence interval estimation for mean of Gaussian with known/unknown variance, variance of Gaussian, mean of Bernoulli, Motivation for hypothesis testing. For lectures spanning 27-10-2014 to 3-11-2014, refer to Chapters 6 and 7 of Sheldon Ross. You can exclude sections 6.5.2, 7.4, 7.6 and 7.8 of Sheldon Ross. Also you can refer to Sections 6.1 and 6.2 of Hogg and Craig.

• Slides.
• Slides (pdf).
• Practice Problem Set 5 and the solutions.

One sided test for sample proportion, One sided and two sided Hypothesis testing for mean of sample. For lectures 4-11-2014 until 6-11-2014, including tutorial problems, refer to following sections of Sheldon Ross: 8.1, 8.2, 8.3, 8.6, 8.7. Following sections are not covered: 8.4, 8.5.1, 8.6.1, 8.7.1

• Slides.
• Slides (pdf).
• Practice Problem Set 5 and the solutions.

Continued: One sided and two sided Hypothesis testing for mean of sample and sample size formula, more examples. For lectures 4-11-2014 until 6-11-2014, including tutorial problems, refer to following sections of Sheldon Ross: 8.1, 8.2, 8.3, 8.6, 8.7. Following sections are not covered: 8.4, 8.5.1, 8.6.1, 8.7.1<

• Slides.
• Slides (pdf).
• Practice Problem Set 5 and the solutions.

Goodness of fit test (in syllabus) and Bayesian Inference (not in syllabus). For goodness of fit test, basic understanding of Section 11.3 of Sheldon Ross is good enough.

• Slides.
• Slides (pdf).

• Question Bank 6 Following questions MUST be attempted: Questions 1 to 17
• Question Bank 7 Following questions MUST be attempted: Questions 1 to 23, 48 to 59, 61 to 65
• Question Bank 8 Following questions MUST be attempted: Questions 1 to 7, 11, 12, 14, 17, 20, 52, 53, 54, 56-62, 68, 69, 70
• Extra Tutorial