1Basic course information
2007-07-27 LectureWhy you should take CS705
22007-07-31 LectureBayesian learning, smoothing and conjugate priors, from least-square to ridge regression
2007-08-03 LectureRidge and Lasso regression, non-linear regression, classification vs. regression
2007-08-04 LectureBayes-optimal and conditional probabilistic classification, discriminants for multivariate Gaussian distributions, loss functions for regression and classification
32007-08-07 LectureScilab Tutorial by Dhaval
2007-08-11 Homework 1

For coding and submission details please contact the TAs.

42007-08-14 LectureSingular value decomposition by Professor Ranade
2007-08-17 LectureSingular value decomposition, linear least square fitting, principal components analysis, eigenvalues and eigenvectors, the Perceptron algorithm
52007-08-21 LectureLinear least square fitting, pseudoinverse, iterative methods, Fisher's linear discriminant
2007-08-24 LectureFisher's linear discriminant, logistic regression, maximum entropy classification
2007-08-25 Homework 2

There may be minor edits and fixes but the broad questions will remain unchanged. I am posting a preliminary version so that students can start early and complete before midterm exams.

62007-08-28 LectureConnections between SVM and Logistic Regression, duality theory recap
2007-08-31 LectureDual SVM QP, the kernel trick
72007-09-04 LectureIntroduction to StructSVM (Prof. Sunita)
2007-09-07 Lecture

TsochantaridisJHA2005structsvm proofs (Prof. Sunita)

2007-09-09 Midterm examWith some solutions and grading policies.
92007-09-18 LectureMidterm review, StructSVM revisit, ramifications
2007-09-21 LectureLinear-time linear SVM, SVM for non-decomposable loss functions
102007-09-25 LectureOrdinal regression and ranking
2007-09-27 Lecture (8:30--9:30am Thu)Designing kernels for regression and classification, RKHS basics
2007-09-28 Lecture (11:30am--12:30pm Fri)More about kernels, infinite dimensional spaces, eigen functions, RKHS, representer theorem
112007-10-05 LectureRKHS and representer theorem, variations on SVM optimization
122007-10-09 LectureCCCP review; transductive learning
2007-10-12 LectureProbabilistic model learning, expectation maximization
132007-10-16 LectureMultinomial mixture, EM, optimization, variational approach
2007-10-19 LectureNMF, dyadic aspect model, cross association and coclustering
142007-10-23 LectureEnsemble learners, bagging and boosting
2007-10-26 LectureMore about bagging and boosting, rate distortion theory
Homework 3
152007-10-30 LectureInformation bottleneck, approximating (functions of) distributions by Monte-Carlo sampling
2007-11-03 Lecture (Sat, not Fri)

Metropolis-Hastings; Generalization theory

162007-11-06 Lecture

Generalization theory and risk bounds

172007-11-13 Lecture

Empirical processes, risk bounds, growth function, VC theory

2007-11-16 LectureRisk bounds, symmetrization, stability-based generalization bounds
2007-11-18 LectureCourse wrap-up
18Final exam9:30am--1pm, Friday 30th November, 2007