Material covered

• Balancing data fit and model complexity with Ridge regression
• Alternative way to write optimization as finding parameters within disk
• Behavior of model weights as disk radius is changed (demo)
• "Evidence-sharing" and geometric reasons for non-sparse solutions
• Lasso regression: replacing L2 model penalty with L1 model penalty
• Casting Lasso regression as a scilab/matlab quadratic program
• Behavior of model weights as L1 limit is changed (demo)
• Reasons for Lasso giving more sparse solutions
• Implicit feature selection effect
• Handling non-linear regression by lifting data points to higher dimension
• Classification with small discrete label sets
• Why classification loss is fundamentally different from regression