Summary

Overview of the course

Representing distributions as graphical models

  • Representation: directed graphical models (Chapter 3 of KF)
  • Directed graphical models: overview, representation of probability distribution and conditional independence statements.
  • Representation: Undirected graphical models (Chapter 4 of KF)
  • Representation: potentials, conditional independence and graph separability, factorization.
  • Constructing undirected models from distributions
  • Relationship between directed and undirected models.
  • Common undirected graphical models: Factor models, Ising and Potts model, Gibbs distribution, log-linear models, CRFs.
  • Feature-based potentials for flexible deployment in many applications.
  • Application in vision and text mining.

Inference in graphical models

  • Overview (Chapter 9.1 of KF)
  • Variable elimination (Chapter 9.2, 9.3 of KF)
  • Junction trees and sum product message passing (Chapters 10.1, 10.2, 10.4 of KF)

Learning graphical model parameters (probabilistic methods) (Lecture slides)

  • Learning conditional graphical models (CRFs), conditional likelihood training. (Chapter 20.3.1--20.3.2)
  • Learning with partially observed data (Chapter 19.2.2 to 19.2.2.5(inclusive) in KF)

Sampling

  • Foreward sampling (chapter 12.1)
  • Importance sampling (12.2 upto 12.2.3.1)
  • MCMC sampling (chapter 12.3): Gibbs and Langevin

Deep Latent Representation Models for High dimensional Objects

  • Variational Auto Encoders (VAEs) paper, Slides 1 above
  • GANs
  • Normalizing flows
  • Diffusion models

Models for continuous variables

Causal Inference, Counterfactual reasoning, Attribution

  • Understanding causality, causal graphs, identifiability
  • Estimation from observational data
  • Counterfactual explanations
  • Paper: Counterfactual Invariance to Spurious Correlations in Text Classification. NeurIPS 2021

Additional Topics

  • Bayesian Neural Networks: Uncertainties in Parameters Estimated with Neural Networks
  • Foundational models: Unsupervised training methods for high dimensional data: Robustness of foundational models
  • Tools: comparing distributions: Kernel methods (MMD, HSIC), Optimal transport,
  • Meta-learning: tutorial I , tutorial I , Turotial II
  • Uncertainty Estimation and out of distribution detection