CS784 - Image Synthesis

Semester 2 (Spring), 2025-2026


Overview

This course deals with the process of image formation or synthesis on a computer. We will begin with detailed discussions of the classical methods from Computer Graphics for rendering and cover this exciting and fast changing field through its modern inception in the era of neural rendering and generative models.

This is NOT a course in deep learning or machine learning. This is a course in visual computing. Students are expected to have done a first course in either of computer graphics, image processing or computer vision.

The course is meant to be a very hands-on course and will include a healthy mix of theory, programming assignments and assignments that require working in open source digital content creation (DCC) platforms like Blender, among others.

Teaser Video

Logistics


Registration Instructions

  • This is open to students from all departments who have a prior background in either of computer graphics, image processing, or computer vision. Prior background in introductory machine learning is also assumed. Note that none of these are stirctly enforced, but are strongly desired. If you take the course without requisite background, you do so at your own risk.

  • You are not allowed to register for this courses if you have done another course that overlaps significantly in content with it. If you have a doubt about this, ask me.

  • Any B.Tech., DD or IDDDP student who is in the 3rd year or above can take the course. Please do not take the course if you are an undergraduate student in the second year or earlier. Please make sure you have the requisite background as given above.

  • Any M.Tech., M.S. or Ph.D. student can take the course. Please make sure you have the requisite background as given above.

  • You can tag the course as whatever you want.
  • Audits are not allowed.

Eligibility/Prerequisites

If you are curious about image synthesis at all - attend the first lecture.

  • Prior knowledge of the following is strongly recommended

    • C/C++ and Python Programming.
    • Basic Linear Algebra is desirable.
    • Data structures and algorithms.
    • Computer Graphics, Image Processing or Computer Vision.
    • Introductory Machine Learning.

  • Learn how to use git.

  • Make sure you have a GitHub id. You will need it for the course.


Course Content

Here is a preliminary list of topics for the course. The contents may change as the course progresses.

  1. Global Illumination and Rendering - Radiometry, Photometry, Ray-tracing, Surface reflectance and BRDFs, Monte Carlo Path Tracing - Importance Sampling, Metropolis Light Transport, Photon Mapping, Ambient Illumination, Shaders, Volume Rendering

  2. Image-based Rendering - Planar panoramas, View Morphing, Novel View Synthesis, Plenoptic function, Lightfields,

  3. Neural and differentiable Rendering - Neural scene representation and rendering, Neural Radiance fields, Differentiable rendering, Inverse rendering

  4. Image Synthesis for Virtual and Augmented Reality - Spherical (360) Panoramas, Neural Antialiasing, Latency, Vergence-Accommodation Conflict and other causes of discomfort in VR, Foveated rendering

  5. Real or synthetic - Statistics of real images, Methods for spotting fake images, Responsible use of synthesized images


Lecture Schedule

Lecures slides for the course will be uploaded to IIT Bombay Moodle.

Additional reading material for all topics can be found here.

Date Topics Tasks and Resources
Jan 5 Introduction to the Course
Jan 8 Ray Tracing For additional readings, see here.
Jan 19
Jan 22		 Digital Light - Pixels, Light and Colour For additional readings, see here. Assignment 1 due on Jan 22.
Jan 29 Radiometry Assignment 2 has been announced.
Feb 3 BRDF For additional readings, see here.
Feb 5
Feb 9		 Monte Carlo Path Tracing For additional readings, see here. Read sections from the PBR book on estimator efficiency increasing methods and sampling of multidimensinal distributions.
Feb 9 Ambient Occlusion, Photon Mapping and Point-based GI Assignment 2 deadline was Feb 8.
Feb 12
Feb 16		 Neural Denoising and Caching Assignment 3 has been announced.
Feb 19 Doubts Clearing Class
Mar 1 Mid Semester Exam CC 103, 11:00am to 1:00pm

Assignments and Homeworks

Assignments for the course can be found on IIT Bombay Moodle

Resources

  1. Recorded Videos on YouTube for Computer Graphics (CS475/CS675)

    1. Recorded videos for the basic course on Computer Graphics can be found on YouTube here.

  2. Books

    1. Physically Based Rendering (4ed, 2023) 302226 from theory to Implementation, Matt Pharr, Wenzel Jacom and GregHumphreys, MIT Press.
    2. Principles of Digital Image Synthesis (1995), Vol 1 & 2,Andrew Glassner, Morgan Kaufmann.
    3. Computer Vision: Algorithms and Applications (2ed, 2023), Richard Szeliski, Springer.
    4. Understanding Deep Learning (2023), Simon J. D. Prince, MIT Press.

  3. Online Readings

    1. Neural Rendering
      a. Course on Neural Rendering from CVPR 2020

  4. Resources

    1. Pixels, Light and Colour
      1. A Biography of the PIXEL, Alvy Ray Smith
      2. Introduction to Light, Color and Color Space
      3. CIE XYZ and xyY Color Spaces Douglas A. Kerr
      4. Perceptually Uniform Color Models, interactive visualization, Max Bo
      5. Causes of Color
    2. Ray Tracing
      1. Ray Tracing
      2. The Internet Ray Tracing Competition (the competition seems to be down at the moment)
      3. The Persistence of Vision Raytracer
      4. Object-Object Intersections
      5. Overview of raytracing on Scratchapixel
    3. Radiometry and BRDF
      1. Fresnel Reflectance
      2. Data from Cornell Light Measurement Lab
      3. Experimental Validation of Analytical BRDF Models, Ngan, Durand, Matusik
      4. Notes on Radiometry by Prof. Steve Marschner, Cornell University
    4. Radiosity
      1. SIGGRAPH Education Radiosity Slide Set
      2. Modeling the interaction of light between diffuse surfaces, Goral, Torrance, Greenberg and Battaile, SIGGRAPH 84.
      3. The hemi-cube: a radiosity solution for complex environments, Cohen and Greenberg, SIGGRAPH 85.
      4. RADIOSITY: An Illuminating Perspective, Drucker, Written Doctoral Exam
    5. Monte-Carlo Raytracing
      1. smallpt: G.I. in 99 lines of C++
      2. The Global Illumination Compendium
      3. State of the Art in Monte Carlo Ray Tracing for Realistic Image Synthesis, SIGGRAPH 2001 course notes
    6. Ambient Occlusion, Photon Mapping, Point-based GI
      1. Point-based Global Illumination, Per H. Christensen, PIXAR
      2. A practical guide to using Photon Maps, Henrik Wann Jensen, SIGGRAPH 2000 Course
      3. Ambient Occlusion, Wikipedia
      4. Ambient Occlusion, GPU Gems, Chapter 17, NVIDIA
    7. Neural Denoising and Caching
      1. Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder, Chaitanya et al., SIGGRAPH/ACM TOG 2017
      2. A Gentle Introduction to ReSTIR: Path Reuse in Real-time, Wyman et al., SIGGRAPH 2023 Course
      3. Real-time Neural Radiance Caching for Path Tracing, Müller et al., SIGGRAPH/ACM TOG 2021

Last updated on Dec 31, 2025