Workshop on
Systems for LLMs
 co-organized by CSE @ IITBombay and IBM Research India

The Workshop on Systems for LLMs is focused on learning basics that will help to get started on System optimizations for LLM workloads. The workshop will cover basics of transformer architecture, usage of GPUs for LLMs, platforms for LLMs, Inference Systems and Distributed training. Created by Systems people for Systems people, the workshop covers various prior art in the space of Systems for LLMs.

The hands-on session of the workshop will cover the following topics:
  • Building your own GPT model
  • Exploring pytorch, profiling LLMs
  • Exploring inference generation and implementing KV cache
  • Implementing simulated version of distributed training algorithms

    Pre-requisites: Exposure/background in the following topics would be useful---
  • Operating Systems, Computer Architecture, Data structures
  • Machine Learning, Deep Learning, LLM Basics
  • Programming with python

    Dates: 6th and 7th September 2025, 9.30am to 5.30pm.

    Venue: CC103 and SL2, Computing Complex
                Department of Computer Science and Engineering, IIT Bombay

    Registration:
    Capacity --- 100 (in-person) participants.
    Only IITB students can attend in-person. External participants can attend online.
    Confirmation of registration will be sent separately, after filling the following registration form. Resources:
    1. Introduction to LLMs
    2. GPUs and LLMs
    3. Inference Systems
    4. Distributed training
    (tentative) Schedule:
    Day 1
    6th Sept.
    		Day 2
    7th Sept.
    9am to 10.30am Deep-dive into Transformers (and Attention) State-of-the-art LLM inferencing
    11am to 1pm Build your own GPT model KV cache setup for LLM inferencing
    2pm to 3.45pm GPUs and the LLM lifecycle Distributed training algorithms with multiple GPUs
    4pm to 5.30pm Resource usage and profiling of LLMs Distributed training algorithms with multiple GPUs