Tutorial: Harnessing LLMs with Prompts: Applications, Challenges, and Maximizing Their Potential
Time and Venue
-
Venue: ICON 2023
-
Time: 10 AM - 1 PM IST, Thursday, December 14th, 2023
Abstract
In recent times, Large Language Models (LLMs) have shown excellent growth in their capabilities in generating language. One of the major factors in their surge is the availability of large-scale monolingual clean corpora and the increased capacity of these models. However, probing techniques are required to extract the most out of these models, which can retrieve appropriate information from these models. To standardize this, the field of prompting was introduced. But even with prompting, circumstances bring forth the massive computing requirements that come with these models to make them powerful by using billions of parameters. Due to this, pre-training and fine-tuning such models becomes an extremely compute-intensive task, making it infeasible to perform them repeatedly. This calls for developing parameter-efficient methods to tune these models for downstream NLP tasks in order to make the most use of these models with minimum cost.
The aim of this tutorial is to introduce prompting and other parameter-efficient fine-tuning (PEFT) methods for LLMs. We also plan to show how to harness the language understanding of LLMs by applying prompt-based techniques to different tasks, namely translation, debiasing, Reasoning and Hallucination. Each segment of our tutorial will be accompanied by practical, hands-on sessions designed to assist attendees.
Tutorial Prerequisite
Attendees should be familiar with the fundamentals of linear algebra and dimensionality reduction. Basic NLP knowledge would be beneficial but is not required. We intend to make the tutorial self-contained. The tutorial materials such as the slides and video recordings will be made publicly available for later reference.
Instructors' Bio
-
Tejomay Kishor Padole is a 2nd year PhD
student in the Department of Computer Science and Engineering at IIT Bombay working under the guidance of Prof. Pushpak Bhattacharyya. Previously, he has worked on Large Language models and prompt engineering applied to Machine Translation. Currently, his research interests lie in diffusion models
for text generation.
-
Meet Doshi is a final year M.Tech student at
the Department of Computer Science Engineering, IIT Bombay, under the guidance of Prof. Pushpak Bhattacharyya. His research focuses on Large Language Models and Multilinguality. He has previously interned at Oracle Labs and Phenom People. He is also the recipient of the prestigious Reliance Foundation Post Graduate Scholarship.
-
Ashita Saxena is a 3rd year MS by Research (CSE) student at IIT Bombay supervised by Prof. Pushpak Bhattacharyya. Her research focuses on hallucination detection and mitigation in NLP tasks and her work has been published in EMNLP. She has worked as a Research Intern at IBM Research on Natural Language Generation (NLG) tasks. She has also worked as a teaching assistant for the courses Deep Learning for Natural Language Processing and Digital Image Processing.
-
Kishan Maharaj is an MS (by research) student in the Department of Computer Science and Engineering, IIT Bombay, guided by Prof. Pushpak Bhattacharyya. His research interest lies in cognitively Inspired Natural language processing, mainly focusing on Hallucination detection and mitigation. His recent work on cognitively inspired hallucination detection was published in EMNLP.
-
Arif Ahmad is currently in the final year of a BTech/MTech dual degree in Electrical Engineering and AI at IIT Bombay. He is working in the area of Fairness and Bias in NLP Data and Models, under the supervision of Prof. Pushpak Bhattacharyya at the CFILT Lab in IIT Bombay.
-
Nihar Ranjan Sahoo is a PhD student in the Computer Science department of Indian Institute
of Technology Bombay, supervised by Prof.
Pushpak Bhattacharyya. His research interest lies in Ethical AI, social biases/toxicity
in languages, fairness in ML, and explainability in NLP. He has worked as a teaching assistant for undergraduate and graduate
students in AI, ML, and Deep Learning for
NLP courses. He has taught a tutorial on end-to-end NLP pipeline. He has co-authored a
computer vision paper published at BMVC
2021 conference and got the best student
paper(Runner’s up) award. He has published
paper on bias detection at conferences such as
LREC, CoNLL, ACL.
-
Dr. Pushpak Bhattacharyya is Professor of Computer Science and Engineering at IIT Bombay. Educated in the IIT System (B.Tech IIT Kharagpur, M.Tech IIT Kanpur, PhD IIT
Bombay), Dr. Bhattacharyya has done extensive research in Natural Language Processing and Machine Learning. He has published more than 350 research papers, has
authored/co-authored 6 books including a textbook on machine translation, and has guided
more than 350 students for their PhD, Masters
and Undergraduate thesis. He has received
many Research Excellence Awards- Manthan
award from Ministry of IT, H.H. Mathur and
P.K.Patwardhan awards from IIT Bombay,
VNMM award from IIT Roorkee, and substantial research grants from Government and
industry. Prof. Bhattacharyya holds the Bha-
gat Singh Rekhi Chair Professorship of IIT
Bombay, is a Fellow of National Academy
of Engineering, Abdul Kalam National Fellow, Distinguished Alumnus of IIT Kharagpur,
past Director of IIT Patna and past President
of Association of Computational Linguistics.