Harsh Poonia

Please feel free to reach out on my email, I'm always up for a chat!

I am a first-year graduate student at Carnegie Mellon University, pursuing my Master's in Machine Learning. I recently graduated from the Indian Institute of Technology Bombay, where I majored in Computer Science and Engineering with honors.

My research interests lie broadly in machine learning and natural language processing, with a focus on generalizability and fairness. More specifically, I am interested in Foundation Models, Reinforcement Learning, Efficient Inference, and Interpretability in Sequence Models.

I have been fortunate to be advised by Prof. Devendra Singh Dhami at TU Eindhoven and Hessian Center for AI on my research around causal machine learning. I worked with him on a novel tractable probabilistic circuit that enabled causal inference in hybrid domains, as real world datasets commonly have both discrete and continuous random variables (published at UAI 2024). More recently, we collaborated on a novel joint optimization scheme for neural granger causality in the context of multivariate time series data. At IIT Bombay, I had the privilege of working with Prof. Preethi Jyothi for my bachelor's thesis. We researched flow matching and other generative approaches to compute invariant representations of speech, to improve recognition performance for non-native speakers. I have also worked with Prof. Sunita Sarawagi on leveraging the in-context learning ability of LLMs for few-shot learning in Text-to-SQL tasks.

In my spare time, I love to play all kinds of sports: football, basketball, badminton, and table tennis being my favorites. I also love to write, and I sketch sometimes.

Proximal Optimization for Sparse Granger Causality

A novel method for predicting sparse granger causal relations between a set of time series variables, using an xLSTM based architecture and a dynamic lasso penalty for inducing sparsity. Sepp Hochreiter recognised our work!

chiSPN: Characteristic Interventional Sum-Product Networks for Causal Inference in Hybrid Domains

We compute post-intervention likelihoods based on true causal relationships between variables. We developed a novel tractable probabilistic circuit, that enabled causal inference for hybrid domains, that is, on systems with both discrete and continuous random variables. By using characteristic functions to represent distributions, we were able to better model asymmetric and long-tailed distributions, which are especially prevalent in real-world datasets.

Generative Modelling for Invariant Speech Representations

I am working towards bridging the performance gap in speech recognition for non-native speakers across underrepresented accents and dialects, to create more inclusive models through equivariant learning approaches. We have also been working on efficient wakeword detection (”Hey Alexa!”), where equivariance is desired with respect to shifts within the audio. Our implementation of a shift-equivariant transformer has shown promising results on command recognition, matching state-of-the-art accuracy while displaying significantly less variance with shifts in input.

In-Context Learning in LLMs for Text-to-SQL

In collaboration with Amazon, I worked on a project to explore the theoretical underpinnings of the in-context learning capability of LLMs, and to better leverage it for something that can aid fast, natural language based retrieval in databases: the domain of Text-to-SQL models. Our approach combined constrained decoding with question decomposition to answer queries represented by sub-trees of the syntax tree of a complex query. We were able to reduce schema hallucinations and improve coherence of generated SQL through this bottom-up approach.