Research

Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of newer evidence. Although recent studies have examined language-model behaviors in defeasible reasoning, the datasets have been static and lack wide coverage of non-monotonic reasoning categories. We introduce DeReLab, a generative framework that produces multi-turn belief-updating conversations from parameterized graph structures across default and inheritance reasoning, with formally verified ground truth at every turn, enabling controlled measurement of how models respond to confirming and disconfirming evidence. This controlled generation process creates a testbed for experimental designs that isolate specific reasoning demands. Applying this capability to the study of confirmation bias, we evaluate nine open and proprietary large language models and find that nearly all exhibit a systematic tendency to accept congruent evidence while resisting incongruent updates, with several models correctly identifying a weakening update yet failing to revise their conclusion. We believe our work and findings will facilitate future research on evaluating language models in defeasible reasoning.

This study investigated gendered stereotypes in emotional attributes within multilingual LLMs for Bangla, analyzing historical patterns revealing how women were often associated with emotions like empathy and guilt while men were linked to anger and authority in Bangla-speaking regions. We evaluated both closed and open-source LLMs to identify gender biases in emotion attribution and developed publicly available datasets.

This project focused on examining social biases (gender and religious) in LLMs for the Bangla language. We developed a curated dataset to benchmark bias measurement and implemented two probing techniques for bias detection. This work represents the first comprehensive bias assessment study for Bangla LLMs.

This work explores the nuances of gender bias detection in Bangla language models. We constructed a curated dataset for detecting gender bias in both static and contextual setups and compared different bias detection methods tailored to Bangla. The study established benchmark statistics using baseline methods and analyzed bias in various models supporting Bangla including BanglaBERT, MuRIL, and XLM-RoBERTa, with a key focus on how context length affects bias detection outcomes.

This research examines the cognitive capabilities of LLMs in the domain of Theory of Mind (ToM) tests across languages. The objective is to ascertain LLM capabilities in multilingual ToM questionnaires and study whether the inclusion of culturally specific elements changes model performance. Supervised two junior students in this project.