Publications

We enhance the ESA protocol for machine translation evaluation by pre-filling error spans with AI-generated quality estimates, cutting annotation costs by up to 24%.

Vilém Zouhar, Tom Kocmi, Mrinmaya Sachan

preprint 2024    

Can LLMs help students learn from mistakes? Models struggle to spot student errors, but a verification step improves the overall quality of tutor response generation.

Nico Daheim, Jakub Macina, Manu Kapur, Iryna Gurevych, Mrinmaya Sachan

EMNLP 2024    

We develop an interactive system that helps users assess the reliability of LLM-generated text by analyzing self-consistency across multiple responses.

Furui Cheng, Vilém Zouhar, Simran Arora, Mrinmaya Sachan, Hendrik Strobelt, Mennatallah El-Assady

CHI 2024    

We present a new balanced human annotation protocol to evaluate the quality of machine translations, which is more economical than the alternatives.

Tom Kocmi, Vilém Zouhar, Eleftherios Avramidis, Roman Grundkiewicz, Marzena Karpinska, Maja Popović, Mrinmaya Sachan, Mariya Shmatova

WMT 2024    

We present ALT (ALignment with Textual feedback), an approach that aligns language models with user preferences expressed in text.

Saüc Abadal Lloret, Shehzaad Dhuliawala, Keerthiram Murugesan, Mrinmaya Sachan

ICML 2024 Workshop MHFAIA    

We propose a framework for generating synthetic teacher-student interactions grounded in a set of textbooks and build a dataset based on it.

Junling Wang, Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury, Mrinmaya Sachan

ACL 2024 (findings)    

We study the role and distribution of guiding questions in academic writing and use language models to generate them for active reading.

Peng Cui, Vilém Zouhar, Xiaoyu Zhang, Mrinmaya Sachan

ACL 2024    

We Develop Prototype tutoring System for Math Word Problems that tracs the solution state deterministically which leveraging the power of LLMs like GPT4 to produce mor natural sounding responses.

Sankalan Pal Chowdhury, Vilém Zouhar, Mrinmaya Sachan

Learning at Scale 2024    

We study whether large language models exhibit the same biases as human children when posed with arithmetic word problems.

Andreas Opedal, Alessandro Stolfo, Haruki Shirakami, Ying Jiao, Ryan Cotterell, Bernhard Schölkopf, Abulhair Saparov, Mrinmaya Sachan

ICML 2024    

We develop novel methods for phonetically informed word embeddings and propose evaluation strategies for their effectiveness.

Vilém Zouhar, Kalvin Chang, Chenxuan Cui, Nathaniel Carlson, Nathaniel Robinson, Mrinmaya Sachan and David Mortensen

LREC-COLING 2024    

We explore a mechanistic interpretation of LMs for arithmetic reasoning tasks using causal mediation analysis.

Alessandro Stolfo, Yonatan Belinkov and Mrinmaya Sachan

EMNLP 2023    

We propose an approach to semi-synthetically generate a pedagogically rich dialog dataset by pairing real teachers with a large language models scaffolded to represent common student errors.

Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury, Tanmay Sinha, Manu Kapur, Iryna Gurevych and Mrinmaya Sachan

EMNLP 2023 (Findings)   Code  

We explore a mechanistic interpretation of LMs for multi-step reasoning tasks.

Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut and Mrinmaya Sachan

EMNLP 2023    

We propose a data synthesis approach that iteratively extrapolates errors made by a small model trained on the synthesized dataset on a real-world validation dataset using LLMs.

Ruida Wang, Wangchunshu Zhou and Mrinmaya Sachan

EMNLP 2023 (Findings)    

We study how topic models can be automatically evaluated using LLMs.

Dominik Stammbach, Vilém Zouhar, Alexander Hoyle, Mrinmaya Sachan and Elliott Ash

EMNLP 2023 (Short)    

We study how user trust is developed and how it can be regained after potential trust-eroding events in Human-AI collaboration.

Shehzaad Dhuliawala, Vilém Zouhar, Mennatallah El-Assady and Mrinmaya Sachan

EMNLP 2023    

We propose a benchmark dataset to test the pure causal inference skills of large language models (LLMs) and identify a key shortcoming of LLMs in terms of their causal inference skills.

Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona Diab and Bernhard Schölkopf

arXiv 2023    

We propose a new NLP task, causal inference in natural language, inspired by the “causal inference engine” postulated by Judea Pearl et al.

Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng LYU, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan and Bernhard Schölkopf

NeurIPS 2023    

We reduce the complexity of structure prediction tasks to linear by casting the relation between tokens as a partial order over the string.

Tianyu Liu, Afra Amini, Mrinmaya Sachan and Ryan Cotterell

EMNLP 2023 (Outstanding Paper Award)    

We present RecurrentGPT, a language based simulacrum of the recurrence mechanism in RNNs which enables us to use ChatGPT to generate long texts.

Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell and Mrinmaya Sachan

arXiv 2023    

We propose DYNAICL, a recipe for efficient prompting with black-box generalist models that dynamically allocate in-context examples according to the input complexity and the computational budget.

Wangchunshu Zhou, Yuchen Eleanor Jiang, Ryan Cotterell and Mrinmaya Sachan

arXiv 2023    

We introduce NLP4SGPAPERS, a scientific dataset with three associated tasks that can help identify NLP for social good (NLP4SG) papers and characterize the NLP4SG landscape.

Fernando Gonzalez, Zhijing Jin, Bernhard Schölkopf, Tom Hope, Mrinmaya Sachan and Rada Mihalcea

EMNLP 2023 (Findings)    

We propose a vision-language model based approach to automatically enhance textbooks with images from the web.

Janvijay Singh, Vilém Zouhar and Mrinmaya Sachan

EMNLP 2023    

We propose a new approach for mitigating hallucination or generation of unverifiable information in dialog models

Nico Daheim, Nouha Dziri, Mrinmaya Sachan, Iryna Gurevych and Edoardo M Ponti

arxiv:2303.17574    

We combine a knowledge tracing model that estimates each student’s evolving knowledge states from their learning history and a controlled text generation model that generates exercises based on the student’s current estimated knowledge state and instructor requirements of desired properties for the next exercise.

Peng Cui and Mrinmaya Sachan

ACL 2023    

We propose a causal framework to quantify the robustness of the reasoning abilities of language models.

Alessandro Stolfo, Zhijing Jin, Kumar Shridhar, Bernhard Schölkopf and Mrinmaya Sachan

ACL 2023 (also at MATHAI Workshop at NeurIPS 22)    

We propose a knowledge distillation framework that leverages step-by-step reasoning capabilities of larger models and distills these capabilities into smaller models.

Kumar Shridhar, Alessandro Stolfo and Mrinmaya Sachan

ACL 2023 (Findings)    

As a step towards developing interpretable language model reasoners, we develop a semantic formalism representing the world depicted in math story problems.

Andreas Opedal, Niklas Stoehr, Abulhair Saparov and Mrinmaya Sachan

ACL 2023 (Findings)    

We formalize BPE as a combinatorial optimization problem and use submodularity to prove that the iterative greedy approach to BPE is approximately optimal.

Vilém Zouhar, Tim Vieira, Clara Meister, Juan Luis Gastaldi, Mrinmaya Sachan and Ryan Cotterell

ACL 2023 (Findings)    

We present InstructCTG, a controlled text generation framework that incorporates different constraints by conditioning the text generation process on natural language descriptions and demonstrations of the constraints.

Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Cotterell and Mrinmaya Sachan

ICML 2023    

We introduce LatFormer, a model that incorporates lattice geometry and topology priors in attention masks for abstract reasoning problems.

Mattia Atzeni, Mrinmaya Sachan and Andreas Loukas

ICML 2023    

We automatically generate math word problems of various difficulties that meet the needs of teachers in teaching and testing students in various educational stages.

Ying Jiao, Kumar Shridhar, Peng Cui, Wangchunshu Zhou and Mrinmaya Sachan

AIED 2023    

We analyze the potential of neural models on dialog tutoring datasets for language learning using a suite of automatic and human evaluations.

Jakub Macina, Nico Daheim, Lingzhi Wang, Tanmay Sinha, Manu Kapur, Iryna Gurevych and Mrinmaya Sachan

EACL 2023   Code  

We propose a self-distillation approach to jointly predict teaching strategies and generate tutor responses accordingly for dialog tutoring.

Lingzhi Wang, Mrinmaya Sachan, Xingshan Zeng and Kam-Fai Wong

EACL 2023 (Short, Findings)    

We reconstruct a long document coreference corpus out of Ontonotes, arriving at a more challenging coreference task and dataset.

Kumar Shridhar, Nicholas Monath, Raghuveer Thirukovalluru, Alessandro Stolfo, Manzil Zaheer, Andrew McCallum and Mrinmaya Sachan

EACL 2023 (Findings)    

We use reinforcement learning and language models to generate sequential subquestions for guiding (machines/humans) in math word problem-solving.

Kumar Shridhar, Jakub Macina, Mennatallah El-Assady, Tanmay Sinha, Manu Kapur and Mrinmaya Sachan

EMNLP 2022 (also at MATHAI Workshop at NeurIPS 22)   Code  

We show that zero-shot text classification can be improved simply by clustering texts in the embedding spaces of LMs.

Yu Fei, Zhao Meng, Ping Nie, Roger Wattenhofer and Mrinmaya Sachan

EMNLP 2022    

We generate synthetic datasets from differentially private LMs as a solution for sharing textual data while protecting the privacy of users.

Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schölkopf and Mrinmaya Sachan

EMNLP 2022    

We model structures as an autoregressive sequence of actions with LMs and achieve strong results on various sturctured prediction problems.

Tianyu Liu, Yuchen Eleanor Jiang, Nicholas Monath, Ryan Cotterell and Mrinmaya Sachan

EMNLP 2022 (Findings, Short paper)    

We enhance multilingual LMs with knowledge from multilingual knowledge graphs to tackle language and knowledge graph tasks across many languages.

Yifan Hou, Wenxiang Jiao, Meizhen Liu, Carl Allen, Zhaopeng Tu and Mrinmaya Sachan

EMNLP 2022 (Findings) / Best paper at the Multilingual Representation Learning (MRL) Workshop    

We introduce a new reasoning task and dataset of Logical fallacy detection.

Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xiaoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea and Bernhard Schölkopf

EMNLP 2022 (Findings)    

We propose a probe model based on graph convolutions to interpret knowledge-enhanced LMs and understand what kind of knowledge is integrated into these models.

Yifan Hou, Guoji Fu and Mrinmaya Sachan

EMNLP 2022 (Findings)   Code  

We present a novel challenge set that highlights the flexibility of the human moral mind, analyze the performance of language models on it, and propose a Moral Chain-of-Thought prompting strategy.

Zhijing Jin, Sydney Levine, Fernando Gonzalez Adauto, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Joshua B. Tenenbaum and Bernhard Schölkopf

NeurIPS 2022 (Oral) / CogSci 2022 (Disciplinary Diversity and Integration Award)    

We propose kernel learning methods that increase the expressiveness of Efficient Transformers while keeping their complexity linear.

Sankalan Pal Chowdhury, Adamos Solomou, Avinava Dubey and Mrinmaya Sachan

TMLR 2022    

We propose a probing approach that adapts the given task into a sentence completion format and performs probing using the built-in language modeling head.

Jiaoda Li, Ryan Cotterell and Mrinmaya Sachan

NAACL 2022    

We propose a novel automatic metric to widen the scope of automatic MT evaluation from sentence to document level.

Yuchen Eleanor Jiang, Tianyu Liu, Shuming Ma, Dongdong Zhang, Jian Yang, Haoyang Huang, Rico Sennrich, Mrinmaya Sachan, Ryan Cotterell and Ming Zhou

NAACL 2022    

We propose a structured model which directly learns to select an optimal set of spans for various span selection problems.

Tianyu Liu, Yuchen Eleanor Jiang, Ryan D Cotterell and Mrinmaya Sachan

NAACL 2022    

We provide a causal analysis of the impact of translationese on Machine Translation performance.

Jingwei Ni, Zhijing Jin, Markus Freitag, Mrinmaya Sachan and Bernhard Schölkopf

NAACL 2022    

We improve BERT’s robustness against word substitution attacks by leveraging self-supervised contrastive learning.

Zhao Meng, Yihan Dong, Mrinmaya Sachan and Roger Wattenhofer

NAACL 2022 (Findings)    

We study language change through the lens of causality in order to model how various distributional factors causally effect language change

Daphna Keidar, Andreas Opedal, Zhijing Jin and Mrinmaya Sachan

ACL 2022    

We propose approaches to calibrate open-domain machine reading systems

Shehzaad Dhuliawala, Leonard Adolphs, Rajarshi Das and Mrinmaya Sachan

ACL 2022 (Findings)    

We propose a case-based reasoning approach to train agents and generalize efficiently out of the training distribution.

Mattia Atzeni, Shehzaad Dhuliawala, Keerthiram Murugesan and Mrinmaya Sachan

ICLR (2022)    

We jointly map contextualized text representations to a lower dimensional space and cluster them for syntax induction.

Vikram Gupta, Haoyue Shi, Kevin Gimpel and Mrinmaya Sachan

AAAI (2022)    

We investigate role of the process of data collection in NLP and explain it using causality.

Zhijing Jin, Julius von Kugelgen, Jingwei Ni, Tejas Vaidhya, Ayush Kaushal, Mrinmaya Sachan and Bernhard Schoelkopf

EMNLP 2021    

We introduce a dataset for literary pieces and their summaries together with descriptions of characters that appear in the texts.

Faeze Brahman, Meng Huang, Oyvind Tafjord, Chao Zhao, Mrinmaya Sachan and Snigdha Chaturvedi

EMNLP 2021 (Findings)    

We introduce a new differentiable subset pruning technique for Transformer head pruning.

Jiaoda Li, Ryan Cotterell and Mrinmaya Sachan

TACL 2021   Code  

We introduce a information-theoretic probe to detect if contextualized text representations encode information in linguistic graphs.

Yifan Hou and Mrinmaya Sachan

ACL 2021   Code  

We introduce a coreference model which scales to documents of any length.

Raghuveer Thirukovalluru, Nicholas Monath, Kumar Shridhar, Manzil Zaheer, Mrinmaya Sachan and Andrew McCallum

ACL 2021 (Findings)    

We propose thinking frameworks to understand the direct and indirect real-world impact of NLP.

Zhijing Jin, Geeticka Chauhan, Brian Tse, Mrinmaya Sachan and Rada Mihalcea

ACL 2021 (Findings)   MIT News Article  

We jointly map contextualized text representations to a lower dimensional space and cluster them for syntax induction.

Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi,Kartik Talamadupula, Mrinmaya Sachan and Murray Campbell

ACL 2021 (Short paper)    

We design new environments to test the ability of RL agents to utilize commonsense knowledge.

Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Pushkar Shukla, Sadhana Kumaravel, Gerald Tesauro, Kartik Talamadupula, Mrinmaya Sachan and Murray Campbell

AAAI 2021    

We harvest structured knowledge of geometry from math textbooks.

Mrinmaya Sachan, Avinava Dubey, Eduard Hovy, Tom Mitchell, Dan Roth and Eric P. Xing

Computational Linguistics (CL) journal - Dec 2019 issue    

We learn a pipeline process that incorporates existing code, pre-learned machine learning models, and human engineered rules.

Mrinmaya Sachan, Avinava Dubey, Tom Mitchell, Dan Roth and Eric P. Xing

NeurIPS 2018    

We propose a meta-learning approach for universal NMT.

Emmanouil Antonios Platanios, Mrinmaya Sachan, Graham Neubig and Tom Mitchell

EMNLP 2018    

We learn to solve geometry problems by imitation demonstrations in textbooks.

Mrinmaya Sachan and Eric P. Xing

StarSem 2017    

We propose a structured prediction approach to answer science questions in textbooks.

Mrinmaya Sachan, Avinava Dubey and Eric P. Xing

ACL 2016 (Short paper)    

We propose an approach to answer reading comprehension questions using AMR graph representations.

Mrinmaya Sachan and Eric P. Xing

ACL 2016 (Short paper)    

We propose a structured prediction approach to answer reading comprehension questions.

Mrinmaya Sachan, Avinava Dubey, Eric P. Xing and Matthew Richardson

ACL 2015 (Outstanding paper)