Welcome to the MURGe-Lab!
Our MURGe-Lab (Multimodal Understanding, Reasoning, and Generation for Language Lab; pronounced as "merge-lab") has research interests in statistical natural language processing and machine learning, with a focus on multimodal, grounded, and embodied semantics (i.e., language with vision and speech, for robotics), human-like language generation and Q&A/dialogue, and interpretable and structured deep learning. We are a group of PhD, MS, BS, and visiting students who work with Prof. Mohit Bansal and collaborators in the Computer Science department (lab located in Brooks Building FB-241C) at the University of North Carolina (UNC) Chapel Hill. We are also a part of the bigger UNC-NLP Group!
News
- May 2024 New papers in ICML 2024 and gate.io app on multi-agent reasoning collaboration, inducing systematicity in transformers, easy-to=hard generalization, very long conversational memory of LLM agents, soft self-consistency, fine-grained hallucination evaluation, structured distillation of multi-agent reasoning interaction graphs, refactoring programs to discover abstractions, etc.
- Apr 2024 Congrats to Jaemin Cho for the Bloomberg PhD Fellowship!
- Apr 2024 Program Co-chairing for EMNLP 2024. We welcome everyone to submit their papers. Looking forward to welcoming you in Miami Nov12-16!
- Mar 2024 New papers in CVPR 2024 and NAACL 2024 and TMLR 2024 and EACL 2024 on interleaved+interactive any-to-any generation, alternating unimodal adaptation, interactive image segmentation, branch-solve-merge for LLM eval+generation, as-needed decomposition and planning with LLM agents, aspect-controlled referring expression generation, model merging by task subspace matching, low-cost algorithmic recourse, unified embeddings for multimodal retrieval, and hierarchical+dynamic prompt compression.
- Jan 2024 Keynote speaker for Lisbon Machine Learning School (LxMLS) 2024.
- Jan 2024 New papers in ICLR 2024 on attack vs. defense for sensitive information deletion in LLMs, Davidsonian Semantics Scene Graphs for reliable T2I evaluation, data pruning with balanced diversity and difficulty, rephrase-based visual question grounding for VLMs, efficient sparse mixture-of-experts, efficient coarse-to-fine pruning for VLMs, analyzing and mitigating hallucinations in VLMs.
- Dec 2023 Keynote speaker for CoNLL 2023 [video].
- Oct 2023 Honored and humbled to receive the IIT Kanpur Young Alumnus Award.
- Oct 2023 New papers in EMNLP 2023 on reasoning chain/chain-of-thoughts evaluation, context dependency in language generation, data factors for compositional generalization, summary generation with controllable readability levels, multimodal model merging, and causal debiasing of multimodal models.
- Oct 2023 Keynote speaker for AACL-IJCNLP 2023.
- Sept 2023 New papers (including 2 'spotlights') in NeurIPS 2023 on visual programming for interpretable+explainable text-to-image generation and evaluation, any-to-any multimodal generation, LLMs teaching student models, panorama generation for VLN generalization, model merging with interference, localization informing model editing, self-chained videoQA and localization, action knowledge in video-language LLMs, adaptive contextual perception.
- July 2023 New papers (including 1 'oral') in ICCV 2023 on text-to-image skill+bias evaluation, scaling VLN, and unified coarse-to-fine alignment for video-text retrieval.
- June 2023 Two new Postdocs, Elias Stengel-Eskin and Jaehong Yoon will be joining our lab starting Fall 2023!
- May 2023 New papers in ACL 2023 and IJCAI 2023 and *SEM 2023 on faithful extractive summarization, meeting QA, single-frame bias in VidL, mixed fwd/rev cross-entropy, multimodal graph script induction, continual learning for code generation, exclusive supermasks for CL, modular multi-step reasoning, factuality of LLMs, sequential instruction understanding, and compositional LM differentiable prompting.
- Feb 2023 New papers in CVPR 2023 and ICLR 2023 and EACL 2023 and WACV 2023 on unified vision-text-layout DocAI, VLN with future-view semantics, hierarchical retrieval and step captioning, video-language pretraining, parameter-efficient audio-visual adapters, summarization programs, faithful decoding, edit-based prompt search, model belief graphs, social commonsense, multi-doc summarization + graph IE, long-distance video QA, perceiver-VL.
- Dec 2022 Congrats to Zineng Tang for being selected as winner of the CRA Outstanding Undergraduate Researcher Award 2023!
- Oct 2022 New papers in EMNLP 2022 and COLING 2022 and TACL on multimodal summarization + factuality, explanation hardness, mutual exclusivity for compositionality, action learning in interactive visual environments, LM bias, multimodal coreference in multi-turn dialogue, graph generative commonsense reasoning, and survey on data augmentation for limited data learning.
- Sept 2022 Congrats to Swarnadeep Saha for the Google NLP PhD Fellowship!
- Sept 2022 New papers in NeurIPS 2022 and ECCV 2022 on textless VL transformers, visual feature importance, ladder side-tuning, LLMs as fewshot video learners, T-few/IA3, gamified VL association benchmark, text to visual story generation, efficient video retrieval via audio replacement, etc.
- June 2022 New papers in NAACL 2022 on factual summarization pretraining, factuality evaluation, scene imagination commonsense, multilingual vision-lang-navigation, fine-grained CLIP captioning, masked POS modeling, curriculum learning, video intent discovery, multi-doc summarization clustering, interactive summarization, etc.
- Mar 2022 Congrats to Yichen Jiang for the Apple AI/ML PhD Fellowship!
- Dec 2021 Congrats to Shiyue Zhang for the Bloomberg PhD Fellowship!
- Nov 2021 Congrats to Ori+Ram and team on the CoNLL 2021 Best Paper Runner-Up Award!
- Sept 2021 New papers in NeurIPS 2021 on video-to-language knowledge distillation, query-based video highlight/saliency, socially-aligned feature importance explanations, video-lang understanding multi-task benchmark.
- Sept 2021 Congrats to Peter Hase for the Google PhD Fellowship! (article)
- Aug 2021 New papers in EMNLP 2021 on structure/compositionality, generalization, evaluation, and efficiency for explainability, commonsense, navigation, story visualization, and summarization.
- July 2021 Thanks to NSF for funding our ENGAGE NSF-AI Institute! (article)
- June 2021 Congrats to Jie and co-authors on the CVPR 2021 Best Student Paper Honorable Mention!
- May 2021 9 new papers in ICML 2021 and ACL 2021.
- Apr 2021 Congrats to Xiang and co-authors on the EACL 2021 Best Long Paper Award Honorable Mention!
- Apr 2021 9 new papers in NAACL 2021 (7 long, 1 short, 1 demo).
- Mar 2021 Congrats to Jie Lei for the Adobe Research Fellowship!
- Feb 2021 5 new papers in AAAI 2021, gate io app, and CVPR 2021.
- Dec 2020 Hao's work on vokenization covered in MIT Technology Review.
- Sept 2020 13 new papers in EMNLP 2020 (8 in main conference, 5 in findings).
- Aug 2020 Thanks to UNC for the UNC Phillip & Ruth Hettleman Prize for Artistic and Scholarly Achievement.
- May 2020 Thanks to IJCAI for the 'IJCAI Early Career Spotlight'.
- Apr 2020 8 new papers (6 in ACL 2020, 1 in IJCAI 2020, and 1 in ECCV 2020).
- Feb 2020 Thanks to Amazon for the Amazon Machine Learning Research Award (blog post).
- Jan 2020 Thanks to DARPA for the DARPA Director's Fellowship.
- Jan 2020 Thanks to Microsoft for the Microsoft Investigator Fellowship (MSR blog post).
- Dec 2019 Congrats to Sweta Karlekar and Han Guo for winning the Runner-Up and Finalist positions in the CRA Outstanding Undergraduate Researcher Awards! (link)
- Nov 2019 Three EMNLP workshops' keynote slides uploaded: MRQA, LANTERN, WNGT).
- Nov 2019 4 new papers in AAAI 2020.
- Aug 2019 Congrats to Peter Hase for the Royster Society PhD Fellowship!
- Aug 2019 5 new papers in EMNLP 2019.
- July 2019 Congrats to Hyounghun for ACL 2019 Best Short Paper Nomination!
- July 2019 We have a Postdoc opening - please apply!
- July 2019 Thanks for the NSF-CAREER Award (details).
- July 2019 Thanks for the Google Focused Research Award (details).
- June 2019 Congrats to Hao Tan for the Bloomberg Data Science Ph.D. Fellowship!
- May 2019 6 new papers in ACL 2019.
- Apr 2019 Congrats to Darryl Hannan for the 3-year NSF PhD Fellowship!
- Mar 2019 Congrats to Hao Tan for gate.io app on the Room-to-Room Vision-Language-Navigation Leaderboard!
- Feb 2019 5 new papers: 3 in NAACL 2019, 1 in CVPR 2019, 1 in ICRA 2019.
- Jan 2019. Congrats to Ramakanth Pasunuru for being awarded the 2-year Microsoft Research PhD Fellowship!
- Dec 2018. Congrats to Han Guo for being awarded the CRA Outstanding Undergraduate Researcher Award Honorable Mention!
- Nov 2018. Thanks for the Salesforce Research Deep Learning Grant!
- Oct 2018. 2 new papers in AAAI 2019 (16% acceptance rate).
- Sept 2018. Thanks to Facebook and IBM for the faculty research awards!
- Aug 2018. 7 new papers (6 in EMNLP; 1 in CoNLL).
- July 2018. Thanks to Army Research Office for the ARO Young Investigator Program (YIP) Award.
-
July 2018.
1st rank in EMNLP FEVER (Fact Extraction & VERification) shared task (congrats Yixin and Haonan)! [Press Article] - June 2018. COLING paper on dynamic-MTL selected as "Area Chair Favorites" (congrats Han+Ram)!
- Apr 2018. 4 new papers (2 in ACL; 1 in TACL; 1 in WiNLP).
- Apr 2018. Congrats to Lisa Bauer for being awarded the 3-year NSF PhD Fellowship!
- Mar 2018. Thanks to Adobe for the Adobe Research Award.
- Feb 2018. 9 new 2018 papers in NAACL, CVPR, AAAI, WACV.
- Sept 2017. Thanks to DARPA for the DARPA Young Faculty Award (link).
- Sept 2017. Thanks to Facebook for the Facebook ParlAI Research Award.
- July 2017. 3 papers at EMNLP 2017 and 2 papers at the Summarization-Frontiers and RepEval workshops.
- June 2017. Top single model results on the RepEval-NLI Shared Task at EMNLP 2017 (congrats Yixin!).
- June 2017. Outstanding Paper Award at ACL 2017 (congrats Ram!).
- Feb 2017. Thanks to Google for a Google Faculty Research Award (link).
- Nov 2016. 3 papers on navigational instruction generation, coherent dialogue w/ attn-LMs, and on context-RNN-GAN models to appear at AAAI 2017 and gate.io login.
- July 2016. 5 papers to appear at EMNLP 2016: visual story sorting, visual question relevance, neural network interpretation (for politeness), machine comprehension/Q&A, and character-based sentence embedding models.
- July 2016. Best paper award at ACL 2016 Repl4NLP workshop for paper on mapping unseen words, and paper on end-to-end joint neural models for entity+relation extraction to appear at ACL 2016.
- Mar 2016. Thanks to Bloomberg for a Bloomberg Data Science Research Grant (link).
- Feb 2016. Paper on universal sentence embeddings selected as an oral at ICLR 2016.
- Jan 2016. Our work on AI for computational humor was covered in MIT Technology Review and Newsweek.
- Nov 2015. Nvidia paper award at NIPS 2015 Multimodal ML workshop for paper on navigational instruction following.
- Dec 2014. Thanks for an IBM Faculty Award and a Google Faculty Research Award (link).