Publications

2024

1. NAACL

   LeanReasoner: Boosting Complex Logical Reasoning with Lean

   Dongwei Jiang , Marcio Fonseca , and Shay B. Cohen

   In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, 2024

   Abstract PDF

   Large language models (LLMs) often struggle with complex logical reasoning due to logical inconsistencies and the inherent difficulty of such reasoning. We use Lean, a theorem proving framework, to address these challenges. By formalizing logical reasoning problems into theorems within Lean, we can solve them by proving or disproving the corresponding theorems. This method reduces the risk of logical inconsistencies with the help of Lean’s symbolic solver. It also enhances our ability to treat complex reasoning tasks by using Lean’s extensive library of theorem proofs. Our method achieves state-of-the-art performance on the FOLIO dataset and achieves performance near this level on ProofWriter. Notably, these results were accomplished by fine-tuning on fewer than 100 in-domain samples for each dataset.

2. arXiv

   Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic

   Nathaniel Weir , Kate Sanders , Orion Weller , Shreya Sharma ,  Dongwei Jiang , Zhengping Jiang , Bhavana Dalvi Mishra , Oyvind Tafjord , Peter A. Jansen , Peter Clark , and Benjamin Van Durme

   CoRR, 2024

   Abstract PDF

   Recent language models enable new opportunities for structured reasoning with text, such as the construction of intuitive, proof-like textual entailment trees without relying on brittle formal logic. However, progress in this direction has been hampered by a long-standing lack of a clear protocol for determining what valid compositional entailment is. This absence causes noisy datasets and limited performance gains by modern neuro-symbolic engines. To address these problems, we formulate a consistent and theoretically grounded approach to annotating decompositional entailment and evaluate its impact on LLM-based textual inference. We find that our new dataset, RDTE (Recognizing Decompositional Textual Entailment), has a substantially higher internal consistency (+9%) than prior decompositional entailment datasets. We also find that training an RDTE-oriented entailment classifier via knowledge distillation and employing it in an entailment tree reasoning engine significantly improves both accuracy and proof quality, illustrating the practical benefit of this advance for textual inference.

3. arXiv

   SELF-[IN]CORRECT: LLMs Struggle with Refining Self-Generated Responses

   Dongwei Jiang , Jingyu Zhang , Orion Weller , Nathaniel Weir , Benjamin Van Durme , and Daniel Khashabi

   CoRR, 2024

   Abstract PDF

   Can LLMs consistently improve their previous outputs for better results? For this to be true, LLMs would need to be better at discriminating among previously-generated alternatives, than generating initial responses. We explore the validity of this hypothesis in practice. We first formulate a unified framework that allows us to compare the generative and discriminative capability of any model on any task. In our resulting experimental analysis of several open-source and industrial LLMs, we observe that models are not reliably better at discriminating among previously-generated alternatives than generating initial responses. This finding challenges the notion that LLMs may be able to enhance their performance only through their own judgment.

4. arXiv

   Benchmarking Language Model Creativity: A Case Study on Code Generation

   Yining Lu , Dixuan Wang , Tianjian Li ,  Dongwei Jiang , and Daniel Khashabi

   CoRR, 2024

   PDF

2021

1. ICASSP

   A Further Study of Unsupervised Pretraining for Transformer Based Speech Recognition

   Dongwei Jiang , Wubo Li , Ruixiong Zhang , Miao Cao , Ne Luo , Yang Han , Wei Zou , Kun Han , and Xiangang Li

   In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, 2021

   Abstract PDF

   Building a good speech recognition system usually requires large amounts of transcribed data, which is expensive to collect. To tackle this problem, many unsupervised pre-training methods have been proposed. Among these methods, Masked Predictive Coding achieved significant improvements on various speech recognition datasets with BERT-like Masked Reconstruction loss and Transformer backbone. However, many aspects of MPC have not been fully investigated. In this paper, we conduct a further study on MPC and focus on three important aspects: the effect of pre-training data speaking style, its extension on streaming model, and how to better transfer learned knowledge from pre-training stage to downstream tasks. Experiments reveled that pre-training data with matching speaking style is more useful on downstream recognition tasks. A unified training objective with APC and MPC provided 8.46% relative error reduction on streaming model trained on HKUST. Also, the combination of target data adaption and layer-wise discriminative training helped the knowledge transfer of MPC, which achieved 3.99% relative error reduction on AISHELL over a strong baseline.

2. ICASSP

   Transformer Based Unsupervised Pre-Training for Acoustic Representation Learning

   Ruixiong Zhang , Haiwei Wu , Wubo Li ,  Dongwei Jiang , Wei Zou , and Xiangang Li

   In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, 2021
3. InterSpeech

   Didispeech: A Large Scale Mandarin Speech Corpus

   Tingwei Guo , Cheng Wen ,  Dongwei Jiang , Ne Luo , Ruixiong Zhang , Shuaijiang Zhao , Wubo Li , Cheng Gong , Wei Zou , Kun Han , and Xiangang Li

   In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, 2021
4. InterSpeech

   Speech SimCLR: Combining Contrastive and Reconstruction Objective for Self-Supervised Speech Representation Learning

   Dongwei Jiang , Wubo Li , Miao Cao , Wei Zou , and Xiangang Li

   In 22nd Annual Conference of the International Speech Communication Association, Interspeech 2021, Brno, Czechia, August 30 - September 3, 2021, 2021

   Abstract PDF

   Self-supervised visual pretraining has shown significant progress recently. Among those methods, SimCLR greatly advanced the state of the art in self-supervised and semi-supervised learning on ImageNet. The input feature representations for speech and visual tasks are both continuous, so it is natural to consider applying similar objective on speech representation learning. In this paper, we propose Speech SimCLR, a new self-supervised objective for speech representation learning. During training, Speech SimCLR applies augmentation on raw speech and its spectrogram. Its objective is the combination of contrastive loss that maximizes agreement between differently augmented samples in the latent space and reconstruction loss of input representation. The proposed method achieved competitive results on speech emotion recognition and speech recognition.

2020

1. InterSpeech

   TMT: A Transformer-Based Modal Translator for Improving Multimodal Sequence Representations in Audio Visual Scene-Aware Dialog

   Wubo Li ,  Dongwei Jiang , Wei Zou , and Xiangang Li

   In 21st Annual Conference of the International Speech Communication Association, Interspeech 2020, Virtual Event, Shanghai, China, October 25-29, 2020, 2020

2019

1. arxiv

   Improving Transformer-based Speech Recognition Using Unsupervised Pre-training

   Dongwei Jiang , Xiaoning Lei , Wubo Li , Ne Luo , Yuxuan Hu , Wei Zou , and Xiangang Li

   CoRR, 2019

   Abstract PDF

   Speech recognition technologies are gaining enormous popularity in various industrial applications. However, building a good speech recognition system usually requires large amounts of transcribed data, which is expensive to collect. To tackle this problem, an unsupervised pre-training method called Masked Predictive Coding is proposed, which can be applied for unsupervised pre-training with Transformer based model. Experiments on HKUST show that using the same training data, we can achieve CER 23.3%, exceeding the best end-to-end model by over 0.2% absolute CER. With more pre-training data, we can further reduce the CER to 21.0%, or a 11.8% relative CER reduction over baseline.

2018

1. ISCSLP

   Comparable Study Of Modeling Units For End-To-End Mandarin Speech Recognition

   Wei Zou ,  Dongwei Jiang , Shuaijiang Zhao , Guilin Yang , and Xiangang Li

   In 11th International Symposium on Chinese Spoken Language Processing, ISCSLP 2018, Taipei City, Taiwan, November 26-29, 2018, 2018
2. ISCSLP

   An Analysis of Decoding for Attention-Based End-to-End Mandarin Speech Recognition

   Dongwei Jiang , Wei Zou , Shuaijiang Zhao , Guilin Yang , and Xiangang Li

   In 11th International Symposium on Chinese Spoken Language Processing, ISCSLP 2018, Taipei City, Taiwan, November 26-29, 2018, 2018
3. arxiv

   Towards End-to-End Code-Switching Speech Recognition

   Ne Luo ,  Dongwei Jiang , Shuaijiang Zhao , Caixia Gong , Wei Zou , and Xiangang Li

   CoRR, 2018