DiscoFuse: A Large-scale Dataset for Discourse-based Sentence Fusion
Abstract
Sentence fusion is the task of joining several
independent sentences into a single coherent
text. Current datasets for sentence fusion are
small and insufficient for training modern neural models. In this paper, we propose a method
for automatically-generating fusion examples
from raw text and present DISCOFUSE, a large
scale dataset for discourse-based sentence fusion. We author a set of rules for identifying
a diverse set of discourse phenomena in raw
text, and decomposing the text into two independent sentences. We apply our approach
on two document collections: Wikipedia and
Sports articles, yielding 60 million fusion examples annotated with discourse information
required to reconstruct the fused text. We develop a sequence-to-sequence model on DISCOFUSE and thoroughly analyze its strengths
and weaknesses with respect to the various discourse phenomena, using both automatic as
well as human evaluation. Finally, we conduct transfer learning experiments with WEBSPLIT, a recent dataset for text simplification. We show that pretraining on DISCOFUSE
substantially improves performance on WEBSPLIT when viewed as a sentence fusion task.
independent sentences into a single coherent
text. Current datasets for sentence fusion are
small and insufficient for training modern neural models. In this paper, we propose a method
for automatically-generating fusion examples
from raw text and present DISCOFUSE, a large
scale dataset for discourse-based sentence fusion. We author a set of rules for identifying
a diverse set of discourse phenomena in raw
text, and decomposing the text into two independent sentences. We apply our approach
on two document collections: Wikipedia and
Sports articles, yielding 60 million fusion examples annotated with discourse information
required to reconstruct the fused text. We develop a sequence-to-sequence model on DISCOFUSE and thoroughly analyze its strengths
and weaknesses with respect to the various discourse phenomena, using both automatic as
well as human evaluation. Finally, we conduct transfer learning experiments with WEBSPLIT, a recent dataset for text simplification. We show that pretraining on DISCOFUSE
substantially improves performance on WEBSPLIT when viewed as a sentence fusion task.
Research Areas
