Explainable Verbal Reasoner Plus (EVR+): A Natural Language Reasoning Framework that Supports Diverse Compositional Reasoning

要約

タイトル：Expalainable Verbal Reasoner Plus (EVR+): 複合的理論をサポートする自然言語推論フレームワーク
要約：
– 言語モデルはNLPの多様な推論タスクに成功しているが、複合的な汎化に依然として問題がある。
– 本論文では、自己解釈可能なブール演算子生成機能と複数の簡便なタスクに分解する自由度を持つ手法「Explainable Verbal Reasoner Plus」を提案している。
– EVR+は、従来の「EVR」と同様の考え方を採用するが、より多様な複合型の推論、例えば巨大な演算子や再帰の異なる形式をサポートする。
– EVR+が推論フレームワークの性能を評価するために、複合的な推論を要する5つのタスクで構成された人工データセットを構築し、その結果を検証した。

要約（英語オリジナル）：
– Language models have been successfully applied to a variety of reasoning tasks in NLP, yet the language models still suffer from compositional generalization.
– In this paper we present Explainable Verbal Reasoner Plus (EVR+), a reasoning framework that enhances language models’ compositional reasoning ability by (1) allowing the model to explicitly generate and execute symbolic operators, and (2) allowing the model to decompose a complex task into several simpler ones in a flexible manner.
– Compared with its predecessor Explainable Verbal Reasoner (EVR) and other previous approaches adopting similar ideas, our framework supports more diverse types of reasoning such as nested loops and different types of recursion.
– To evaluate our reasoning framework, we build a synthetic dataset with five tasks that require compositional reasoning. Results show that our reasoning framework can enhance the language model’s compositional generalization performance on the five tasks, using a fine-tuned language model. We also discussed the possibility and the challenges to combine our reasoning framework with a few-shot prompted language model.

要約(オリジナル)

Languages models have been successfully applied to a variety of reasoning tasks in NLP, yet the language models still suffer from compositional generalization. In this paper we present Explainable Verbal Reasoner Plus (EVR+), a reasoning framework that enhances language models’ compositional reasoning ability by (1) allowing the model to explicitly generate and execute symbolic operators, and (2) allowing the model to decompose a complex task into several simpler ones in a flexible manner. Compared with its predecessor Explainable Verbal Reasoner (EVR) and other previous approaches adopting similar ideas, our framework supports more diverse types of reasoning such as nested loops and different types of recursion. To evaluate our reasoning framework, we build a synthetic dataset with five tasks that require compositional reasoning. Results show that our reasoning framework can enhance the language model’s compositional generalization performance on the five tasks, using a fine-tuned language model. We also discussed the possibility and the challenges to combine our reasoning framework with a few-shot prompted language model.

arxiv情報

提供元, 利用サービス

arxiv.jp, OpenAI