Modular Generation-and-Selection Framework Improves Faithful Summarization Under Budget Constraints
A novel modular framework for abstractive summarization separates the processes of generation and selection to tackle issues such as factual inconsistency, redundancy, and inadequate length control. This method employs a pretrained generator to create several candidate summaries, which are then broken down into individual sentence-level options. A combinatorial selector finalizes the summary by optimizing for relevance, factual accuracy, and redundancy while adhering to a specified sentence limit. The framework accommodates various selection techniques, including MMR, ILP, and a log-determinant objective inspired by DPP, without necessitating generator retraining. Tests conducted on CNN/DailyMail, Multi-News, FaithBench, and TofuEval demonstrate consistent enhancements in factuality and source-grounding, especially in multi-document summarization, albeit with reduced reference-overlap scores. Human assessments reveal improved perceived consistency, relevance, clarity, and conciseness, alongside a slight dip in coherence. The paper can be found on arXiv with the identifier 2608.03655.
Key facts
- The framework decouples generation and selection for sentence-budget-constrained summarization.
- A pretrained generator produces multiple candidate summaries.
- Candidates are decomposed into sentence-level candidates.
- A combinatorial selector balances relevance, factuality, and redundancy under an explicit budget.
- Supports MMR, ILP, and a DPP-inspired log-determinant objective without retraining the generator.
- Experiments on CNN/DailyMail, Multi-News, FaithBench, and TofuEval show improvements in factuality and source-grounding.
- Improvements are especially notable for multi-document summarization.
- Human evaluation shows higher perceived consistency, relevance, clarity, and conciseness, with a small reduction in coherence.
Entities
Institutions
- arXiv