Hierarchical Neural Story Generation
Angela Fan、Mike Lewis Facebook AI Research, Menlo Park {angelafan, mikelewis,、Yann Dauphin
Abstract
We explore story generation: creative systems that can build coherent and fluent passages of text about a topic. We collect a large dataset of 300K human-written stories paired with writing prompts from an online forum. Our dataset enables hierarchical story generation, where the model first generates a premise, and then transforms it into a passage of text. We gain further improvements with a novel form of model fusion that improves the relevance of the story to the prompt, and adding a new gated multi-scale self-attention mechanism to model long-range context. Experiments show large improvements over strong baselines on both automated and human evaluations. Human judges prefer stories generated by our approach to those from a strong non-hierarchical model by a factor of two to one.
中文速览
故事自动生成长期面临两大难题：模型容易跑题、无法维持跨段落的长程连贯性。研究者从Reddit的WritingPrompts论坛收集了30万条人类创作的"提示词-故事"配对数据，构建了一套分层生成框架（hierarchical generation）——先由提示词生成一句故事前提，再以该前提为锚点展开完整叙事，从而为模型提供明确的情节支撑。在此基础上，他们引入了两项关键改进：一是"模型融合"（model fusion）机制，通过让第二个模型学习第一个预训练模型遗漏的内容，迫使生成结果真正呼应输入提示；二是"门控多尺度自注意力"（gated multi-scale self-attention）机制，让卷积解码器能够在不同时间粒度上回顾任意历史词语，突破了卷积网络只能看有限上下文的瓶颈。实验表明，人类评估者选择该方法生成故事的频率是非分层基线模型的两倍，这一工作为神经网络在长文档创意写作领域树立了新的性能标杆。
原文 arXiv:1805.04833;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1805.04833v1