Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
Jiahui Yu Yuanzhong Xu Jing Yu Koh Thang Luong Gunjan Baid Zirui Wang Vijay Vasudevan Alexander Ku Yinfei Yang Burcu Karagol Ayan Ben Hutchinson Wei Han Zarana Parekh Xin Li Han Zhang Jason Baldridge Yonghui Wu {jiahuiyu, yuanzx, jykoh, thangluong, gunjanbaid, ziruiw, vrv, alexku, jasonbaldridge, Equal contribution. Core contribution. Google Research Correspondence to {jiahuiyu, jasonbaldridge,
Abstract
We present the Pathways [1] Autoregressive Text-to-Image (Parti) model, which generates high-fidelity photorealistic images and supports content-rich synthesis involving complex compositions and world knowledge. Parti treats text-to-image generation as a sequence-to-sequence modeling problem, akin to machine translation, with sequences of image tokens as the target outputs rather than text tokens in another language. This strategy can naturally tap into the rich body of prior work on large language models, which have seen continued advances in capabilities and performance through scaling data and model sizes. Our approach is simple: First, Parti uses a Transformer-based image tokenizer, ViT-VQGAN, to encode images as sequences of discrete tokens. Second, we achieve consistent quality improvements by scaling the encoder-decoder Transformer model up to 20B parameters, with a new state-of-the-art zero-shot FID score of 7.23 and finetuned FID score of 3.22 on MS-COCO. Our detailed analysis on Localized Narratives as well as PartiPrompts (P $2$ ), a new holistic benchmark of over 1600 English prompts, demonstrate the effectiveness of Parti across a wide variety of categories and difficu
中文速览
谷歌提出的Parti模型想解决一个核心难题：如何让AI根据一段文字描述，生成高保真、内容丰富、能体现复杂构图和世界知识的图像。它把文字生成图像的任务类比成机器翻译——把文字序列"翻译"成图像的离散token序列，借此直接复用大语言模型领域成熟的扩展经验：先用基于Transformer的图像分词器ViT-VQGAN把图片压缩成离散token，再用编码器-解码器Transformer学习从文字token到图像token的映射，最终将模型规模扩展到200亿参数。在标准基准MS-COCO上，Parti取得了零样本FID 7.23、微调FID 3.22的当时最优成绩，同时在更长描述的Localized Narratives数据集上也表现优异。研究团队还发布了包含1600多条提示词的综合评测集PartiPrompts，覆盖12个类别和11个难度维度，这项工作证明自回归模型在文本生成图像领域同样是强有力的竞争者，为该方向的规模化扩展路径提供了有力支撑。
原文 arXiv:2206.10789;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2206.10789v1