SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
Dustin Podell、Zion English、Kyle Lacey、Andreas Blattmann、Tim Dockhorn、Jonas Müller、Joe Penna、Robin Rombach
Abstract
We present SDXL, a latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffusion, SDXL leverages a three times larger UNet backbone: The increase of model parameters is mainly due to more attention blocks and a larger cross-attention context as SDXL uses a second text encoder. We design multiple novel conditioning schemes and train SDXL on multiple aspect ratios. We also introduce a refinement model which is used to improve the visual fidelity of samples generated by SDXL using a post-hoc image-to-image technique. We demonstrate that SDXL shows drastically improved performance compared to previous versions of Stable Diffusion and achieves results competitive with those of black-box state-of-the-art image generators. In the spirit of promoting open research and fostering transparency in large model training and evaluation, we provide access to code and model weights.
中文速览
文生图（text-to-image）领域长期被不开源的黑盒模型所主导，Stable Diffusion虽然开放，但生成质量与顶级闭源系统仍有差距。SDXL通过三项核心改进来缩小这一差距：将UNet骨干网络扩大至26亿参数并引入双文本编码器、设计图像尺寸与裁剪参数的微条件机制（micro-conditioning）来消除训练数据引入的伪影、以及训练一个专门的扩散精炼模型（refinement model）对基础模型的输出做二次降噪以提升细节质量。用户研究和定量评估均显示，SDXL大幅超越此前所有Stable Diffusion版本，并在生成质量上达到与主流闭源模型相当的水平。该工作同时开放了代码与模型权重，为社区在高质量开源生成模型上的进一步研究和应用提供了坚实基础。
原文 arXiv:2307.01952;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2307.01952v1