Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model
Ruoxi Shi1 Hansheng Chen2 Zhuoyang Zhang3 Minghua Liu1 Chao Xu4 Xinyue Wei1 Linghao Chen5 Chong Zeng5 Hao Su1 1UC San Diego 2Stanford University 3Tsinghua University 4UCLA 5Zhejiang University
Abstract
We report Zero123++, an image-conditioned diffusion model for generating 3D-consistent multi-view images from a single input view. To take full advantage of pretrained 2D generative priors, we develop various conditioning and training schemes to minimize the effort of finetuning from off-the-shelf image diffusion models such as StableDiffusion. Zero123++ excels in producing high-quality, consistent multi-view images from a single image, overcoming common issues like texture degradation and geometric misalignment. Furthermore, we showcase the feasibility of training a ControlNet on Zero123++ for enhanced control over the generation process. The code is available at https://github.com/SUDO-AI-3D/zero123plus.
中文速览
从单张图片还原出物体多角度外观，是3D内容生成的核心难题，此前方法要么各视角独立生成导致几何不一致，要么因训练分辨率偏低损失细节。Zero123++通过将六个视角的图像拼成一张图一起生成，从根本上保证了多视角之间的联合一致性；同时针对噪声调度、局部参考注意力（Reference Attention）和全局CLIP图像嵌入等关键环节进行精细设计，最大程度复用了Stable Diffusion的预训练先验，只需轻量微调。在Objaverse验证集上，Zero123++的LPIPS指标优于Zero-1-to-3 XL和SyncDreamer，且能泛化到真实照片、AI生成图乃至二次元插画；此外还展示了基于ControlNet的深度控制版本，进一步将LPIPS提升至0.086。这项工作为后续单图转3D网格的流程提供了更可靠的多视角基础，对整个3D内容生成社区具有重要的基础模型价值。
原文 arXiv:2310.15110;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2310.15110v1