Objaverse-XL: A Universe of 10M+ 3D Objects
Matt Deitke†ψnormal-†absent𝜓{}^{\dagger\psi}start_FLOATSUPERSCRIPT † italic_ψ end_FLOATSUPERSCRIPT, Ruoshi Liuγ𝛾{}^{\gamma}start_FLOATSUPERSCRIPT italic_γ end_FLOATSUPERSCRIPT, Matthew Wallingfordψ𝜓{}^{\psi}start_FLOATSUPERSCRIPT italic_ψ end_FLOATSUPERSCRIPT, Huong Ngoψ𝜓{}^{\psi}start_FLOATSUPERSCRIPT italic_ψ end_FLOATSUPERSCRIPT, Oscar Michel†normal-†{}^{\dagger}start_FLOATSUPERSCRIPT † end_FLOATSUPERSCRIPT, Aditya Kusupatiψ𝜓{}^{\psi}start_FLOATSUPERSCRIPT italic_ψ end_FLOATSUPERSCRIPT, Alan Fanψ𝜓{}^{\psi}start_FLOATSUPERSCRIPT italic_ψ end_FLOATSUPERSCRIPT, Christian Laforteσ𝜎{}^{\sigma}start_FLOATSUPERSCRIPT italic_σ end_FLOATSUPERSCRIPT, Vikram Voletiσ𝜎{}^{\sigma}start_FLOATSUPERSCRIPT italic_σ end_FLOATSUPERSCRIPT, Samir Yitzhak Gadreγ𝛾{}^{\gamma}start_FLOATSUPERSCRIPT italic_γ end_FLOATSUPERSCRIPT, Eli VanderBilt†normal-†{}^{\dagger}start_FLOATSUPERSCRIPT † end_FLOATSUPERSCRIPT, Aniruddha Kembhavi†ψnormal-†absent𝜓{}^{\dagger\psi}start_FLOATSUPERSCRIPT † italic_ψ end_FLOATSUPERSCRIPT, Carl Vondrickγ𝛾{}^{\gamma}start_FLOATSUPERSCRIPT italic_γ end_FLOATSUPERSCRIPT, Georgia Gkioxariδ𝛿{}^{\delta}start_FLOATSUPERSCRIPT italic_δ end_FLOATSUPERSCRIPT, Kiana Ehsani†normal-†{}^{\dagger}start_FLOATSUPERSCRIPT † end_FLOATSUPERSCRIPT, *{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPTLudwig Schmidt†ψℓnormal-†absent𝜓normal-ℓ{}^{\dagger\psi\ell}start_FLOATSUPERSCRIPT † italic_ψ roman_ℓ end_FLOATSUPERSCRIPT, *{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPTAli Farhadiψ𝜓{}^{\psi}start_FLOATSUPERSCRIPT italic_ψ end_FLOATSUPERSCRIPT ††{}^{\dagger}start_FLOATSUPERSCRIPT † end_FLOATSUPERSCRIPTAllen Institute for AI ψ𝜓{}^{\psi}start_FLOATSUPERSCRIPT italic_ψ end_FLOATSUPERSCRIPTUniversity of Washington, Seattle γ𝛾{}^{\gamma}start_FLOATSUPERSCRIPT italic_γ end_FLOATSUPERSCRIPTColumbia University σ𝜎{}^{\sigma}start_FLOATSUPERSCRIPT italic_σ end_FLOATSUPERSCRIPTStability AI δ𝛿{}^{\delta}start_FLOATSUPERSCRIPT italic_δ end_FLOATSUPERSCRIPTCalifornia Institute of Technology ℓℓ{}^{\ell}start_FLOATSUPERSCRIPT roman_ℓ end_FLOATSUPERSCRIPTLAION *{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPTEqual Senior Contribution
Abstract
Natural language processing and 2D vision models have attained remarkable proficiency on many tasks primarily by escalating the scale of training data. However, 3D vision tasks have not seen the same progress, in part due to the challenges of acquiring high-quality 3D data. In this work, we present Objaverse-XL, a dataset of over 10 million 3D objects. Our dataset comprises deduplicated 3D objects from a diverse set of sources, including manually designed objects, photogrammetry scans of landmarks and everyday items, and professional scans of historic and antique artifacts. Representing the largest scale and diversity in the realm of 3D datasets, Objaverse-XL enables significant new possibilities for 3D vision. Our experiments demonstrate the improvements enabled with the scale provided by Objaverse-XL. We show that by training Zero123 on novel view synthesis, utilizing over 100 million multi-view rendered images, we achieve strong zero-shot generalization abilities. We hope that releasing Objaverse-XL will enable further innovations in the field of 3D vision at scale.
中文速览
三维视觉领域长期面临高质量训练数据匮乏的困境，而自然语言处理和二维视觉模型早已凭借海量数据取得突破性进展。为此，研究团队通过网络爬取 GitHub、Sketchfab、Thingiverse、Polycam 及史密森学会等多个来源，构建了名为 Objaverse-XL 的超大规模三维物体数据集，涵盖超过 1000 万个经去重处理的三维模型，规模比此前最大的三维数据集 Objaverse 1.0 大一个数量级，比 ShapeNet 大两个数量级。基于该数据集，他们用逾 1 亿张多视角渲染图像重新训练了新视角合成模型 Zero123（novel view synthesis），在真实感资产、卡通、素描等多种复杂场景下取得了显著更强的零样本泛化能力，且性能随数据规模持续提升、尚无饱和迹象。这一工作为三维生成、重建和理解等任务提供了类似 LAION-5B 之于二维视觉的数据基础，有望推动三维 AI 进入"规模化"新阶段。
原文 arXiv:2307.05663;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2307.05663v1