Masked Autoencoders for Point Cloud Self-supervised Learning
Yatian Pang Wenxiao Wang Francis E.H. Tay Wei Liu Yonghong Tian Li Yuan Corresponding author223322445511
Abstract
As a promising scheme of self-supervised learning, masked autoencoding has significantly advanced natural language processing and computer vision. Inspired by this, we propose a neat scheme of masked autoencoders for point cloud self-supervised learning, addressing the challenges posed by point cloud’s properties, including leakage of location information and uneven information density. Concretely, we divide the input point cloud into irregular point patches and randomly mask them at a high ratio. Then, a standard Transformer based autoencoder, with an asymmetric design and a shifting mask tokens operation, learns high-level latent features from unmasked point patches, aiming to reconstruct the masked point patches. Extensive experiments show that our approach is efficient during pre-training and generalizes well on various downstream tasks. Specifically, our pre-trained models achieve 85.18% accuracy on ScanObjectNN and 94.04% accuracy on ModelNet40, outperforming all the other self-supervised learning methods. We show with our scheme, a simple architecture entirely based on standard Transformers can surpass dedicated Transformer models from supervised learning. Our approach also
中文速览
三维点云（point cloud）的自监督学习长期面临标注数据不足、模型架构不统一等瓶颈，而在自然语言和图像领域大放异彩的掩码自编码器（masked autoencoder）方案能否迁移过来，却并不简单——点云天然携带位置信息，直接套用会造成位置信息泄露，而且点云各部分的信息密度分布不均匀。针对这些挑战，本文提出 Point-MAE：将输入点云分割成不规则的点块（point patch），以 60%–80% 的高比例随机遮盖，再用一个完全由标准 Transformer 构成的非对称编解码器来学习未遮盖部分的高层特征并重建被遮盖部分；关键设计是把掩码标记（mask token）从编码器输入端后移至解码器输入端，从而延迟位置信息暴露、同时大幅降低计算开销。实验结果表明，Point-MAE 在真实场景数据集 ScanObjectNN 上达到 85.18% 分类精度、在 ModelNet40 上达到 94.04%，不仅超越所有其他自监督方法，甚至优于专门为点云设计的监督学习 Transformer 模型，少样本分类精度也提升了 1.5%–2.3%；这项工作还表明，语言和图像领域的统一架构范式可以平滑地推广到点云模态，为跨模态统一表示学习提供了重要参考。
原文 arXiv:2203.06604;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2203.06604v2