Zero-Shot Learning on 3D Point Cloud Objects and Beyond
Ali Cheraghian Shafin Rahman Townim F. Chowdhury Dylan Campbell Lars Petersson
Abstract
Zero-shot learning, the task of learning to recognize new classes not seen during training, has received considerable attention in the case of 2D image classification. However, despite the increasing ubiquity of 3D sensors, the corresponding 3D point cloud classification problem has not been meaningfully explored and introduces new challenges. In this paper, we identify some of the challenges and apply 2D Zero-Shot Learning (ZSL) methods in the 3D domain to analyze the performance of existing models. Then, we propose a novel approach to address the issues specific to 3D ZSL. We first present an inductive ZSL process and then extend it to the transductive ZSL and Generalized ZSL (GZSL) settings for 3D point cloud classification. To this end, a novel loss function is developed that simultaneously aligns seen semantics with point cloud features and takes advantage of unlabeled test data to address some known issues (e.g., the problems of domain adaptation, hubness, and data bias). While designed for the particularities of 3D point cloud classification, the method is shown to also be applicable to the more common use-case of 2D image classification. An extensive set of experiments is c
中文速览
让机器识别训练时从未见过的三维物体类别（零样本学习，Zero-Shot Learning），是一个比二维图像更棘手的挑战：现有三维点云（3D point cloud）的预训练特征质量差、类别簇边界模糊，还会额外放大"枢纽问题"（hubness，少数类别被反复错误预测）和"域偏移"（domain shift，模型泛化到新类时偏向已见类别）这两大顽疾。针对这些问题，作者提出了一套新方法：先用归纳式（inductive）框架建立基线，再引入直推式（transductive）学习，设计了一种无需标签即可在测试数据上运行的三元组损失（triplet loss）来缩小类内距离、拉开类间距离，并额外引入一个基于批次预测偏度的损失项专门压制枢纽问题，同时用无偏距离损失平衡已见与未见类别的得分。在合成数据集 ModelNet40/10、McGill 和真实扫描数据集 ScanObjectNN 上，该方法在零样本及广义零样本（GZSL）分类任务上均刷新了最优结果，且同样适用于二维图像分类场景。这项工作填补了三维点云零样本识别的研究空白，为低资源三维感知系统提供了实用的技术路径。
原文 arXiv:2104.04980;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2104.04980v1