Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced Training
Chuxiong Sun Emerging Technology Research Division China Telecom Research Institute Beijing, China、Hongming Gu Emerging Technology Research Division China Telecom Research Institute Beijing, China Jie Hu Emerging Technology Research Division China Telecom Research Institute Beijing, China
Abstract
Besides of the existing neighbor sampling techniques applied on common Graph Neural Networks (GNNs), scalable methods allowing normal minibatch training can more easily scale to large scaled graphs. They decouple graph convolutions and other learnable transformations into preprocessing and a scalable classifier. A complex and graph structure-aware classifier is important to achieve competitive performances. By replacing redundant concatenation operation in Scalable Inception Graph Neural Networks (SIGN) with a more graph structure-aware attention mechanism, we propose Scalable and Adaptive Graph Neural Networks (SAGN). SAGN can adaptively gather neighborhood information among different hops. To further improve scalable GNNs by introducing the existing techniques applied on common GNNs for semi-supervised learning tasks, we propose Self-Label-Enhanced (SLE) training approach combining the self-training approach and label propagation in depth. We add the base model with a scalable label model. Then we iteratively train models and enhance the training set in several stages. To generate input of the label model, we apply label propagation based on one-hot encoded label vectors without
中文速览
可扩展图神经网络在大规模图上往往因固定的预处理卷积而损失表达能力，且如何将自训练和标签传播这两类半监督技巧引入其中也缺乏系统研究。针对这两个问题，作者提出了 SAGN（Scalable and Adaptive Graph Neural Networks），用节点自适应的注意力机制替换 SIGN 中粗糙的拼接操作，让模型能为不同节点动态权衡不同跳距的邻域信息；同时设计了 SLE（Self-Label-Enhanced）训练框架，将自训练与标签传播深度融合——发现只要传播步数足够多，无需对训练集做随机掩码也能有效避免标签泄漏，并让每阶段生成的高置信度伪标签参与下一轮标签传播，从而把模型知识扩散到整张图。在归纳式和直推式多个数据集上的实验表明，SAGN 单独使用已优于或持平于各类采样与非采样基线，叠加 SLE 后性能进一步提升最高达 8.4%，并在 ogbn-products 和 ogbn-papers100M 两个超大规模基准上达到当时最优水平，证明该方案兼顾了效率、可扩展性与准确性。
原文 arXiv:2104.09376;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2104.09376v3