Open Graph Benchmark: Datasets for Machine Learning on Graphs
Weihua Hu1, Matthias Fey2, Marinka Zitnik3, Yuxiao Dong4, Hongyu Ren1, Bowen Liu5, Michele Catasta1, Jure Leskovec1 1Department of Computer Science, 5Chemistry, Stanford University 2Department of Computer Science, TU Dortmund University 3Department of Biomedical Informatics, Harvard University 4Microsoft Research, Redmond、Steering Committee Regina Barzilay, Peter Battaglia, Yoshua Bengio, Michael Bronstein, Stephan Günnemann, Will Hamilton, Tommi Jaakkola, Stefanie Jegelka, Maximilian Nickel, Chris Re, Le Song, Jian Tang, Max Welling, Rich Zemel
Abstract
We present the Open Graph Benchmark (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale, encompass multiple important graph ML tasks, and cover a diverse range of domains, ranging from social and information networks to biological networks, molecular graphs, source code ASTs, and knowledge graphs. For each dataset, we provide a unified evaluation protocol using meaningful application-specific data splits and evaluation metrics. In addition to building the datasets, we also perform extensive benchmark experiments for each dataset. Our experiments suggest that OGB datasets present significant challenges of scalability to large-scale graphs and out-of-distribution generalization under realistic data splits, indicating fruitful opportunities for future research. Finally, OGB provides an automated end-to-end graph ML pipeline that simplifies and standardizes the process of graph data loading, experimental setup, and model evaluation. OGB will be regularly updated and welcomes inputs from the community. OGB datasets as well as data loaders, evaluation scrip
中文速览
图神经网络（Graph Neural Network）领域长期依赖一些规模极小、划分随机、评测标准混乱的数据集，导致不同论文的结果根本无法公平比较，也看不出模型在真实场景下的泛化能力。为此，作者推出了开放图基准（Open Graph Benchmark，OGB）——一套涵盖社交网络、分子图、知识图谱、源代码等多个领域的大规模真实数据集，规模比现有基准大一到几个数量级，并按节点属性预测、链接预测、图属性预测三类任务提供统一的、贴近实际应用的数据划分和评测指标。实验表明，OGB上的主流模型面临严峻的可扩展性挑战，在基于时间或结构等真实划分下的分布外泛化（out-of-distribution generalization）表现也明显下滑，远未饱和。OGB还配套提供了自动化的数据加载、模型评测脚本和公开排行榜，有望像ImageNet之于计算机视觉那样，为图机器学习社区提供一个公平、可复现的长期竞技平台。
原文 arXiv:2005.00687;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2005.00687v7