Graph Neural Architecture Search with Large Language Models
Haishuai Wang Yang Gao Xin Zheng Peng Zhang Jiajun Bu Philip S. Yu Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems, College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China School of Public Health, Zhejiang University, Hangzhou 310058, China Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou 510006, China Department of Computer Science, University of Illinois at Chicago, Chicago 60607, USA
Abstract
Graph Neural Architecture Search (GNAS) has shown promising results in finding the best graph neural network architecture on a given graph dataset. However, existing GNAS methods still require intensive human labor and rich domain knowledge when designing the search space and search strategy. To this end, we integrate Large Language Models (LLMs) into GNAS and present a new GNAS model based on LLMs (GNAS-LLM for short). The basic idea of GNAS-LLM is to design a new class of GNAS prompts for LLMs to guide LLMs towards understanding the generative task of graph neural architectures. The prompts consist of descriptions of the search space, search strategy, and search feedback of GNAS. By iteratively running LLMs with the prompts, GNAS-LLM generates more accurate graph neural network architectures with fast convergence. Experimental results show that GNAS-LLM outperforms the state-of-the-art GNAS methods on four benchmark graph datasets, with an average improvement of 0.7% on the validation sets and 0.3% on the test sets. Besides, GNAS-LLM achieves an average improvement of 1.0% on the test sets based on the search space from AutoGEL.
中文速览
自动搜索图神经网络（Graph Neural Network, GNN）最优架构一直需要大量人工设计搜索空间和搜索策略，门槛很高。研究者将大语言模型（Large Language Model, LLM）引入图神经架构搜索（Graph Neural Architecture Search, GNAS），提出了 GNAS-LLM 方法，核心做法是设计一套专用提示词（prompt），把搜索空间（用邻接矩阵描述候选操作及其连接方式）、搜索策略（探索与利用的强化学习描述）以及历次评估反馈一并告知 LLM，让 LLM 充当搜索控制器，迭代生成越来越好的 GNN 架构。在四个基准图数据集上的实验表明，GNAS-LLM 的验证集平均精度比现有最优方法提升 0.7%、测试集提升 0.3%，同时平均减少约 56% 的搜索迭代次数。这项工作首次将 LLM 的生成能力与图神经架构搜索深度结合，大幅降低了人工设计的负担，也为后续自动化图学习研究提供了新的范式。
原文 arXiv:2310.01436;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2310.01436v3