Title: Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Bias Show Chinese title

Title: 对抗学习偏差的大规模属性图的节点表示学习的标签去卷积

Abstract: Node representation learning on attributed graphs -- whose nodes are associated with rich attributes (e.g., texts and protein sequences) -- plays a crucial role in many important downstream tasks. To encode the attributes and graph structures simultaneously, recent studies integrate pre-trained models with graph neural networks (GNNs), where pre-trained models serve as node encoders (NEs) to encode the attributes. As jointly training large NEs and GNNs on large-scale graphs suffers from severe scalability issues, many methods propose to train NEs and GNNs separately. Consequently, they do not take feature convolutions in GNNs into consideration in the training phase of NEs, leading to a significant learning bias relative to the joint training. To address this challenge, we propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias by a novel and highly scalable approximation to the inverse mapping of GNNs. The inverse mapping leads to an objective function that is equivalent to that by the joint training, while it can effectively incorporate GNNs in the training phase of NEs against the learning bias. More importantly, we show that LD converges to the optimal objective function values by the joint training under mild assumptions. Experiments demonstrate LD significantly outperforms state-of-the-art methods on Open Graph Benchmark datasets. Show Chinese abstract

Abstract: 属性图上的节点表示学习——其中的节点与丰富的属性（例如文本和蛋白质序列）相关联——在许多重要的下游任务中起着至关重要的作用。 为了同时编码属性和图结构，近期的研究将预训练模型与图神经网络（GNNs）相结合，其中预训练模型作为节点编码器（NEs）来编码属性。 由于在大规模图上联合训练大型NEs和GNNs存在严重的可扩展性问题，许多方法提出分别训练NEs和GNNs。 因此，在NEs的训练阶段没有考虑GNNs中的特征卷积，导致相对于联合训练有显著的学习偏差。 为了解决这一挑战，我们提出了一种高效的标签正则化技术，即标签解卷积（LD），通过一种新颖且高度可扩展的GNN逆映射近似来减轻学习偏差。 逆映射导致的目标函数等效于联合训练的结果，同时可以在NEs的训练阶段有效结合GNNs以对抗学习偏差。 更重要的是，我们证明了在轻度假设下，LD收敛到联合训练下的最优目标函数值。 实验表明，LD在Open Graph Benchmark数据集上显著优于最先进的方法。