Holistic Graph Neural Networks based on a global-based attention mechanism

• Published in: Knowledge-based systems 2022-03, Vol.240, p.108105, Article 108105
• Main Authors: Rassil, Asmaa, Chougrad, Hiba, Zouaki, Hamid
• Format: Article
• Language: English
• Subjects: Bioinformatics; Datasets; Feature extraction; Global pooling; Graph classification; Graph Neural Networks; Message passing; Neighborhoods; Neural networks; Node representation; Nodes; Recursive functions; Social networks; Structured data
• ISSN: 0950-7051; 1872-7409
• DOI: 10.1016/j.knosys.2021.108105

Graph Neural Networks (GNNs) have become increasingly popular due to their impressive capacity to perform classification or regression on high-dimensional graph-structured data. However, standard message passing GNNs typically define nodes embeddings through a recursive neighborhood aggregation process which updates the representation vector of each node with reference to its neighborhood only. In this paper, we propose the Holistic Graph Neural Network (HGNN), a two-fold architecture which introduces a global-based attention mechanism for learning and generating nodes embeddings. The global features we inject, summarize the overall global behavior of the graph in addition to the local semantic and structural information. These global features will make each individual node aware of the global behavior of the graph outside the borders of the local neighborhood. We further propose a variant of the HGNN, we call HGNNα based on a more sophisticated hierarchical global-feature extraction mechanism. We explore diverse global pooling strategies to derive highly expressive global features. We also show that state-of-the-art GNNs can significantly benefit from the addition of the global-based attention introduced. Furthermore, we prove the efficiency of the HGNN model theoretically and adapt it to support graph data which carries edge attributes for example the Molecular datasets from the Open Graph Benchmark. Experiments on Bioinformatics datasets, Social Networks and Molecular datasets demonstrate that our proposed models achieve much better performance than state-of-the-art methods, for instance we achieved improvements of +11% on COLLAB and +13% on IMDB-BINARY datasets. •We propose the Holistic Graph Neural Network (HGNN) with a global-based attention mechanism.•We also propose the Alpha Holistic Graph Neural Network a variant based on a hierarchical weighted aggregation mechanism.•We test the proposed architecture on several benchmarks and achieve remarkable results.•We investigate the best global graph pooling strategy which would enable our proposed architecture to gain in time and efficiency.•We further demonstrate that other GNNs can greatly benefit from the addition of the global-based attention.