The main goal of reading paper is not just understanding it. Try to understand the key concept, but we need to get new ideas and research directions from the paper.
Paper information
- title: GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
- authors: Qiu, J., Chen, Q., Dong, Y., Zhang, J., Yang, H., Ding, M., ... & Tang, J.
- venue: KDD`20
- pdf link: pdf link
- github: github link
- abstract:
Graph representation learning has emerged as a powerful technique for addressing real-world problems. Various downstream graph learning tasks have benefited from its recent developments, such as node classification, similarity search, and graph classification. However, prior arts on graph representation learning focus on do- main specific problems and train a dedicated model for each graph dataset, which is usually non-transferable to out-of-domain data. In- spired by the recent advances in pre-training from natural language processing and computer vision, we design Graph Contrastive Cod- ing (GCC)1—a self-supervised graph neural network pre-training framework—to capture the universal network topological proper- ties across multiple networks. We design GCC’s pre-training task as subgraph instance discrimination in and across networks and leverage contrastive learning to empower graph neural networks to learn the intrinsic and transferable structural representations. We conduct extensive experiments on three graph learning tasks and ten graph datasets. The results show that GCC pre-trained on a collection of diverse datasets can achieve competitive or better performance to its task-specific and trained-from-scratch counter- parts. This suggests that the pre-training and fine-tuning paradigm presents great potential for graph representation learning.
Summary: problems to address, key ideas, quick results
presentation link
Questions about the paper?
What do you like?
- They apply contrastive learning method on graph pre training task. In this way, they can get different views of a graph by utilizing the randomness of sampling methods.
-The writing is good and easy to follow.
What you don't like?
How to improve?
Any new ideas?
Similar to the mask LM in Bert, we can adopt the generative method to predict the target node with the surrounding nodes by masking the features of the nodes, or mask some edges of the subgraph, and then reconstruct the subgraph with the model
Reproducing results (if any)
The main goal of reading paper is not just understanding it. Try to understand the key concept, but we need to get new ideas and research directions from the paper.
Paper information
Graph representation learning has emerged as a powerful technique for addressing real-world problems. Various downstream graph learning tasks have benefited from its recent developments, such as node classification, similarity search, and graph classification. However, prior arts on graph representation learning focus on do- main specific problems and train a dedicated model for each graph dataset, which is usually non-transferable to out-of-domain data. In- spired by the recent advances in pre-training from natural language processing and computer vision, we design Graph Contrastive Cod- ing (GCC)1—a self-supervised graph neural network pre-training framework—to capture the universal network topological proper- ties across multiple networks. We design GCC’s pre-training task as subgraph instance discrimination in and across networks and leverage contrastive learning to empower graph neural networks to learn the intrinsic and transferable structural representations. We conduct extensive experiments on three graph learning tasks and ten graph datasets. The results show that GCC pre-trained on a collection of diverse datasets can achieve competitive or better performance to its task-specific and trained-from-scratch counter- parts. This suggests that the pre-training and fine-tuning paradigm presents great potential for graph representation learning.
Summary: problems to address, key ideas, quick results
presentation link
Questions about the paper?
What do you like?
-The writing is good and easy to follow.
What you don't like?
How to improve?
Any new ideas?
Similar to the mask LM in Bert, we can adopt the generative method to predict the target node with the surrounding nodes by masking the features of the nodes, or mask some edges of the subgraph, and then reconstruct the subgraph with the model
Reproducing results (if any)