Accompanied repositories for our ICML 2024 spotlight paper Position:Graph Foundation Models Are Already Here
- (BioRxiv '22)[Protein]Language models of protein sequences at the scale of evolution enable accurate structure prediction [Paper]
- (Arxiv '24)[Molecule]A foundation model for atomistic materials chemistry [Paper]
- (ICLR '24)[Molecule] From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction [Paper]
- (Arxiv '23)[Molecule]Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learning [Paper]
- (Arxiv '23)[Molecule] DPA-2: Towards a universal large atomic model for molecular and material simulation [Paper]
- (Arxiv '24)[Molecule]MatterSim: A Deep Learning Atomistic Model Across Elements, Temperatures and Pressures[Paper]
- (ICLR '23)[Molecule]Mole-BERT: Rethinking Pre-training Graph Neural Networks for Molecules [Paper]
- (Arxiv '24)[Molecule]On the Scalability of GNNs for Molecular Graphs [Paper]
- (Arxiv '24)[Molecule]MiniMol: A Parameter-Efficient Foundation Model for Molecular Learning [Paper]
- (ICML '24)[Molecule] A Graph is Worth K Words: Euclideanizing Graph using Pure Transformer [Paper]
- (Arxiv '24)[Knowledge Graph Reasoning] Zero-shot Logical Query Reasoning on any Knowledge Graph [Paper]
- (ICLR '24)[Knowledge Graph Reasoning] Towards Foundation Models for Knowledge Graph Reasoning [Paper]
- (Arxiv '24)[Graph Reasoning] Let Your Graph Do the Talking: Encoding Structured Data for LLMs [Paper]
- (Arxiv '24)[Node Classification] GraphFM: A Scalable Framework for Multi-Graph Pretraining [Paper]
- (Arxiv '24)[Node Classification] GraphAny: A Foundation Model for Node Classification on Any Graph [Paper]
- (Arxiv '23)[Node Classification] GraphText: Graph Reasoning in Text Space [Paper]
- (LoG '22) [Algorithm Reasoning]A Generalist Neural Algorithmic Learner [Paper]
- (Arxiv '24) GOFA: A Generative One-For-All Model for Joint Graph Language Modeling [Paper]
- (Arxiv '24) Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models [Paper]
- (ICLR '24) One For All: Towards Training One Graph Model For All Classification Tasks [Paper]
- (NeurIPS '23) PRODIGY: Enabling In-context Learning Over Graphs [Paper]
- (Arxiv '24) Towards Neural Scaling Laws for Foundation Models on Temporal Graphs [Paper]
- (Arxiv '24) Neural Scaling Laws on Graphs [Paper]
- (NIPS '23) Uncovering neural scaling laws in molecular representation learning [Paper]
- (NMI '23) Neural scaling of deep chemical models [Paper]
We put a collection of large-scale real-world datasets below.
| Name | URL | Description |
|---|---|---|
| TU-Dataset | https://chrsmrrs.github.io/datasets/ | A collection of graph-level prediction datasets |
| NetworkRepository | https://networkrepository.com/ | The largest graph datasets, with graphs coming from 30+ different domains |
| Open Graph Benchmark | https://ogb.stanford.edu/ | Contains a bunch of large-scale graph datasets |
| Pyg | https://pytorch-geometric.readthedocs.io/en/latest/modules/datasets.html | Official datasets provided by Pyg, containing popular datasets for benchmark |
| SNAP | https://snap.stanford.edu/data/ | Mainly focus on social network |
| Aminer | https://www.aminer.cn/data/ | A collection of academic graphs |
| OAG | https://www.aminer.cn/open-academic-graph | A large-scale academic graph |
| MalNet | https://www.mal-net.org/#home | A large-scale function calling graph for malware detection |
| ScholKG | https://scholkg.kmi.open.ac.uk/ | A large-scale scholarly knowledge graph |
| Graphium | https://github.com/datamol-io/graphium | A massive dataset for molecular property prediction |
| Live Graph Lab | https://livegraphlab.github.io/ | A large-scale temporal graph for NFT transactions |
| Temporal Graph Benchmark | https://docs.tgb.complexdatalab.com/ | A large-scale benchmark for temporal graph learning |
| MoleculeNet | https://moleculenet.org/ | A benchmark for molecular machine learning |
| Recsys data | https://cseweb.ucsd.edu/~jmcauley/datasets.html | A collection of datasets for recommender systems |
| LINKX | https://github.com/CUAI/Non-Homophily-Large-Scale | A collection of large-scale non-homophilous graphs |
| GADBench | https://github.com/squareRoot3/GADBench | A collection of datasets for graph-based anomaly detection |
| Text-attributed datasets | https://github.com/CurryTang/TSGFM | A collection of text-attributed graph datasets |
| TAGLAS | https://arxiv.org/abs/2406.14683 | A collection of text-attributed graph datasets |
A curated list of papers grounding the theoretical foundations of GFMs
- Geometric ML Book Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [Book]
- (Arxiv '24) Understanding Transformer Reasoning Capabilities via Graph Algorithms [Paper]
- (Arxiv '24) Future Directions in Foundations of Graph Machine Learning [Paper]
- (ICML '23) On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology [Paper]
- (ICLR '22) Understanding over-squashing and bottlenecks on graphs via curvature [Paper]
- (ICLR '21) HOW NEURAL NETWORKS EXTRAPOLATE: FROM FEEDFORWARD TO GRAPH NEURAL NETWORKS [Paper]
- (ICMLW '20) A Note on Over-Smoothing for Graph Neural Networks [Paper]
- (ICLR '20) WHAT CAN NEURAL NETWORKS REASON ABOUT? [Paper]
- (ICLR '20) The Logical Expressiveness of Graph Neural Networks [Paper]
- (Arxiv '24) Understanding Heterophily for Graph Neural Networks [Paper]
- (NeurIPS '23) Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All? [Paper]
- (NeurIPS '23) When Do Graph Neural Networks Help with Node Classification? Investigating the Impact of Homophily Principle on Node Distinguishability [Paper]
- (NIPS '23) Demystifying Oversmoothing in Attention-Based Graph Neural Networks [Paper]
- (ICLR '23) Graph domain adaptation via theory-grounded spectral regularization [Paper]
- (NIPS '22) Revisiting Heterophily For Graph Neural Networks [Paper]
- (NIPS '22) Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum [Paper]
- (ICLR '22) Is Homophily a Necessity for Graph Neural Networks? [Paper]
- (ICLR '20) Graph Neural Networks Exponentially Lose Expressive Power for Node Classification [Paper]
- (Annual Review of Sociology '01) Birds of a Feather: Homophily in Social Networks [Paper]
- (ICLR '24) Revisiting link prediction a data perspective [Paper]
- (NeurIPS '23) A Theory of Link Prediction via Relational Weisfeiler-Leman on Knowledge Graphs [Paper]
- (NIPSW '23) A Multi-Task Perspective for Link Prediction with New Relation Types and Nodes [Paper]
- (NIPSW '23) Double Equivariance for Inductive Link Prediction for Both New Nodes and New Relation Types [Paper]
- (ACL '23) Are Message Passing Neural Networks Really Helpful for Knowledge Graph Completion? [Paper]
- (TMLR '23) You Only Transfer What You Share: Intersection-Induced Graph Transfer Learning for Link Prediction [Paper]
- (ICLR '22) Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks [Paper]
- (NeurIPS '21) Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning [Paper]
- (ICLR '20) On the Equivalence between Positional Node Embeddings and Structural Graph Representations [Paper]
- (Scientific Reports '19) Structural transition in social networks: The role of homophily [Paper]
- (ICML '19) Position-aware Graph Neural Networks [Paper]
- (NeurIPS '18) Link Prediction Based on Graph Neural Networks [Paper]
- (TKDE '15) Triadic closure pattern analysis and prediction in social networks [Paper]
- (KDD '02) SimRank: A Measure of Structural-Context Similarity [Paper]
- (Social Networks '03) Friends and neighbors on the Web [Paper]
- (Psychometrika '53) A new status index derived from sociometric analysis [Paper]
- (ICLR '24) On the Stability of Expressive Positional Encodings for Graph Neural Networks [Paper]
- (ICLR '24) Beyond weisfeiler-lehman: A quantitative framework for gnn expressiveness [Paper]
- (NeurIPS '23) Fine-grained Expressivity of Graph Neural Networks [Paper]
- (JMLR '23) Weisfeiler and Leman go Machine Learning: The Story so far [Paper]
- (ICML '23) WL meet VC [Paper]
- (ICLR '23) Sign and Basis Invariant Networks for Spectral Graph Representation Learning [Paper]
- (ICLR '23) Rethinking the Expressive Power of GNNs via Graph Biconnectivity [Paper]
- (NeurIPS '22) Tree Mover’s Distance: Bridging Graph Metrics and Stability of Graph Neural Networks [Paper]
- (NeurIPS '21) Rethinking Graph Transformers with Spectral Attention [Paper]
- (Physics Reports '20) Networks beyond pairwise interactions: structure and dynamics [Paper]
- (Science '16) Higher-order organization of complex networks [Paper]
- (JMLR '10) Graph Kernels [Paper]
- (Science '02) Network Motifs: Simple Building Blocks of Complex Networks [Paper]
- (Arxiv '23) Graph Prompt Learning: A Comprehensive Survey and Beyond [Paper]
- (NIPS '23) Universal Prompt Tuning for Graph Neural Networks [Paper]
- (KDD '23) All in One: Multi-Task Prompting for Graph Neural Networks [Paper]
- (WWW '23) GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks [Paper]
- (KDD '22) GPPT: Graph Pre-training and Prompt Tuning to Generalize Graph Neural Networks [Paper]
- (ICMLW '24) Zero-Shot Generalization of GNNs over Distinct Attribute Domains [Paper]
- (Arxiv '24) GraphAlign: Pretraining One Graph Neural Network on Multiple Graphs via Feature Alignment [Paper]
- (Arxiv '24) Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights [Paper]
- (ICLR '24) One For All: Towards Training One Graph Model For All Classification Tasks [Paper]
- (Arxiv '23) GraphText: Graph Reasoning in Text Space [Paper]
- (ICML '23) GRAFENNE: Learning on Graphs with Heterogeneous and Dynamic Feature Sets [Paper]
- (CVPR '23) Deep graph reprogramming [Paper]
- (ICLR '23) Confidence-Based Feature Imputation for Graphs with Partially Known Features [Paper]
- (Future Generation Computer Systems '21) Graph convolutional networks for graphs containing missing features [Paper]
- (Arxiv '24) Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models [Paper]
- (NIPS '23) Data-Centric Learning from Unlabeled Graphs with Diffusion Model [Paper]
- (ICLR '23) Digress: Discrete denoising diffusion for graph generation [Paper]
- (ICML '22) Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations [Paper]
- (KDD '22) GraphWorld: Fake Graphs Bring Real Insights for GNNs [Paper]
- (ICML '21) Graphdf: A discrete flow model for molecular graph generation [Paper]
- (ICML '20) Hierarchical Generation of Molecular Graphs using Structural Motifs [Paper]
- (JMLR '10) Kronecker Graphs: An Approach to Modeling Networks [Paper]
- (NIPS '08) Mixed membership stochastic blockmodels [Paper]
- (Social Networks '07) An introduction to exponential random graph (p*) models for social networks [Paper]
- (Reviews of modern physics '02) Statistical mechanics of complex networks [Paper]
A paper list of graph transformers [Awesome graph transformers]
- (TMLR '23) Graph Neural Networks Designed for Different Graph Types: A Survey [Paper]
- (Arxiv '23) GraphGPT: Graph Learning with Generative Pre-trained Transformers [Paper]
- (Arxiv '23) Attending to Graph Transformers [Paper]
- (Arxiv '22) GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction [Paper]
- (Arxiv' 24) Can Large Language Models Improve the Adversarial Robustness of Graph Neural Networks? [Paper]
- (Arxiv '24) STARK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases [Paper]
- (Arxiv '24) G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering [Paper]
- (Arxiv '24) Let Your Graph Do the Talking: Encoding Structured Data for LLMs [Paper]
- (Arxiv '24) InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment [Paper]
- (ICML '24) LLaGA: Large Language and Graph Assistant [Paper]
- (ICLR '24) Talk like a Graph: Encoding Graphs for Large Language Models [Paper]
- (WWW '24) GraphGPT: Graph Instruction Tuning for Large Language Models [Paper]