Merge pull request #234 from FluxML/develop

Correct GCNConv with normalized_adjacency_matrix

Author: yuehhua

Project.toml

@@ -5,13 +5,13 @@ version = "0.7.7"
 
 [deps]
 CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
+ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
 DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
 FillArrays = "1a297f60-69ca-5386-bcde-b61e274b549b"
 Flux = "587475ba-b771-5e3f-ad9e-33799f191a9c"
-GraphLaplacians = "a1251efa-393a-423f-9d7b-faaecba535dc"
 GraphMLDatasets = "21828b05-d3b3-40ad-870e-a4bc2f52d5e8"
 GraphSignals = "3ebe565e-a4b5-49c6-aed2-300248c3a9c1"
-LightGraphs = "093fc24a-ae57-5d10-9952-331d41423f4d"
+Graphs = "86223c79-3864-5bf0-83f7-82e725a168b6"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
 NNlib = "872c559c-99b0-510c-b3b7-b6c96a88d5cd"
 NNlibCUDA = "a00861dc-f156-4864-bf3c-e6376f28a68d"
@@ -21,14 +21,14 @@ Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
 Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
 
 [compat]
-CUDA = "3.3"
+CUDA = "3"
+ChainRulesCore = "1.7"
 DataStructures = "0.18"
-FillArrays = "0.11, 0.12"
+FillArrays = "0.12"
 Flux = "0.12"
-GraphLaplacians = "0.1"
 GraphMLDatasets = "0.1"
-GraphSignals = "0.2"
-LightGraphs = "1.3"
+GraphSignals = "0.3"
+Graphs = "1.4"
 NNlib = "0.7"
 NNlibCUDA = "0.1"
 Reexport = "1.1"

README.md

@@ -34,7 +34,7 @@ GeometricFlux handles graph data (the topology plus node/vertex/graph features)
 thanks to the type `FeaturedGraph`.
 
 A `FeaturedGraph` can be constructed out of 
-adjacency matrices, adjacency lists, LightGraphs' types...
+adjacency matrices, adjacency lists, Graphs' types...
 
 ```julia
 fg = FeaturedGraph(adj_list)   

docs/make.jl

@@ -13,7 +13,8 @@ makedocs(
     pages = ["Home" => "index.md",
              "Get started" => "start.md",
              "Basics" =>
-               ["Building layers" => "basics/layers.md",
+               ["Graph convolutions" => "basics/conv.md",
+                "Building layers" => "basics/layers.md",
                 "Graph passing" => "basics/passgraph.md"],
              "Cooperate with Flux layers" => "cooperate.md",
              "Abstractions" =>

docs/src/basics/conv.md

@@ -0,0 +1,3 @@
+# Graph convolutions
+
+Graph convolution can be classified into spectral-based graph convolution and spatial-based graph convolution. Spectral-based graph convolution, such as `GCNConv` and `ChebConv`, performs operation on features of *whole* graph at one time. Spatial-based graph convolution, such as `GraphConv` and `GATConv`, performs operation on features of *local* graph instead. Message-passing scheme is an abstraction for spatial-based graph convolutional layers. Any spatial-based graph convolutional layer can be implemented under the framework of message-passing scheme.

docs/src/basics/layers.md

@@ -2,7 +2,7 @@
 
 Building GNN is as simple as building neural network in Flux. The syntax here is the same as Flux. `Chain` is used to stack layers into a GNN. A simple example is shown here:
 
-```
+```julia
 model = Chain(GCNConv(adj_mat, feat=>h1),
               GCNConv(adj_mat, h1=>h2, relu))
 ```
@@ -21,7 +21,6 @@ When using GNN layers, the general guidelines are:
 * If you pass in a ``n \times d`` matrix of node features, and the layer maps node features ``\mathbb{R}^d \rightarrow \mathbb{R}^k`` then the output will be in matrix with dimensions ``n \times k``. The same ostensibly goes for edge features but as of now no layer type supports outputting new edge features.
 * If you pass in a `FeaturedGraph`, the output will be also be a `FeaturedGraph` with modified node (and/or edge) features. Add `node_feature` as the following entry in the Flux chain (or simply call `node_feature()` on the output) if you wish to subsequently convert them to matrix form.
 
-
-## Customize layers
+## Create custom layers
 
 Customizing your own GNN layers are the same as customizing layers in Flux. You may want to reference [Flux documentation](https://fluxml.ai/Flux.jl/stable/models/basics/#Building-Layers-1).

docs/src/basics/passgraph.md

@@ -10,7 +10,7 @@ A static graph is used to reduce redundant computation during passing through la
 GCNConv(adj_mat, feat=>h1, relu)
 ```
 
-`Simple(Di)Graph`, `SimpleWeighted(Di)Graph` or `Meta(Di)Graph` provided by the packages LightGraphs, SimpleWeightedGraphs and MetaGraphs, respectively, are valid arguments for passing as a static graph to this layer. An adjacency list is also accepted in the type of `Vector{Vector}` is also accepted.
+`Simple(Di)Graph`, `SimpleWeighted(Di)Graph` or `Meta(Di)Graph` provided by the packages Graphs, SimpleWeightedGraphs and MetaGraphs, respectively, are valid arguments for passing as a static graph to this layer. An adjacency list is also accepted in the type of `Vector{Vector}` is also accepted.
 
 ## Variable graph
 
@@ -20,7 +20,7 @@ Variable graphs are supported through `FeaturedGraph`, which contains both the g
 FeaturedGraph(adj_mat, features)
 ```
 
-`Simple(Di)Graph`, `SimpleWeighted(Di)Graph` or `Meta(Di)Graph` provided by the packages LightGraphs, SimpleWeightedGraphs and MetaGraphs, respectively, are acceptable for constructing a `FeaturedGraph`. An adjacency list is also accepted, too.
+`Simple(Di)Graph`, `SimpleWeighted(Di)Graph` or `Meta(Di)Graph` provided by the packages Graphs, SimpleWeightedGraphs and MetaGraphs, respectively, are acceptable for constructing a `FeaturedGraph`. An adjacency list is also accepted, too.
 
 ## Cached graph in layers
 

examples/gae.jl

@@ -6,8 +6,7 @@ using Flux: @epochs
 using JLD2
 using Statistics: mean
 using SparseArrays
-using LightGraphs.SimpleGraphs
-using LightGraphs: adjacency_matrix
+using Graphs.SimpleGraphs
 using CUDA
 
 @load "data/cora_features.jld2" features
@@ -21,13 +20,13 @@ target_catg = 7
 epochs = 200
 
 ## Preprocessing data
-adj_mat = Matrix{Float32}(adjacency_matrix(g)) |> gpu
+fg = FeaturedGraph(g) |> gpu
 train_X = Matrix{Float32}(features) |> gpu  # dim: num_features * num_nodes
-train_y = adj_mat  # dim: num_nodes * num_nodes
+train_y = fg  # dim: num_nodes * num_nodes
 
 ## Model
-encoder = Chain(GCNConv(adj_mat, num_features=>hidden1, relu),
-                GCNConv(adj_mat, hidden1=>hidden2))
+encoder = Chain(GCNConv(fg, num_features=>hidden1, relu),
+                GCNConv(fg, hidden1=>hidden2))
 model = Chain(GAE(encoder, σ)) |> gpu
 
 ## Loss

examples/gat.jl

@@ -6,8 +6,8 @@ using Flux.Data: DataLoader
 using JLD2
 using Statistics: mean
 using SparseArrays
-using LightGraphs.SimpleGraphs
-using LightGraphs: adjacency_matrix
+using Graphs.SimpleGraphs
+using Graphs: adjacency_matrix
 using CUDA
 
 @load "data/cora_features.jld2" features

examples/gcn.jl

@@ -5,8 +5,7 @@ using Flux: @epochs
 using JLD2
 using Statistics
 using SparseArrays
-using LightGraphs.SimpleGraphs
-using LightGraphs: adjacency_matrix
+using Graphs.SimpleGraphs
 using CUDA
 using Random
 
@@ -25,12 +24,12 @@ epochs = 100
 ## Preprocessing data
 train_X = Matrix{Float32}(features) |> gpu  # dim: num_features * num_nodes
 train_y = Matrix{Float32}(labels) |> gpu  # dim: target_catg * num_nodes
-adj_mat = Matrix{Float32}(adjacency_matrix(g)) |> gpu
+fg = FeaturedGraph(g) |> gpu
 
 ## Model
-model = Chain(GCNConv(adj_mat, num_features=>hidden, relu),
+model = Chain(GCNConv(fg, num_features=>hidden, relu),
               Dropout(0.5),
-              GCNConv(adj_mat, hidden=>target_catg),
+              GCNConv(fg, hidden=>target_catg),
               ) |> gpu
 
 ## Loss

examples/gde.jl

@@ -2,7 +2,6 @@ using GeometricFlux, Flux, JLD2, SparseArrays, DiffEqFlux, DifferentialEquations
 using Flux: onehotbatch, onecold, logitcrossentropy, throttle
 using Flux: @epochs
 using Statistics: mean
-using LightGraphs: adjacency_matrix
 
 # Load the dataset
 @load "data/cora_features.jld2" features
@@ -19,22 +18,22 @@ epochs = 40
 # Preprocess the data and compute adjacency matrix
 train_X = Matrix{Float32}(features)  # dim: num_features * num_nodes
 train_y = Float32.(labels)  # dim: target_catg * num_nodes
-adj_mat = Matrix{Float32}(adjacency_matrix(g))
+fg = FeaturedGraph(g)
 
 # Define the Neural GDE
 diffeqarray_to_array(x) = reshape(cpu(x), size(x)[1:2])
 
 node = NeuralODE(
-    GCNConv(adj_mat, hidden=>hidden),
+    GCNConv(fg, hidden=>hidden),
     (0.f0, 1.f0), Tsit5(), save_everystep = false,
     reltol = 1e-3, abstol = 1e-3, save_start = false
 )
 
-model = Chain(GCNConv(adj_mat, num_features=>hidden, relu),
+model = Chain(GCNConv(fg, num_features=>hidden, relu),
               Dropout(0.5),
               node,
               diffeqarray_to_array,
-              GCNConv(adj_mat, hidden=>target_catg),
+              GCNConv(fg, hidden=>target_catg),
               softmax)
 
 # Loss