GML Express: Graph ML in Industry Workshop, Geometric Deep Learning, and New Software.

"The real voyage of discovery consists not in seeking new lands but seeing with new eyes." Marcel Proust

Hello!

I’m happy to cheer you again after the summer 🌞 Hopefully, you enjoyed the sun and rested well for the next academic season as it promises quite a packed agenda to explore. In today’s episode, we will look back at the recent talks, latest trends, and upcoming events. Let’s go!

---

Videos 👀

Geometric Deep Learning @ML Street Talk

Michael Bronstein, Petar Veličković, Taco Cohen and Joan Bruna are special guests in the new 3.5 hours episode of ML Street Talk talking Geometric DL and explaining the concepts covered in their recent book and pretty much all the current state of the art in the field.

Graph Learning Workshop

A workshop organized by Stanford on the latest advances in Graph Neural Networks. The program includes applications, frameworks, and industry panels on the challenges of graph-based machine learning models.

GDL Course

A course on Geometric Deep Learning, which closely follows the contents of the GDL proto-book. It contains 12 lectures, 2 tutorials, and 4 seminars coverings topics such as graphs, grids, geodesics, and more.

GNN User Group Videos

July’s video from a regular meeting of the GNN user group includes a discussion of the new features of DGL library and two research talks on storing node feature for large graphs and locally private GNNs.

Interpretable Deep Learning for New Physics Discovery

In this video, Miles Cranmer discusses a novel method for converting a neural network into an analytic equation using a particular set of inductive biases. With it, they discover gravity along with planetary masses from data; they learn a technique for doing cosmology with cosmic voids and dark matter halos; and they show how to extract the Euler equation from a graph neural network trained on turbulence data.

LOGML Videos

LOGML is an exciting summer school with projects and talks about graph ML. A collection of videos that includes presentations of the cutting-edge research as well as industrial applications from leading companies are available now for everyone.

ICML 2021 Videos

One of the main important conferences in ML, ICML, features about 50 papers on graphs, for which recordings are now available. For the list of relevant graph papers, check out this repo.

Software 💻

PyG 2.0 Release

A major new release of Pytorch Geometric -- a collaboration between Stanford and TU Dortmund. In addition to a constantly growing number of supported GNN architectures, the 2.0 version features heterogeneous graph support, GraphGym - a whole platform for designing and experimenting with GNNs, pre-defined models, half-precision support, and other smaller improvements to make your GNN journey easier.

DGL 0.7 Release

DGL v0.7 brings improvements on the low-level system infrastructure as well as on the high-level user-facing utilities. Among improvements are GPU-based neighbor sampling that removes the need to move samples from CPU to GPU, improved CPU message-passing kernel, DGL Kubernetes Operator, and search over existing models and applications.

TorchDrug: a powerful and flexible machine learning platform for drug discovery

Jian Tang and co-workers from MILA open-sourced a new library TorchDrug on drug modeling with machine learning. It includes an easy interface for property prediction, pretrained molecular representations, de-novo molecule design & optimization, knowledge graph reasoning, and more.

Posts 📚

Graph Neural Networks as Neural Diffusion PDEs

A post by Michael Bronstein about the connection of GNNs and differential equations that govern diffusion on graphs. This gives a new mathematical framework for studying different architectures on graphs as well as a blueprint for developing new ones.

Review: Deep Learning on Sets

A new blog post by Fabian Fuchs and others about recent approaches to applying deep learning on sets. It digests several paradigms such as permuting & averaging, sorting, approximating invariance and learning on graphs as a way to overcome permutation invariance of machine learning algorithms.

GNN Tutorial & Graph Convolution Intuition @ Distill

As the last bow of Distill.pub, the authors prepared two very cool articles breaking down message passing and graph convolutions: A Gentle Introduction to Graph Neural Networks and Understanding Convolutions on Graphs.

Knowledge Graphs in Natural Language Processing @ ACL 2021

Already a regular update from Michael Galkin on the SOTA applications of KG in the world of words. This year’s focus is on neural databases & retrieval, KG-augmented language models, KG embeddings & link prediction, entity alignment, KG construction, entity linking, relation extraction, KGQA: temporal, conversational, and AMR.

Future events 🔮

The Learning on Graphs and Geometry Reading Group

A graph and geometry reading group organized by Hannes Stärk with supervision from Pietro Liò at Cambridge. It includes many interesting fresh papers on graphs.

Graph ML in Industry Workshop

Complementary to the Stanford’s event above, we organize a mini-workhop that would gather industry researchers to talk about their experience of applying graph models in production. Our motivation was an apparent shortage of industry applications for GNNs and graph embeddings in the real-world and desire to share success stories and pain points of those who go from research to production. Please, join us at the Zoom and YouTube on 23rd September, 17-00 Paris time (should be more or less time-friendly for most of the time zones).

---

That’s all for today 👋 If you like the post 👍, make sure to subscribe to my twitter 🐦and graph ML telegram channel 📝 Until next time!

Sergey