What is... an Equivariant Neural Network?
Lek‐Heng Lim, Bradley J. Nelson
Abstract
We explain equivariant neural networks, a notion underlying breakthroughs in machine learning from deep convolutional neural networks for computer vision [KSH12] to AlphaFold 2 for protein structure prediction [JEP + 21], without assuming knowledge of equivariance or neural networks.The basic mathematical ideas are simple but are often obscured by engineering complications that come with practical realizations.We extract and focus on the mathematical aspects, and limit ourselves to a cursory treatment of the engineering issues at the end.We also include some materials with machine learning practitioners in mind.Let and be sets, and ∶ → a function.If a group acts on both and , and this action commutes with the function : ( ⋅ ) = ⋅ () for all ∈ , ∈ , then we say that is -equivariant.The special case where acts trivially on is called -invariant.Linear equivariant maps are well-studied in representation theory and continuous equivariant maps are well-studied in topology.The novelty of equivariant neural networks is that they