$$\alpha$$ILP: thinking visual scenes as differentiable logic programs
Hikaru Shindo, Viktor Pfanschilling, Devendra Singh Dhami, Kristian Kersting
Abstract
Abstract Deep neural learning has shown remarkable performance at learning representations for visual object categorization. However, deep neural networks such as CNNs do not explicitly encode objects and relations among them. This limits their success on tasks that require a deep logical understanding of visual scenes, such as Kandinsky patterns and Bongard problems. To overcome these limitations, we introduce $$\alpha {\textit{ILP}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>α</mml:mi><mml:mi>ILP</mml:mi></mml:mrow></mml:math> , a novel differentiable inductive logic programming framework that learns to represent scenes as logic programs—intuitively, logical atoms correspond to objects, attributes, and relations, and clauses encode high-level scene information. $$\alpha$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>α</mml:mi></mml:math> ILP has an end-to-end reasoning architecture from visual inputs. Using it, $$\alpha$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>α</mml:mi></mml:math> ILP performs differentiable inductive logic programming on complex visual scenes, i.e., the logical rules are learned by gradient descent. Our extensive experiments on Kandinsky patterns and CLEVR-Hans benchmarks demonstrate the accuracy and efficiency of $$\alpha {\textit{ILP}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>α</mml:mi><mml:mi>ILP</mml:mi></mml:mrow></mml:math> in learning complex visual-logical concepts.