Dynamic Algorithm Configuration: Foundation of a New Meta-Algorithmic Framework

Abstract

The performance of many algorithms in the fields of hard combinatorial problem solving, machine learning or AI in gen- eral depends on parameter tuning. Automated methods have been proposed to alleviate users from the tedious and error-prone task of manually searching for performance-optimized configurations across a set of problem instances. However, there is still a lot of untapped potential through adjusting an algorithm’s parameters online since different parameter values can be optimal at different stages of the algorithm. Prior work showed that reinforcement learning is an ef- fective approach to learn policies for online adjustments of algorithm parameters in a data-driven way. We extend that approach by formu- lating the resulting dynamic algorithm configuration as a contextual MDP, such that RL not only learns a policy for a single instance, but across a set of instances. To lay the foundation for studying dynamic algorithm configuration with RL in a controlled setting, we propose white-box benchmarks covering major aspects that make dynamic al- gorithm configuration a hard problem in practice and study the per- formance of various types of configuration strategies for them. On these white-box benchmarks, we show that (i) RL is a robust candi- date for learning configuration policies, outperforming standard pa- rameter optimization approaches, such as classical algorithm config- uration; (ii) based on function approximation, RL agents can learn to generalize to new types of instances; and (iii) self-paced learning can substantially improve the performance by selecting a useful sequence of training instances automatically

Topics & Concepts