Deep Reinforcement Learning Based Adaptive Operator Selection for Evolutionary Multi-Objective Optimization
Ye Tian, Xiaopeng Li, Haiping Ma, Xingyi Zhang, Kay Chen Tan, Yaochu Jin
Abstract
Evolutionary algorithms (EAs) have become one of the most effective techniques for multi-objective optimization, where a number of variation operators have been developed to handle the problems with various difficulties. While most EAs use a fixed operator all the time, it is a labor-intensive process to determine the best EA for a new problem. Hence, some recent studies have been dedicated to the adaptive selection of the best operators during the search process. To address the exploration versus exploitation dilemma in operator selection, this paper proposes a novel operator selection method based on reinforcement learning. In the proposed method, the decision variables are regarded as states and the candidate operators are regarded as actions. By using deep neural networks to learn a policy that estimates the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> value of each action given a state, the proposed method can determine the best operator for each parent that maximizes its cumulative improvement. An EA is developed based on the proposed method, which is verified to be more effective than the state-of-the-art ones on challenging multi-objective optimization problems.