Double-Layer Q-Learning-Based Joint Decision-Making of Dual Resource-Constrained Aircraft Assembly Scheduling and Flexible Preventive Maintenance
Qi Yan, Hongfeng Wang
Abstract
This study investigates an integrated optimization problem of aircraft assembly scheduling and flexible preventive maintenance (PM) with decision-making (FPM/DM), in which both machine flexibility and worker flexibility are considered. A mixed integer linear programming (MILP) model targeted at minimizing the assembly cycle is first established to formulate the problem. Then, CPLEX is used for solving the exact solution of the MILP model, however, it is time-consuming and difficult to apply to complex scenarios. For the tradeoff between solution quality and computational efficiency, an improved double-layer Q-learning (QL) algorithm with PM decision-making (IDLQL/PM) is further designed. Specifically, upper-layer QL is responsible for learning a proper machine selection heuristic from the given action set to ensure the machine load balance, and lower-layer QL provides a proper action (either an operation or a PM) for the selected machine to reduce unnecessary idle time and PM. Next, some instances are generated to evaluate the performance of IDLQL/PM. In a small-sized instance compared to CPLEX, the average solution gap is 1.8%, while the model solving time is reduced by 481%. As for 16 large-sized instances, IDLQL/PM presents a clear advantage over the well-known genetic algorithm and two other QL-based approaches. In addition, three state-of-the-art maintenance strategies are selected as rivals to validate the effectiveness of FPM/DM. It is observed that the proposed FPM/DM strategy can avoid improper maintenance during aircraft final assembly and ensure higher assembly efficiency.