Predictive Action Planning for Hole Cleaning Optimization and Stuck Pipe Prevention Using Digital Twinning and Reinforcement Learning

Abstract

Abstract Poor hole cleaning leads to problems such as pack-offs, stuck pipe incidents, and lost circulation events due to increased equivalent circulating density (ECD). Hole cleaning issues can be mitigated by using a digital twinning system that constantly monitors borehole condition with real-time data and process models, and suggests optimal actions. The solution space of possible actions, however, is very large causing computational hurdles for real time implementation. This paper describes a time-efficient digital twinning approach that uses reinforcement learning (RL) to simulate scenarios corresponding to multiple hole cleaning actions. Digital twinning of cuttings transport in a wellbore requires the development of a system with multiple integrated physics and data-based models to quantitatively detect symptoms of impending issues early, and to take actions to mitigate these problems. First, the state of the borehole is quantified in terms of ECD, concentration of cuttings in flow, and cuttings bed height. Next, a solution space of all legal and feasible actions based on the current system state is obtained. Finally, the alternate action sequences are evaluated to suggest the most suitable path forward. To classify the state of the borehole in terms of ECD, cuttings bed height and cuttings concentration in the flow stream, hydraulics and cuttings transport models were implemented, tested and validated against real field and experimental data. Based on the state of the system, allowable actions were evaluated in terms of some combination of hole cleaning parameters that can be controlled in near real-time, such as flow rate, rotary speed, mud density, mud rheology and the weight on bit for rate of penetration (ROP) control. To limit the size of the solution space, those actions that were unrealistic for implementation in real-time were discarded. Also, any actions that violated the limits of the available drilling margin were discarded. Finally, by defining immediate rewards associated with different states-action pairs based on their effect on the wellbore stability and hole condition, the concepts of Markov reward process (MRP) and scenario realizations were used to suggest the optimal action for a given state. The predicted output of the algorithm for multiple operational scenarios was validated by comparing it with actions that a hole cleaning / extended reach drilling (ERD) expert would have taken when given similar scenarios. The automated process of identifying hole cleaning system states, generating multiple viable hole cleaning action sequences, and finally evaluating the probability of successful hole cleaning for multiple actions in real-time and deciding the best course of action, is a novel contribution of this work. It will benefit practitioners who struggle with hole cleaning / stuck pipe-related non-productive time and pave the way for hole cleaning automation.

Topics & Concepts