Intelligent decision making on PCP production parameters of CBM wells based on reinforcement learning

In order to realize the continuous decision making and continuous control on the production parameters of progressive cavity pump (PCP) in coalbed methane (CBM) wells and ensure the efficient and stable production of CBM wells in the long term, this paper put forward the framework of the reinforcement model with the ability of action self-optimization for PCP production of CBM well, as well as Q learning & Sarsa & Sarsa (lambda) algorithms by taking the maximum cumulative gas production within a PCP production cycle as the optimization target. The dynamic environment is rewarded and punished flexibly by means of the interactive learning with environment, so that the intelligent agent can perform intelligent decision making and parameter optimization in the complicated environment. In this way, the optimal coordinated control on the PCP production of CBM well can be realized effectively, and the problem that the traditional methods fail to make an adjustment quickly based on the environmental change to improve the production effect is solved. It is experimentally indicated that for a given gas production rate curve of CBM well, by taking PCP’s frequency as the single control variable, the Q learning method can effectively provide the optimal strategy of frequency conversion control on PCP production. Obviously, this method has a certain application potential.