The article discusses the problem of optimal well placement, it is proposed to solve it using the Alpha Zero reinforcement learning algorithm, which has proven itself as an artificial intelligence for games and for solving optimization problems in quantum optimal control theory. It is assumed that it can be no less effective in solving the problem of optimal well placement. The main components of the algorithm that influence decision making are the Monte Carlo tree search and the neural network. To select the location of the next well, a limited number of simulations are carried out along the tree, the root of which is the current sector from which the selection is made. During simulations, different tree branches are explored and new nodes are added for unexplored branches. The neural network gives an estimate of the NPV for the unexplored variant branches and the desirability of the following actions. As states in our approach, we will consider sectors of a fixed size, taken from a hydrodynamic model (hereinafter HDM), calculated for the entire field. Each sector is characterized by maps of properties of the HDM cut along the contour of the sector. Additionally, a vector of economic parameters is set. Note that a great advantage of the chosen approach is that there is no need for a complete enumeration of placement options - only branches with good estimates are revealed more deeply. The results of the system prototype operation according to the developed algorithm are presented and a brief comparison with the results of a hydrodynamic simulator is made. The developed prototype showed correct operation for synthetic models during the placement of production wells. The proposed well placement algorithm has shown its performance, comparable to the results of the hydrodynamic simulator.
References
1. Sugaipov D.A., Nekhaev S.A., Perevozkin I.V., et al., Optimization of well pattern for oil rim fields (In Russ.), Neftyanoe khozyaystvo = Oil Industry, 2019, no. 12, pp. 44–46.
2. Vladimirov I.V., Al'mukhametova E.M., Efficient location of the rows of injection and production wells in high viscosity oil deposits with extended reservoir tightness zones (In Russ.), Problemy sbora, podgotovki i transporta nefti i nefteproduktov, 2019, no. 3, pp. 62–74.
3. Ilsik Jang, Seeun Oh, Yumi Kim et al., Well-placement optimization using sequential artificialneural networks, Energy Exploration & Exploitation, 2018, V. 36 (3), pp. 433–449.
4. Hamida Z., Azizi F., Saad G., An efficient geometry-based optimization approach for well placement in oil fields, Journal of Petroleum Science and Engineering, 2017, V. 149, pp. 383–392.
5. Silver D. et al., A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play, Science, 2018, V. 362, no. 6419, pp. 1140–1144.
6. Pumperla M., Ferguson K., Deep learning and the game of Go, Manning, 2019, V. 231, pp. 279.
7. Dalgaard M. et al., Global optimization of quantum dynamics with AlphaZero deep exploration, npj Quantum Information, 2020, V. 6, no. 1, DOI: 10.1038/s41534-019-0241-0
8. Takhautdinov Sh.F., Khisamutdinov N.I., Taziev M.Z. et al., Sovremennye metody resheniya inzhenernykh zadach na pozdney stadii razrabotki neftyanogo mestorozhdeniya (Modern methods for solving engineering problems at a late stage of oil field development), Moscow: Publ. of VNIIOENG, 2000, 104 p.
9. Lozhkin A., Bozek P., Maiorov K., The method of high accuracy calculation of robot trajectory for the complex curve, Management Systems in Production Engineering, 2020, V. 28, no. 4, pp. 247-252.
10. Božek P. et al., Information technology and pragmatic analysis, Computing and informatics, 2018, V. 37, no. 4, pp. 1011–1036.
