Vol. 337 No. 8 (2026)
DOI https://doi.org/10.18799/24131830/2026/8/5806
Multimodal intelligent decision support system for improving industrial safety at oil and gas facilities
Relevance. Ensuring industrial safety at oil and gas facilities requires simultaneous monitoring of process parameters, equipment status, personnel actions, and early warning signs of emergency situations. In practice, this data often remains dispersed among independent subsystems: video surveillance, fire alarms, gas analysis, access control systems, and industrial automation. As a result, the operator receives not a unified risk picture, but a set of disparate signals that must be quickly correlated by time, location, and meaning. Aim. To substantiate an approach to creating a local multimodal intelligent decision support system that combines video, audio and sensory data. Methods. The methodological basis of the work is system analysis, conceptual modeling, multimodal data fusion, and microservice architecture principles. Results. The authors have proposed a system model, including a data collection layer, preprocessing, computer vision, audio analytics, and sensor diagnostics modules, a RabbitMQ message bus, a correlation aggregator, a decision support module, and an operator interface. It is demonstrated that cross-verification of events across multiple independent sources can improve the reliability of alarm messages and reduce operator workload. The scientific significance of this work lies in the formalization of an approach to the joint interpretation of heterogeneous data, while its practical significance lies in the feasibility of applying a local architecture to critical information infrastructure facilities.
For citation: Chumakov K.V., Zhdanova A.O. Multimodal intelligent decision support system for improving industrial safety at oil and gas facilities. Bulletin of the Tomsk Polytechnic University. Geo Assets Engineering, 2026, vol. 337, no. 8, pp. 184–199. http://doi.org/10.18799/24131830/2026/8/5806
Keywords:
Industrial safety, oil and gas industry, artificial intelligence, multimodal analytics, computer vision, sensor diagnostics, decision support
References:
1. Federal Law of July 21, 1997 no. 116-FZ «On Industrial Safety of Production Facilities» (In Russ.) Available at: http://publication.pravo.gov.ru/document/0001202311140010?index=1 (accessed 6 July 2026).
2. Federal Law of July 26, 2017 no. 187-FZ «On the Security of Critical Information of the Russian Federation» (In Russ.) Available at: http://publication.pravo.gov.ru/document/0001201707260023 (accessed 6 July 2026).
3. NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology, 2023. 42 p. DOI: https://doi.org/10.6028/NIST.AI.100-1
4. Khurram M., Zhanget C., Muhammad S., Kishnani H., An K., Abeywardena K., Chadha U., Behdinan K. Artificial intelligence in manufacturing industry worker safety: a new paradigm for hazard prevention and mitigation. Processes, 2025, vol. 13, no. 5, Article number 1312.
5. Abaturov V.Yu., Lepikhin A.V. Industrial safety. Legal aspects in the field of industrial safety. Student Bulletin, 2023, no. 35-3 (274), pp. 42–43. (In Russ.)
6. Vukicevic A.M., Petrovic M., Milosevic P., Peulic A., Jovanovic K., Novakovic A. A systematic review of computer vision-based personal protective equipment compliance in industry practice: advancements, challenges and future directions. Artificial Intelligence Review, 2024, vol. 57, Article number 319.
7. Borodenko I.Ni., Gubina D.I. Analysis of information security in industrial production. Languages in Professional Communicatio, 2022, pp. 10–17.
8. Ye T., Peng T., Yang L. Review on sound-based industrial predictive maintenance: from feature engineering to deep learning. Mathematics, 2025, vol. 13, no. 11, Article number 1724.
9. Petrushkov S.V. Method for preventing false alarm triggering in a system of several security detectors and alarm system: patent RF, no. 2825412, 2024. (In Russ.)
10. Restrepo D., Wu C., Vásquez-Venegaset C., Nakayama L.F., Celi L.A., López D.M. DF-DM: a foundational process model for multimodal data fusion in the artificial intelligence era. This work is licensed under a CC BY 4.0 License, 2024, DOI: 10.21203/rs.3.rs-4277992/v1.
11. Hamasha M.M., Albedoor Q., Hamasha S., Ali H., Qamar A., Berrah F. A comprehensive framework for IoT-driven predictive maintenance: leveraging al and edge computing for enhanced equipment reliability. Journal of Applied Engineering Science, 2025, vol. 23 (3), no. 1287, pp. 471–486.
12. SS R 53325-2012. Fire-fighting equipment. Fire-fighting automation equipment. General technical requirements and test methods. Moscow, Standartinform Publ., 2014. 121 p. (In Russ.)
13. SS R 51558-2014. Security television equipment and systems. Classification. General technical requirements. Test methods. Moscow, Standartinform Publ., 2014. 24 p. (In Russ.)
14. Perez-Cerrolaza J., Abella J., Borg M., Donzella V., Englund C., Tauber M., Nikolakopoulos G., Flores J.L., Cazorla F.J. Artificial intelligence for safety-critical systems in industrial and transportation domains: a survey. ACM Computing Surveys, 2024, vol. 56, Article number 176, pp. 1–10. DOI: 10.1145/3626314.
15. Chen Z., Li Z.W., Huang J., Liu S.Z., Long H.X. An effective method for anomaly detection in industrial Internet of Things using XGBoost and LSTM. Scientific Reports, 2024, vol. 14, Article number 23969. DOI: 10.1038/s41598-024-74822-6.
16. Chizhevskaya E.L., Vydrenkov A.D., Zemenkova M.Yu., Zemenkov Yu.D. Solving problems of predictive analytics of hydrate-free operation of field gas pipelines using machine and deep learning algorithms. Bulletin of Tomsk Polytechnic University. Geo Assets Engineering, 2025, vol. 336, no. 11, pp. 55–65. (In Russ.) DOI: 10.18799/24131830/2025/11/5178.
17. Filatov V.M., Rastvorova I.I., Zhurba E.D. Review of radio-electronic wave methods and devices for diagnostics and monitoring of oil. Bulletin of Tomsk Polytechnic University. Geo Assets Engineering, 2025, vol. 336, no. 12, pp. 164–181 (In Russ.) DOI: 10.18799/24131830/2025/12/4945.
18. Lyapichev D.M., Admakin M.M., Romanov S.V., Poletaev M.G., Zhuravlev A.S. Analysis of the results of pilot operation of a system for monitoring the technical condition of buildings, structures, process equipment and pipelines of a compressor station. Equipment and technologies for the oil and gas complex, 2019, no. 2 (110), pp. 54–58. (In Russ.)
19. SS R ISO 45001-2020. Occupational health and safety management systems. Requirements with guidance for use. Moscow, Standartinform Publ., 2012. 19 p. (In Russ.)
20. SS 12.1.004-91. Occupational safety standards system. Fire safety. General requirements. Moscow, Standartinform Publ., 2006. 64 p. (In Russ.)
21. Federal environmental, industrial and nuclear supervision service of russia. annual report on the activities of the federal environmental, industrial and nuclear supervision service in 2024. Moscow, 2025. 140 p. (In Russ.) Available at: https://www.gosnadzor.ru/public/annual_reports/Пр-259%20от%2004.08.2025%20Годовой%20отчет.pdf (accessed: 6 July 2026).
