Simulação de orientação de aeronave: integrando realidade virtual e aprendizado de máquina em uma solução de baixo custo para otimização do treinamento da Marinha do Brasil

Abstract

Landing Signalman (LS) is the military on the helipad of a moving ship in charge of assisting the pilot, employing visual signs during landing and takeoff and ensuring general safety conditions of the flight deck area. This task requires self-confidence, knowledge, skills, team coordination, and the ability to react appropriately, which can be only achieved with intensive training. After being introduced to theoretical concepts, the Brazilian Navy LS trainees go directly to the practical stage. This hands-on training phase straight after the classroom reveals a series of limitations in LS trainee’s performance. In order to better prepare the LS trainee, reduce training costs and provide a safe environment to perform several training situations, a low-cost and easy to transport Virtual Reality (VR) simulator was developed to increase training, denominated Helicopter Visual Signal Simulator. Due to its specificities, a Machine Learning gesture recognition method was integrated into the virtual environment, and two data sets were created. This dissertation details the process to achieve the proposed objectives and the development of the simulator prototype, in addition to presenting the analysis of the tests carried out with 15 experienced instructors. The results indicate that this simulator meets all requirements to provide a reliable and low-cost training solution for the Brazilian Navy.