Inteligência Artificial generativa: Construção e avaliação de um modelo em apoio à autoridade marítima brasileira

Abstract

ABSTRACT GENERATIVE ARTIFICIAL INTELLIGENCE: construction and evaluation of a model in support of the Brazilian Maritime Authority This work presents the development of a generative artificial intelligence model with a retrieval, augmentation, and generation architecture for the application of Brazilian Maritime Authority regulations and related legislation, to determine its performance. The study addresses the construction of a vector database from selected normative documents, the implementation of the tool using a large language model and the evaluation of its performance. The technical methodology employed involved the use of models from Google and OpenAI, as well as algorithms and frameworks in the Python programming language. The developed model was evaluated using questions from a public contest of the Brazilian Navy for entry into the technical staff in the area of water traffic safety, achieving an accuracy rate of 75%. Additional metrics such as response fidelity, response relevance and context relevance were also measured. The results achieved indicate a robust performance of the tool, comparable to benchmarks of advanced models on the market. Despite the promising numbers, the study also discusses limitations and challenges in the implementation of generative artificial intelligence, including hallucinations, biases, issues of interpretability, security and privacy. Finally, the work concludes that the developed model represents a significant step towards considering the application of this emerging technology as a support tool for the Brazilian Maritime Authority, with the potential to automate queries, speed up normative interpretations and improve personnel training, contributing to ensuring regulatory compliance and strengthening Brazil's maritime power.