Image captioning using attention network and machine learning approaches

The automatic prediction of image captions is a very challenging task in natural language processing (NLP). Many studies have employed convolutional neural networks as encoders and decoders. Nevertheless, to accurately predict image captions, a model must comprehend the semantic relationships among the numerous objects available in the given image. Attention-based mechanism carries out a linear grouping of encoder and decoder state operations. It places equal emphasis on the semantic information that is found in the caption as well as the visual knowledge that is contained within a given image. In this research paper, we integrated the local attention approach with two pre-trained convolutional neural networks (CNN) known as VGG19 and Inception_V3 in order to provide the textual description of any given image. These models are employed as an encoder, while the recurrent neural network serves as the decoder in the system. Together with the attention mechanism, these encoders are capable of transmitting the semantic-context knowledge to the decoder and achieved the BLUE Score of 64.7.