Attention-based CNN-GRU model for automatic medical images captioning : ImageCLEF 2021
Djamila-Romaissa, Beddiar; Oussalah, Mourad; Seppänen, Tapio (2021-09-24)
Djamila-Romaissa, B., Oussalah, M., Seppänen, T., Attention-based CNN-GRU model for automatic medical images captioning : ImageCLEF 2021, 2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021, 21-24 September, Bucharest, Romania, 1613-0073, 1160-1173
© 2021 for the individual papers by the papers' authors. Copyright © 2021 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).
https://creativecommons.org/licenses/by/4.0/
https://urn.fi/URN:NBN:fi-fe2021102151849
Tiivistelmä
Abstract
The action of understanding and interpretation of medical images is a very important task in the medical diagnosis generation. However, manual description of medical content is a major bottleneck in clinical diagnosis. Many research studies were devoted to develop automated alternatives to this process, which would have enormous impact in terms of efficiency, cost and accuracy in the clinical workflows. Different approaches and techniques have been presented in the literature ranging from traditional machine learning methods to deep learning based models. Inspired by the outperforming results of the later techniques, we present in the current paper, our team participation (RomiBed) to the ImageCLEF medical caption prediction task. We addressed the challenge of medical image captioning by combining a CNN encoder model with an attention-based GRU language generator model whereas a multi-label CNN classifier is used for the concept detection task. Using the provided data in the training, validation and test subsets, we obtain an average F_measure of 14.3% and a BLEU score of 0.243 on the ImageCLEF concept detection and the caption prediction challenges, respectively.
Kokoelmat
- Avoin saatavuus [31928]