Self-Attention Neural Bag-of-Features
Chumachenko, Kateryna; Iosifidis, Alexandros; Gabbouj, Moncef (2022)
Chumachenko, Kateryna
Iosifidis, Alexandros
Gabbouj, Moncef
IEEE
2022
2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing, MLSP 2022
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202302102255
https://urn.fi/URN:NBN:fi:tuni-202302102255
Kuvaus
Peer reviewed
Tiivistelmä
In this work, we propose several attention formulations for multi-variate sequence data. We build on top of the recently introduced 2D-Attention and reformulate the attention learning methodology by quantifying the relevance of feature/temporal dimensions through latent spaces based on self-attention rather than learning them directly. In addition, we propose a joint feature-temporal attention mechanism that learns a joint 2D attention mask highlighting relevant information without treating feature and temporal representations independently. The proposed approaches can be used in various architectures and we specifically evaluate their application together with Neural Bag of Features feature extraction module. Experiments on several sequence data analysis tasks show the improved performance yielded by our approach compared to standard methods.
Kokoelmat
- TUNICRIS-julkaisut [16983]