Salient Mask-Guided Vision Transformer for Fine-Grained Classification

Dmitry Demidov, Muhammad Sharif, Aliakbar Abdurahimov, Hisham Cholakkal, Fahad Khan

2023

Abstract

Fine-grained visual classification (FGVC) is a challenging computer vision problem, where the task is to automatically recognise objects from subordinate categories. One of its main difficulties is capturing the most discriminative inter-class variances among visually similar classes. Recently, methods with Vision Transformer (ViT) have demonstrated noticeable achievements in FGVC, generally by employing the self-attention mechanism with additional resource-consuming techniques to distinguish potentially discriminative regions while disregarding the rest. However, such approaches may struggle to effectively focus on truly discriminative regions due to only relying on the inherent self-attention mechanism, resulting in the classification token likely aggregating global information from less-important background patches. Moreover, due to the immense lack of the datapoints, classifiers may fail to find the most helpful inter-class distinguishing features, since other unrelated but distinctive background regions may be falsely recognised as being valuable. To this end, we introduce a simple yet effective Salient Mask-Guided Vision Transformer (SM-ViT), where the discriminability of the standard ViT’s attention maps is boosted through salient masking of potentially discriminative foreground regions. Extensive experiments demonstrate that with the standard training procedure our SM-ViT achieves state-of-the-art performance on popular FGVC benchmarks among existing ViT-based approaches while requiring fewer resources and lower input image resolution.

Paper Citation

in Harvard Style

Demidov D., Sharif M., Abdurahimov A., Cholakkal H. and Khan F. (2023). Salient Mask-Guided Vision Transformer for Fine-Grained Classification. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 27-38. DOI: 10.5220/0011611100003417

in Bibtex Style

@conference{visapp23,
author={Dmitry Demidov and Muhammad Sharif and Aliakbar Abdurahimov and Hisham Cholakkal and Fahad Khan},
title={Salient Mask-Guided Vision Transformer for Fine-Grained Classification},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP},
year={2023},
pages={27-38},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011611100003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP
TI - Salient Mask-Guided Vision Transformer for Fine-Grained Classification
SN - 978-989-758-634-7
AU - Demidov D.
AU - Sharif M.
AU - Abdurahimov A.
AU - Cholakkal H.
AU - Khan F.
PY - 2023
SP - 27
EP - 38
DO - 10.5220/0011611100003417
PB - SciTePress