Predicting Visual Importance of Mobile UI Using Semantic Segmentation

Ami Yamamoto, Yuichi Sei, Yasuyuki Tahara, Akihiko Ohsuga

2023

Abstract

When designing a UI, it is necessary to understand what elements are perceived to be important to users. The UI design process involves iteratively improving the UI based on feedback and eye-tracking results on the UI created by the designer, but this iterative process is time-consuming and costly. To solve this problem, several studies have been conducted to predict the visual importance of various designs. However, no studies specifically focus on predicting the visual importance of mobile UI. Therefore, we propose a method to predict visual importance maps from mobile UI screenshot images and semantic segmentation images of UI elements using deep learning. The predicted visual importance maps were objectively evaluated and found to be higher than the baseline. By combining the features of the semantic segmentation images appropriately, the predicted map became smoother and more similar to the ground truth.

Paper Citation

in Harvard Style

Yamamoto A., Sei Y., Tahara Y. and Ohsuga A. (2023). Predicting Visual Importance of Mobile UI Using Semantic Segmentation. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-623-1, pages 260-266. DOI: 10.5220/0011655800003393

in Bibtex Style

@conference{icaart23,
author={Ami Yamamoto and Yuichi Sei and Yasuyuki Tahara and Akihiko Ohsuga},
title={Predicting Visual Importance of Mobile UI Using Semantic Segmentation},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2023},
pages={260-266},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011655800003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - Predicting Visual Importance of Mobile UI Using Semantic Segmentation
SN - 978-989-758-623-1
AU - Yamamoto A.
AU - Sei Y.
AU - Tahara Y.
AU - Ohsuga A.
PY - 2023
SP - 260
EP - 266
DO - 10.5220/0011655800003393