Image Generation from a Hyper Scene Graph with Trinomial Hyperedges

Ryosuke Miyake, Tetsu Matsukawa, Einoshin Suzuki

2023

Abstract

Generating realistic images is one of the important problems in the field of computer vision. In image generation tasks, generating images consistent with an input given by the user is called conditional image generation. Due to the recent advances in generating high-quality images with Generative Adversarial Networks, many conditional image generation models have been proposed, such as text-to-image, scene-graph-to-image, and layout-to-image models. Among them, scene-graph-to-image models have the advantage of generating an image for a complex situation according to the structure of a scene graph. However, existing scene-graph-to-image models have difficulty in capturing positional relations among three or more objects since a scene graph can only represent relations between two objects. In this paper, we propose a novel image generation model which addresses this shortcoming by generating images from a hyper scene graph with trinomial edges. We also use a layout-to-image model supplementally to generate higher resolution images. Experimental validations on COCO-Stuff and Visual Genome datasets show that the proposed model generates more natural and faithful images to user’s inputs than a cutting-edge scene-graph-to-image model.

Paper Citation

in Harvard Style

Miyake R., Matsukawa T. and Suzuki E. (2023). Image Generation from a Hyper Scene Graph with Trinomial Hyperedges. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 185-195. DOI: 10.5220/0011699300003417

in Bibtex Style

@conference{visapp23,
author={Ryosuke Miyake and Tetsu Matsukawa and Einoshin Suzuki},
title={Image Generation from a Hyper Scene Graph with Trinomial Hyperedges},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={185-195},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011699300003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - Image Generation from a Hyper Scene Graph with Trinomial Hyperedges
SN - 978-989-758-634-7
AU - Miyake R.
AU - Matsukawa T.
AU - Suzuki E.
PY - 2023
SP - 185
EP - 195
DO - 10.5220/0011699300003417
PB - SciTePress