Manipulating Attributes of Natural Scenes via Hallucination

Levent Karacan1    Zeynep Akata2    Aykut Erdem1    Erkut Erdem1

Hacettepe University1

University of Amsterdam2

Paper | PyTorch code coming soon

Abstract

In this study, we explore building a two-stage framework for enabling usersto directly manipulate high-level attributes of a natural scene. The key toour approach is a deep generative network which can hallucinate images of a scene as if they were taken at a different season (e.g. during winter),weather condition (e.g. in a cloudy day) or time of the day (e.g. at sunset).Once the scene is hallucinated with the given attributes, the corresponding look is then transferred to the input image while preserving the semantic details intact, giving a photo-realistic manipulation result. As the proposed framework hallucinates what the scene will look like, it does not require any reference style image as commonly done in most of the appearance or style transfer approaches. Moreover, it allows to simultaneously manipulate a given scene according to a large set transient attributes to a large extent within a single model, eliminating the need to train multiple networks per each translation task. Our comprehensive set of qualitative and quantitative results demonstrate the effectiveness of our approach against the competing methods.

paper thumbnail

Paper

arxiv 1808.07413, 2018.

Citation

Levent Karacan, Zeynep Akata, Aykut Erdem, and Erkut Erdem. "Manipulating Attributes of Natural Scenes via Hallucination", in Arxiv 1808.07413.
Bibtex

Code: PyTorch | Coming soon!


Demo


Input

Output

Results in our Paper

Attribute Manipulation


 

Comparison


 

Attibute Transition


 

Additional Results


Videos

Hallucination

Appearance Transfer

Image Editing Tool



Related Work



Acknowledgement

We would like to thank NVIDIA Corporation for the donation of GPUs used in this research.