Controllable Shadow Generation Using Pixel Height Maps
- Yichen Sheng* Purdue University
- Yifan Liu* (R.I.P.🕯️) University of Adelaide
- Jianming Zhang Adobe
- Wei Yin University of Adelaide
- A. Cengiz Oztireli University of Cambridge
- He Zhang Adobe
- Zhe Lin Adobe
- Eli Shechtman Adobe
- Bedrich Benes Purdue University
Abstract
Shadows are essential for realistic image compositing from 2D image cutouts. Physics-based shadow rendering methods require 3D geometries, which are not always available. Deep learning-based shadow synthesis methods learn a mapping from the light information to an object’s shadow without explicitly modeling the shadow geometry. Still, they lack control and are prone to visual artifacts. We introduce “Pixel Height”, a novel geometry representation that encodes the correlations between objects, ground, and camera pose. The Pixel Height can be calculated from 3D geometries, manually annotated on 2D images, and can also be predicted from a single-view RGB image by a supervised approach. It can be used to calculate hard shadows in a 2D image based on the projective geometry, providing precise control of the shadows’ direction and shape. Furthermore, we propose a data-driven soft shadow generator to apply softness to a hard shadow based on a softness input parameter. Qualitative and quantitative evaluations demonstrate that the proposed Pixel Height significantly improves the quality of the shadow generation while allowing for controllability.
More Results
Technical Video
BibTeX
Acknowledgements
Most of the work was done during Yifan and Yichen’s internship at Adobe. This work was also supported by a UKRI Future Leaders Fellowship [grant number G104084]. We thank Dr. Zhi Tian for the discussions. I hope Yifan is happy in heaven. Happy to enjoy life, do research and teach students.
The website template was borrowed from Michaël Gharbi.