Inserting 3D Models Into Photographs

A group of students at the University of Illinois have created intelligent software for inserting three dimensional objects into photographs. Designed to be usable by anyone, even people without photo manipulation experience, the software inserts a 3D model into a photo and matches the perspective in relation to the other objects that were photographed. With some help from the user, it also detects the light sources in the image, and even matches the reflection of light from the walls or floor. The results are truly amazing.

How does it work? Well, you start off with a photograph and a 3D model of any object you would like to manipulate into the image. Using the software, you can specify the vanishing points in the image, thereby determining the correct perspective. You can show the program where the light sources are by masking them in a pen-tool like fashion, and the app will determine the intensity and direction of the light on its own. You can also mask foreground subjects in the 2D image, using a clever quick mask tool that automatically identifies the edges of the object for you. This means that a light source inserted into the image will realistically cast shadows behind a two dimensional object in the original image.

All in all, as it is a graduate project, it’s very brainy stuff. The video wasn’t created to showcase the program to end consumers, but it is still very, very interesting to watch. Maybe we can expect this technology in future versions of Adobe Photoshop?

Via Dean Claassens, our web administrator at Orms.