First Object Match-Move and Composite Attempt

For my first object match move, I decided to try a model I had created for an animation class of a soda can. My end goal for this match move and composite was to create a realistic looking render of the can so that at first glance it was not apparent the object was a 3D model. I also decided to use the brand "Bonk Energy Drink," which is not real but in fact an item in the online video game Team Fortress 2.

The final video can be seen here

Some things that worked: reflections, color correction

What didn't work: 3D camera track, not all reflections looked accurate, too dark, too little depth of field, color correction needed improvement.

The "Set"

tracker

I followed a similar process to my 3D ball track, but with some changes. Since this was an interior shoot, I was very concearned about light. I used my desk lamp as a key light off to the left of where the object would be, in order to be able to re-create this light in 3D space later on. I used a small post it note with dots drawn on it as a tracking marker, hoping this would give After Effects enough information to generate a point cloud. However, I later found out that this was not enough as after a nine hour render, my first match move was a failure. The track had too many errors and bounced around in the final video.




I also made some mistakes with my camera settings, as my lens aperture was set to f/2.8. With such extreme depth of field, my final 3D object looked too crisp and out of place when compared to the rest of the objects in the video.

point cloud data

track solid, used later as a shadow catcher

 Once in After Effects I tracked a solid and camera to my marker. This solid I later used as a shadow catcher for my light source, making sure to set it as the origin before moving into Cinema 4D. This ensured that once in C4D, my object would be in the correct place in 3D space.






Moving into Cinema 4D, I exported the point data from After Effects to generate a 3D scene that matched that of my original clean plate. I changed the color of my track solid to white in order to make it easy to composite out as a shadow catcher. I then proceeded to bring in my can model, adjusting some of the material properties trying to match that of a real soda can. This was an area I had difficulties with, and plan to focus on for my next attempt.

track solid in C4D

 After adjusting some of the material properties I created a physical sky in Cinema 4D. This sky serves as a place to insert my HDRI map I took of my room, allowing the can to "see" accurate reflections of what was around it. However, this proved to be difficult as my floor was showing up in the reflections of the can. By adding a compositing tag to the floor I was able to solve this problem, but of course after my nine hour render!

HDRI map from the cans perspective 

I then moved to my render settings, ensuring to enable multi-pass rendering and selecting: RGBA image, Shadows, Ambient Occlusion and object buffer.

Moving back into after effects I duplicated my C4D file layer three times in order to have the four layers needed for this multi pass-render. I finally did some modest color correction to the can in order to try and mimic the color of the scene. I then hit render and prayed to the After Effects gods.

Ambient Occlusion Pass

First Object Match-move and Composite

For my first 3D match move and composite of an object, I decided to try and create a 3D ball with a shiny mirror-like texture. I knew that there would be several elements to creating this including: a clean plate that was easy to track, a lighting scenario that was easy to replicate in 3D space, and a 3D image of the location I wished the mirror texture of the ball to reflect. Next i will break down the shooting, 3D element creation, and finally composing of the final video, which can be seen here


THE SHOOT:

Backyard HDRI image flattened

I shot a clean plate in my backyard, utilizing the brick patio as an ideal surface to provide a great deal of tracking information for the 3D camera tracker. I made sure to plan out my camera move in advance and "leave space" for where I intended to place the 3D ball. I also noted the time of day and where the sun was pointing in relation to the other real elements in the shot. I then took an environment map of the backyard on my phone using the Microsoft app Photo synth. The app stitches together pictures much like a panorama app would, but is able to create a 3D spherical environment. I made sure to take the pictures from the perspective of where I wanted the the place to be placed as it is going to later provide the information needed to create the reflected environment.


Editing:

The original clean plate and 3D track points and target

First, I took my footage into After Effects and conformed it to my composition so all of the sequence settings matched my original footage. Next, I used the 3D camera tracker to obtain information from the clean plate (the bricks turned out to be quite useful when using the camera tracker). I selected a number of track points to use that replicated the area I wanted to place the 3D solid on. I made sure to create a camera and a track solid from the 3D camera track, as well as set my origin and ground plane from the track points. This information would be used later on in Cinema 4D. In order to get all of this tracking data into C4D, I created a new C4D file from the original video track containing the 3D tracking information.

Cinema 4D screengrab 

Once in C4D, i was left with a blank canvas with a tracking solid at the origin of the composition (0,0,0), and a camera move in 3D space that directly mimicked the original camera move I had shot. This seamless integration between After Effects and C4D was incredible to me already. Next, I created a 3D sphere in C4D as well as a sky environment, and began to tweak the settings for each. I also applied the HDRI map of my backyard to my sky and created an infinite light where I believed my sun was during the physical shoot.

Frame 148 in Cinema 4D

 I then applied a metallic texture to my sphere and turned reflections on. This allowed the sphere to reflect the sky and therefore the HDRI map I had placed on my sky element. I finally moved to my render settings and enabled Multi-pass Rendering including: ambient occlusion, shadow, object buffer, and RGBA image. Then I saved my work and jumped back into After Effects.

Frame 148 in After Effects

Compositing:
Thanks to the smooth integration between After Effects and Cinema 4D, I was able to save my work in C4D, and then have the file in After Effects update automatically in my composition; they call this feature dynamic link and it is super powerful. I began to go through all of my layer files and work on compositing my elements into the comp. Through the C4D plugin for After Effects, I was able to bring in each element (ambient occlusion, shadow, object buffer, and RGBA image) as separate layers. This allowed me to target various elements individually when compositing them together. The final video took about 4 hours to render.

All in all I am happy with this first test and track of a 3D object. I feel as though the shadows need to be improved on, but otherwise I am very happy with the result.