First Object Match-Move and Composite Attempt

For my first object match move, I decided to try a model I had created for an animation class of a soda can. My end goal for this match move and composite was to create a realistic looking render of the can so that at first glance it was not apparent the object was a 3D model. I also decided to use the brand "Bonk Energy Drink," which is not real but in fact an item in the online video game Team Fortress 2.

The final video can be seen here

Some things that worked: reflections, color correction

What didn't work: 3D camera track, not all reflections looked accurate, too dark, too little depth of field, color correction needed improvement.

The "Set"

tracker

I followed a similar process to my 3D ball track, but with some changes. Since this was an interior shoot, I was very concearned about light. I used my desk lamp as a key light off to the left of where the object would be, in order to be able to re-create this light in 3D space later on. I used a small post it note with dots drawn on it as a tracking marker, hoping this would give After Effects enough information to generate a point cloud. However, I later found out that this was not enough as after a nine hour render, my first match move was a failure. The track had too many errors and bounced around in the final video.




I also made some mistakes with my camera settings, as my lens aperture was set to f/2.8. With such extreme depth of field, my final 3D object looked too crisp and out of place when compared to the rest of the objects in the video.

point cloud data

track solid, used later as a shadow catcher

 Once in After Effects I tracked a solid and camera to my marker. This solid I later used as a shadow catcher for my light source, making sure to set it as the origin before moving into Cinema 4D. This ensured that once in C4D, my object would be in the correct place in 3D space.






Moving into Cinema 4D, I exported the point data from After Effects to generate a 3D scene that matched that of my original clean plate. I changed the color of my track solid to white in order to make it easy to composite out as a shadow catcher. I then proceeded to bring in my can model, adjusting some of the material properties trying to match that of a real soda can. This was an area I had difficulties with, and plan to focus on for my next attempt.

track solid in C4D

 After adjusting some of the material properties I created a physical sky in Cinema 4D. This sky serves as a place to insert my HDRI map I took of my room, allowing the can to "see" accurate reflections of what was around it. However, this proved to be difficult as my floor was showing up in the reflections of the can. By adding a compositing tag to the floor I was able to solve this problem, but of course after my nine hour render!

HDRI map from the cans perspective 

I then moved to my render settings, ensuring to enable multi-pass rendering and selecting: RGBA image, Shadows, Ambient Occlusion and object buffer.

Moving back into after effects I duplicated my C4D file layer three times in order to have the four layers needed for this multi pass-render. I finally did some modest color correction to the can in order to try and mimic the color of the scene. I then hit render and prayed to the After Effects gods.

Ambient Occlusion Pass

First Object Match-move and Composite

For my first 3D match move and composite of an object, I decided to try and create a 3D ball with a shiny mirror-like texture. I knew that there would be several elements to creating this including: a clean plate that was easy to track, a lighting scenario that was easy to replicate in 3D space, and a 3D image of the location I wished the mirror texture of the ball to reflect. Next i will break down the shooting, 3D element creation, and finally composing of the final video, which can be seen here


THE SHOOT:

Backyard HDRI image flattened

I shot a clean plate in my backyard, utilizing the brick patio as an ideal surface to provide a great deal of tracking information for the 3D camera tracker. I made sure to plan out my camera move in advance and "leave space" for where I intended to place the 3D ball. I also noted the time of day and where the sun was pointing in relation to the other real elements in the shot. I then took an environment map of the backyard on my phone using the Microsoft app Photo synth. The app stitches together pictures much like a panorama app would, but is able to create a 3D spherical environment. I made sure to take the pictures from the perspective of where I wanted the the place to be placed as it is going to later provide the information needed to create the reflected environment.


Editing:

The original clean plate and 3D track points and target

First, I took my footage into After Effects and conformed it to my composition so all of the sequence settings matched my original footage. Next, I used the 3D camera tracker to obtain information from the clean plate (the bricks turned out to be quite useful when using the camera tracker). I selected a number of track points to use that replicated the area I wanted to place the 3D solid on. I made sure to create a camera and a track solid from the 3D camera track, as well as set my origin and ground plane from the track points. This information would be used later on in Cinema 4D. In order to get all of this tracking data into C4D, I created a new C4D file from the original video track containing the 3D tracking information.

Cinema 4D screengrab 

Once in C4D, i was left with a blank canvas with a tracking solid at the origin of the composition (0,0,0), and a camera move in 3D space that directly mimicked the original camera move I had shot. This seamless integration between After Effects and C4D was incredible to me already. Next, I created a 3D sphere in C4D as well as a sky environment, and began to tweak the settings for each. I also applied the HDRI map of my backyard to my sky and created an infinite light where I believed my sun was during the physical shoot.

Frame 148 in Cinema 4D

 I then applied a metallic texture to my sphere and turned reflections on. This allowed the sphere to reflect the sky and therefore the HDRI map I had placed on my sky element. I finally moved to my render settings and enabled Multi-pass Rendering including: ambient occlusion, shadow, object buffer, and RGBA image. Then I saved my work and jumped back into After Effects.

Frame 148 in After Effects

Compositing:
Thanks to the smooth integration between After Effects and Cinema 4D, I was able to save my work in C4D, and then have the file in After Effects update automatically in my composition; they call this feature dynamic link and it is super powerful. I began to go through all of my layer files and work on compositing my elements into the comp. Through the C4D plugin for After Effects, I was able to bring in each element (ambient occlusion, shadow, object buffer, and RGBA image) as separate layers. This allowed me to target various elements individually when compositing them together. The final video took about 4 hours to render.

All in all I am happy with this first test and track of a 3D object. I feel as though the shadows need to be improved on, but otherwise I am very happy with the result.

OSX Mavericks and After Effects Troubleshooting

So there has been a bit of a lull as I have been doing a ton of research with Cinema 4D and it's integration into After Effects. However, there has been a bit of a glitch. With the introduction of the new Apple OS X, Mavericks, I received an error message while attempting to launch After Effects. The message read "You can't use this version of the application 'Adobe After Effects CC' with this version of OS X. You have 'Adobe After Effects CC' 12.0." I spent several hours trouble shooting and resorted to un-installing and re-installing the program, but to no avail.

I will post again once I can fix this issue. Until then I will spend some time shooting clean plate footage.

cheers

Mavericks After Effects Error Message

3D Text Tracking Test

Final shot screen-shot in After Effects

After spending several hours I was finally able to create my first 3D text track. The final video is here:

http://youtu.be/rPQyBvsdqb8

I shot some handheld footage in order to add some depth and dynamic movement as I felt the camera slider was limiting at times regarding x,y,and z planar information. I then took the footage into After Effects and began to work with the 3D tracker.

raw shot

 I used the advanced method for the camera solve and spent some time finding the best points to use for the track. I then created a camera and text based of these points, tuning the text 90 on the x axis to bring it upright.

raw shot with tracking point data

different layers and elements of the comp

 After making sure the track was clean, I turned on the raytracing render mode in After Effects. This allowed me to extrude the text out and create some lights and shadow casting. I placed a shadow catcher and light beneath the text in order to generate more shadow and messed around with the settings to get a look and feel I felt was appropriate. Now, I experimented with the different textures i could apply to the text and finalized the comp.

shadow catcher preferences and options

text and tracking information for shadow catcher

Render queue

Then, I sent the comp to media encoder and let the render fun begin...

Initial 3D Camera Tracking

After taking the footage into After Effects, I began to color grade the material. I slightly adjusted the contrast as well as the sharpness and saturation in order to give the footage a little more life. 

The links to the videos from this first section can be found here and here.

Final shot with tracking points.

3D camera tracker settings.

Next, I opened the 3D camera tracker and began to adjust some of the settings. I found that I could adjust the size of my tracking points, as well as the target size generated from the points. In addition, I was able to chose the type of camera move, how detailed the camera solve was, and what sort of auxiliary component was generated with the 3D camera created (null, object, or text). I decided for my first test I would try to use text.

I wrote the word hello and attached it in 3D space to the rear mudguard of one of the bikes I filmed. I have attached my results as youtube links at the bottom.

Overall I feel that this was a successful first track. I was able to see how the 3D camera tracker worked and understand how to shoot future clean plates to function with the information the tracker is looking for. I think I will opt for more handheld shooting for some future tests as I can gain more rotational and depth data this way.

Adding the "hello" text

Shoot Day 2

Here are just a few more stills from another day of shooting clean plates to use later for compositing and matchmoving. I have been using the slider much more frequently than originally planned as I feel it truly gives a cinematic look and feel to the footage I am shooting. Here are several more pictures from production.

With regards to gear and settings, here is a list of the camera, glass, and other equipment used:

Gear:
Canon Rebel T4i
Tokina 11-16mm f2.8 Lens
Manfrotto 502AH fluid tripod head
Kessler pocket dolly (Philip Bloom edition)

Camera Settings:
Technicolor Picture Style (cinestyle)
24 fps
f/9-f/13
11mm

Quick reveal shot outside the Library

More bikes, reveal shot outside Travers.

Another angle of the Travers shot.

Path behind the Tennis courts.

Library test shot.

Clean Plate Shoot Session

This week I spent some time shooting clean plate video clips. When shooting, I tried to shoot footage that fit into certain movement criteria. The 3D camera tracker in After Effects often needs some short of movement in order to establish and gain tracking information; if you starve the tracker of movement and depth data, there will be no tracking information for the system to solve. Yet, if the movement is too complex or shaky, the tracker cannot compute the depth and movement information. Therefore, I attempted to shoot footage that fit the following instance criterium:

• handheld pans or tilts
• handheld static shots
• supported slides or dolly shots
• steadicam tracking/moving shots

Next I will plan out more shots and attempt to track my first 3D elements into the live action footage. I have attached some photos of the shoot and the slider I used to achieve the dolly shots.

Tracking shot of bikes, camera mounted on slider.

Another view of the camera and slider.

Still image from the clip.