One of the problems with the basic Maya Software Renderer is that it sometimes ignores the way that light behaves in the real world. As such, objects like the one below can tend to look flat and artificial. One way to increase the realism of a shot is to add an occlusion pass. What occlusion does is calculate the areas where it is unlikely that light would reach and makes them darker. However, the reason that Maya doesn't give you occlusion by default is that it is extremely expensive to calculate. For a complex scene, rendering can take up to 30 minutes per frame.
Setting up a occulsion pass is fairly simple. Go the Layer Editor and click the Render radio button. Create a new render layer and then Right-click it, selecting Presets → Occlusion from the menu. This should turn your geometry white. Rendering on this layer will produce an occlusion pass instead of the normal render. By default, the quality of the occlusion isn't great, as seen by the image on the left. It looks grainy and spreads farther than it should.
You can edit the occlusion by clicking the material override button (
) on the render layer. Follow the Out Color mapping of the Surface Shader that appears. This will bring you to the occlusion node. The three attributes of interst are Samples, Spread, and Max Distance. These values will determine the quality and speed of the occlusion. Set Samples to 250, Spread to 0.5, and Max Distance to 3. If you re-render, you should see that the occlusion is much less grainy and is focused around the nooks and cranies of the geometry.
An occlusion pass isn't much good by itself. Once you have rendered it out, it must be composited with your regular render. Typically this is done in After Effects with an entire series of frames, but for brevity, we will composite a single frame in Photoshop. Place your color on one layer and the occlusion on a layer above it. Set mode of the occlusion layer to Multiply. Simple as that. The difference is subtle, but occlusion adds a lot of richness to a render. The renders below show the difference between a normal render and a render with composited occlusion.
It would be nice to only incur the cost of occlusion once, not for every frame. Lucky for us, Maya provides a process called baking that will allow us to generate an occlusion texture. The first step is to lay out the UVs of your object. We will use an Automatic Mapping for the sake of speed. Go to Window → Settings/Preferences → Plug-in Manager. Make sure that the RenderMan_for_Maya.mll plug-in is Loaded.
In the Polygons menu set, go to Color → Batch Bake (mental ray) (Options). Set Objects to bake to Selected, Bake to Texture, and check the Use bake set override box. Change the Color mode to Occlusion and Occlusion rays 250. Verify that the File format is TIFF and the X and Y resolution match the dimensions of your color map. Click Convert and Close to begin the process. This will lock up Maya for a while, but eventually it should finish.
A new occlusion texture should be applied to the object you selected. The new texture should reside somewhere in your default project. If you need help finding it, follow the Incandescence mapping to the File node. If you click View, it should open fcheck, from which you can save the texture somewhere reasonable. All you need to do now is to multipy the occlusion texture on top of your color map. One benefit of doing occlusion per frame is that it can account for the occlusion that appears when objects get near each other, something static occlusion doesn't reflect.
