Rendering Pipeline/Using the Renderfarm

Goal To learn how to use the Renderfarm.

Important Terms

Render: Creates an image of a single frame of your scene by calculating and determining the interaction between all the mesh, lights, and other objects in a scene.

Batch Render: Uses your computer to render your frames. This can slow down your computer significantly and prevent you from doing other work while you wait for it to finish, so rendering using the farm is generally much more efficient for bigger projects.

Render farm: A group of machines located outside the lab that can be accessed remotely and are used to render your frames for you. There are multiple machines, meaning they can all work on your renders at once, instead of batching where only a single computer is doing all the work.

Render layer: Different sets of objects in your scene that can be rendered separately to reduce render time and make it easier to achieve a desired aesthetic look.

Tractor: A Pixar render manager that allows you to monitor the progress of your renders after they've been sent to the farm.

Black Hole Shader: This is what we call a completely black surface shader. The concept behind this is to mask certain things in layers so that once a shot is composited, objects aren't passing through other objects. For example, if a character walks behind a desk, we don't want the character's legs showing through the desk.

NIMBY: Stands for "Not In My BackYard." This restricts a certain render node if it's been dropping frames or erroring out in any way, so that when you send out new renders, they will not be sent to this problem node. NOTE: If you ever do this, email support explaining the problem and CC Barbara. NIMBYing a node restricts it for everyone using the farm, not just you, and not just our class.

Preparing to Render

Creating Render Layers

NOTE: We are using the Render Setup in Maya 2017. If you are still using Legacy render layers then you will need to change it. Go to windows and under settings/preferences, open up the preference editor. Then scroll down to the render menu and change "Preferred Render Setup System" so it says Render Set up(NOT Legacy Render Layers) It can be accessed by going to Windows - Rendering Editors - Render Setup. Or by selecting the Render Setup button

It is dependent upon the needs of the specific shot which layers are created, but there are generally 3 basic layers:

Character Beauty: (contains just character meshes, any props or objects that the characters will be interacting with, and any lights used to light the characters)

Background Beauty: (contains just background objects, and any lights used to light the background)

To create a layer open the Render Setup window and a the window below should pop up. Here is what each part does

A: This is the master layer control layer. This Layer you cannot edit but it holds all of your assets in it, your lights, AOVs and Render Settings. By double clicking on any of the bars in this layer, a window will pop-up that will allow you to edit these attributes.

B: Create a Layer button. This will create a render layer in the window below.

C: Render layers. These are your render layers where you can control the visibility, render-ability and more from inside here.

a. This is the Render Layer. It holds all the assets for that particular layer. It is identifiable by its red bar on the lefthand side.

b. The eyeball icon will allow you to view, specifically that layer. If you want to look at a layer you will hit this so that the renderview will render that layer.

c. The bar with the yellow stripe on the side is a collection. Collections contain whatever assets you want inside them, from lights to characters to backgrounds.

d. The stop sign will hide the collection. If you want to make a certain collection not visible so that it doesn't render you can hit this.

e. The dotted circle with the smaller circle will achieve a result similar to the eye, but it will only isolate the collection instead of the whole layer.

D: This box is where you can add in assets to your collection. You can do this by either selecting the item with the center scroll of your mouse and dragging it into the box. Or by selecting the object in your outliner and then hitting the add button next to the collection box.

E: Attribute overider. This box can help you add specific aspects to your collection, such as shader overides and color overides.

AOV's

AOV's (Arbitrary Output Variables) are extremely helpful in determining where to increase samples/quality in your scene. By double clicking on the AOV bar in the master layer editor, you can access the AOV editor.

In order to activate certain AOV's you can select one from the list on the left and then hit the buttons moving to the right and it will make that AOV active. Now you are able to view just that AOV and figure out how to increase samples accordingly. These layers can also be rendered out just by themselves.

How to Set up your Render Layers

First Create a layer and name it accordingly. Let's say you are creating a render layer that will render out your background but not your characters, so you will name it "background_layer". First you want to add in the necessary characters so right-click your background layer and select "create collection" and name this collection "set". Then you can either add your background and set into the layer by selecting the objects in the outliner and hitting the add button in the render layer window. You can also select the objects you want to add and drag them into the box using the scroll on your mouse.

NOTE: if for some reason certain objects or lights aren't loading properly in your renderview, try selecting the eyeball icon on the masterlayer and then selecting the eyeball icon on the layer you are currently looking editing. This should help maya load the changes you made.

Now that you have your objects in the scene let's add in the lights. First you will go to the lights box in your scene window in the render window and drag it into your render layer. This will add in all the lights automatically, but we want to add in specific lights so you want to hit the stop sign on this to disable all the lights, but still keep the information of all the lights present into your collection.

Now let's add in the lights. The same way you added in the background you can add your lights in that are necessary for this layer. You want to add in lights that aren't linked to the background too, because these often cause bounces and diffuse light in a scene and you still want that light to be visible.

Now you've set up the basics for a render layer!

Part 2: Overrides, Black-hole Shaders, and the Renderfarm

Now you've set up the necessary objects in your layer right? Not quite yet. While you may have the background and necessary lights, you still need to include the objects that can cause shadows and bounce light in your scene, because if they aren't added into your layer they won't render out the bounce light, or shadows that those objects cause. So we are going to create a black-hole matte override on those items so that they don't appear in a render but still cause the shadow and diffuse light effect. To do this you are going to create a collection with those objects in it. For this example let's call it "characters", and add all your necessary characters into the layer.

Next select one of the objects you want to have the black-hole shader on them, right-click and select material attributes. Then go down to matte option and right-click and select "create Absolute Override for Visible Layer". It should create a green bar with and editable menu in the propery editor. From here you can directly affect the objects that have the black-hole shader and enable it or turn it off. This will create a black hold shader that will not render out anything that the override is connected to. Whatever collections are directly above that override will be affected by it.

You have now successfully set up a render layer! These are important because it can save you time from having to re-render a scene. Say your motion changes but not the set, you only need to render the character layer which can save a lot of render time. This also helps a lot for post-production when working in after effects.

Sending out a Render

Once you've created all your layers and made sure everything is working correctly (test each layer before sending it out), Make sure the enable render button is on for every layer that you want to send out, it's time to send out a render. Open up your Render Settings and use these settings:

A: Always render using Arnold

B: Make sure your path is set to the correct location. To do this, go to File, Set Project... and then select your destination folder. If you do not already have a renders folder set up, it will tell you, and you can click Create Default Workspace, and Maya will create one for you. Make sure your file path and any textures you use in your shot always start with //csenetid, NOT the O drive (O:\).

Naming conventions for file paths are extremely important, not just in rendering, but throughout the whole pipeline. Some important conventions to remember when creating folders or file paths:

- Only use underscores, lowercase letters, and numbers

- No spaces

- No special characters

C: Your image format should always be tif. With compression at Izw and format at int8.

D: The Frame/Animation ext is what Maya names each frame of your render as it goes through the job frame by frame. We use this naming convention for consistency.

E: The frame padding adds this number of zeroes to the name of each frame of your render. Again, this is for consistency. Always keep this at 4.

F: The start and end frame are dependent on how long your scene is. Sometimes you may want to render only part of your shot, and in that case your start frame may not be 1.000. If you want to render from the beginning of the shot, always start with frame 1.000, not 0.000.

G: Always render through your render_cam. Make sure you have no other cameras in this section of the render settings. If you do, press the trashcan to the right of the name to delete them.

H: You should use 1280x720 for final renders. Keep in mind, the bigger the image, the longer you have to wait for it to render.

Go into the Arnold Renderer tab:

I: The Sampling menu is where we can adjust different aspects of how certain parts of a scene are rendering. Each slider affects the quality of a different aspect of the scene and how light falls on an object.

- Camera: Acts as an exponential multiplier for all the other samples.

- Diffuse: How light spreads across an object, will usually be higher than most samples.

- Specular: Affects quality of specular highlights on objects

- Transmission: Affects light that passes through transparent objects, such as glass or water.

- SSS: Stands for Sub-Surface Scattering, This is the effect that light has when it contacts an object and some light enters the objects and scatters around inside of the object and then reflects back out. This is most oftenly seen on fingers when it is intensely backlit, you can see through parts of your fingers. This is a more subtle lighting affect but can add a nice effect, especially for close ups.

- Volume Direct: How light scatters and bounces across smoke/fog/particles.

J: Adjust these different sliders as necessary and to achieve the best render time as well. Certain shots will require different amounts of quality in different areas so find what works best before sending a shot out to render.

Once all your settings are correct, make sure that everything in your scene is SMOOTHED. Disable/Enable your render layers so you're sending out only your desired layers.

Sending out the character layer and the background layer together is usually a pretty good bet, since they render pretty quickly. NOTE: If your camera is static (the background does not move) then you DO NOT need to render out the background layer. You can just render out a single frame of the background, save it as an image, and use that when you're compositing. This saves a lot of render time. Sending out the occlusion layer by itself is a good idea, since it usually takes the longest to render of all the layers by far

TEST YOUR RENDERS

Always test any layers you're planning on sending out in Maya first, so you don't end up with a folder full of useless renders.

When you have your settings done, your renders are working, everything is smoothed, the file path is correct, and you have your desired layers enabled for rendering, click the Tractor Button.

K: Frames Per Node is the amount of frames that one node takes on at a time, we usually set this number to 1. When Rendering with Arnold there are times where using a higher number of nodes can cause unusual glitches while your shot renders.

L: Job Priority is the importance of the job in the farm. Normally we can leave this number at 0, but if your render is very important and needs to get done quickly, increasing the priority will bump it to the top of the list in the render farm if there is a backlog of jobs.

M: Renderer is always Arnold.

N: Spool is just the Tractor's term for sending your job to the farm.

When you're done with all of these settings, click Spool. You can click the Tractor Monitor button at the bottom right corner once you've spooled your job to go to the Tractor render manager to monitor your renders' progress.

Tractor Monitor

This render farm manager makes it super easy for you to monitor the progress of your renders. It can be found here.

A: This button takes you to the render log. This is a list of all the renders on the farm, and they can be sorted by many different filters. In this image, all the renders on the screen are finished, so they are grey.

B: This button takes you to a list of the render nodes. We will look at this list below.

C: This is an outline of the different render nodes rendering your job. Each box represents a different group of frames being rendered by a different node. Depending on what you set for your Frames Per Node value, each of these boxes represent a different number of frames.

D: This is an outline of the stats for your job. If you need to bump up your render's priority while it is in the farm, you can change the Job Priority here.

You may find that certain nodes are causing issues for your renders. You can tell which node is rendering which frames by hovering over the node on the right of the screen. We have three main clusters of rendering machines. Their names are buzz, jessie, and woody. Sometimes a certain node can be malfunctioning, and you may not want your render jobs going to that node. You can restrict this node from rendering by turning on NIMBY. By clicking on the button (B) that takes you to the render nodes, you can see a list of all the render nodes.

By right clicking on a node, you can turn NIMBY on, and this will not allow this node to render any job that gets sent to the farm.

Once you've sent a job out, it's also possible to control it within the Tractor Monitor. There are several possible actions to take once your job is on the farm. If you right click on a job on the farm, the following menu will appear:

Pause New Task Launches will stop the render farm from picking up any new nodes within an already active job. This can be useful if you're sending out a large job, and want to force the farm to pace itself by preventing it from starting on new frames until it has finished previous ones.

Restart Job will stop the currently active job, and restart it from the beginning. This is sometimes a good idea to try if you're noticing errors occurring on the farm.

Archive (Delete) Job will delete the job from the farm. Sometimes this is necessary to do if a job is taking too long and using up all the render machines so other jobs can't get through.