Render Farm tips

Quick tips for using render farm


More details on important issues regarding the render farm

By this point most of you are familiar with the new "Farm Render" menu option in Maya. That is on the surface what you need to do. But there are a number of gotchas involved, and ways to make life easier.
  1. Use the right file format.

    We are best prepared to handle frames rendered as Targa (TGA) files. Except in special cases (none known yet), we do not need alpha channels. Alpha channels aren't inherently bad, they just take up extra space and make loading frames slower. Set this in Render Globals.

  2. Use the right frame numbering convention.

    Please use "name.##.ext" for frame naming. You should also pick which frames you want rendered. All this gets set in the render globals.

  3. Other render globals issues

    Image resolution in render globals is yours to control. Likewise anti-aliasing.

    Frame padding (which adjusts whether frames are written out as foo.1.tga or foo.000001.tga) is automatically overridden to be 6 by the farm. So telling it anything else won't change it.

    Frame renumbering doesn't do very well when going through the farm. Best to not use it.

  4. Make sure all your textures are in the right spot.

    All shaders that use file textures should be looking in z:\cse490ca_finalproject\textures\ for the textures. If you still have any referenced in your private home directories, we are going to start playing darts on your backside. >:}

    Seriously, this is about to become hyper-critical. Not too long from now we are going to be running not one but TWO labs full of render machines. Tracking down textures in the masses of the 50 or 60 gigabytes and thousands of files on Z: does not seem pleasant to me, since each and every texture will need duplicated to the new server. Having all textures in the textures directory is a lot more reasonable.

  5. Submit the job with a UNIQUE FILENAME!!!

    When you pick the "Farm Render" option, it asks you where to save. Anywhere on Z: is good (except in the textures directory). All the rendered frames will go here.

    VERY IMPORTANT: You must MUST, MUST! give each submitted job a different filename. Like if I told the thing "foo.mb" when I hit farm render, the next time I did a farm render I must give it a different name. "bar.mb" is good. "foo2.mb" is also good. "foo.mb" is baaaad. Deleting the original "foo.mb" and saving a new one is not enough. You must use a totally different name.

    Failure to do this can cause wierd effects where half the frames render with the new version of the file, and half the frames render with the old version. Yes, even if you deleted the old version. :)

    Uh huh. We'll fix this bug sometime. :)

  6. Use only one period in the filename

    When you submit a job, naming it "foo.mb" will work. Naming it "foo.preview.mb" will not. The farm doesn't like filenames with more than one period.

    Uh huh, we'll fix this too. :)

  7. Watch the status page

    Right underneath the "Farm Render" menu is another option to bring up the farm status page. You will find it interesting. It gives you power.

    You can check the machines to see which machine is working on what job. If you have friends, you can watch their jobs too. If you have enemies, you can move your render "up" until it is above their jobs. The things at the top get rendered first. As machines finish frames, they will start the next frame of whatever job is at the top of the status list.

    You can also delete jobs here. So if you realize a sudden "Oh SH**! I just told it to render 600 frames full quality when I wanted only 30 frames at preview quality!" you can delete your job. Deleting other people's jobs is not very nice.

  8. The extra job?

    To use correct terminolgy for the first time in this e-mail, when you tell the farm to render something, that is actually a "bag." Each frame in the bag is a "job." On the status screen you might notice there is one more job than you asked for. Like if I ask for frames 1-30 to be rendered, my bag will show up as having 31 jobs.

    This last job is the little pop-up window you get at the end of the render that says the thing is done. Likewise if you see that a machine took 0 or 1 seconds to process a job, it was probably assigned the task of sending one of these little pop-up messages. Just so you know.

  9. FRONTSEAT

    At the moment, the computer next to the door (with Deep Paint on it) is rendering very very slowly. Like, what other machines might take a few minutes per frame, it might take 20 or more minutes for a single frame.

    If you have a very long render, let it run. It still helps. Even one frame every 20 minutes is an extra frame done.

    On the other hand, especially with short renders, what often happens is that every frame will get done except the one FRONTSEAT is working on, and then you wait 20 or so minutes for it before you can look at the results. In this case, pulling FRONTSEAT out of the farm is much better, because it's just slowing you down.

    To do so, just shut it down, or log in and screen lock. (I prefer the first solution, both are fine.) Don't worry if it is rendering a frame when you do this, the farm will detect that and reassign the frame to another computer.

  10. Delay Times

    Any machine that is in the farm shows up on the farm status page.

    It takes 1 minute or so after you log off a computer before that machine is added back into the farm.

    It takes 5 minutes after you reboot (power cycle) a machine before it is added back into the farm.

    There also exists the possibility that the render client might have detected an error on some machine and stopped running. So if you see a machine that you know should be in the farm (it's been on and logged out for a while) but it doesn't show up in the status page, just power cycle it. This is a rare case, and I really don't expect you to be going around looking for it, just something to keep in mind if you're pacing the floor and waiting for frames and you're convinced the farm is going extra slow just because you're in a hurry.

    To help you check, in fact, I have pinned on the wall a map of every computer in LA2 and what its name is. :)