Re: [Condor-users] Condor for a renderfarm

[Jason Stowe <jstowe@xxxxxxxxxxxxxxxxxx>]

Campbell,
Absolutely, Condor is a great way to manage a render farm. Entire
films have been rendered on Condor pools:
http://www.cs.wisc.edu/condor/images/WildLetter.pdf

Condor's flexible policy configuration, ability to scavenge available
resources, manage dependencies through DAGMan, and configurable
prioritization (like license limits, quotas for particular projects,
and various priority features), make it great for this purpose. There
are also easy ways to use Condor on the cloud to run renderers like
blender that don't have license considerations.

With regard to the concerns about file transfer, if you have a shared
file system this is a non-issue (you don't need to transfer files). If
you don't have a shared file-system,  Condor doesn't slow down the
transfers you'll have to do anyway.

>From an advice perspective, I'll try give a few highlights. It is easy
to get started with Condor, and as mentioned earlier, through the
scripting that is common in 3d rendering pipelines, Condor will enable
automated control over your pipeline and job priorities. I worked on a
feature film that used Condor, and Cycle has customers that use Condor
for video game/film/vfx render farm today.

First, a key thing with rendering is frequently prioritization and
dependencies. Condor's DAGMan has a great set of features for managing
dependencies between jobs. You can create your geo's, render shadows,
render layers, and composite final frames all in one workflow, or DAG
(directed acyclic graph), with pre/post scripts to handle
errors/checking.

For priorities, rather than piece-meal changing
PREEMPTION_REQUIREMENTS, RANK and other expressions, write down the
priority policies you want to have for job quotas, job priority,
job/machine preferences, and workstation availability. These might
include:
-I want to account for use of my pool based upon the project( or user
or sequence/shot, or department ) that the job is for,
-I want to have minimum quotas for each of the 3 projects I have so
they are guaranteed capacity,
-I want the first, middle, and last frames of my submissions to have
higher priority and render first, so I can check them
-One processor of my 4-core workstations will always render jobs on
the queue, and other processors will only be available if the
keyboard/mouse are idle for 30 minutes
-I will have two types of jobs, normal renders, and low-res previews.
Low-res previews should run immediately, but never run for more than
10 minutes.
-I have 50 licenses to plug-in X, so never run more than that many
jobs for that plug-in
-Jobs tagged as isURGENT=True should take over the farm.
All of these policies are easily specified in Condor, and you'll find
it easier to implement policies if you have them written out and
quantified ahead of time.

Cycle has considerable experience with this, management tools that can
help you with submission pipelines/priorities, and I'll touch base
with you about other specific advice. Meanwhile, I hope this helps,
and good luck!

Best,
Jason

-- 
===================================
Jason A. Stowe
cell: 607.227.9686
main: 888.292.5320

Cycle Computing, LLC
Leader in Condor Grid Solutions
Enterprise Condor Support and Management Tools

http://www.cyclecomputing.com
http://www.cyclecloud.com


On Thu, Aug 6, 2009 at 11:49 AM, Preston Smith<psmith@xxxxxxxxxx> wrote:
> There's many folks who've used Condor for rendering, it's very well
> suited for it.
>
> I'm sure somebody from Cycle Computing may chime in, they have a lot
> of experience in this..
>
> We (Purdue) have the TeraDRE available to TeraGrid users:   http://teradre.rcac.purdue.edu/
> Indiana University has RenderPortal that is used by their students:    http://www.avl.iu.edu/?RenderPortal
>
>
> -Preston
>
> On Aug 5, 2009, at 7:33 PM, Campbell Barton wrote:
>
>> Hi there, (first post here)
>> I'm looking at options for running a renderfarm with Blender3D.
>>
>> The last renderfarm I setup used sungrid but was far too complicated
>> to manage IMHO.
>> http://www.bigbuckbunny.org/index.php/our-renderfarm-and-how-it-works/
>>
>> I have seen a post that shows condor has been used with blender before
>> but would like to know how condor goes with larger filesets.
>> ref: http://blenderartists.org/forum/archive/index.php/t-102226.html
>>
>> Details
>> - all local PCs (our own renderfarm this time :) )
>> - Modifying blender to better work with condor is acceptable, or
>> building against other libs.
>> - only needs to run blender.
>> - renderfarm will be 10-20 quad core PC's, all the same hardware.
>> - Linux 64bit.
>> - 4-16gig of ram
>> - Condors suspend/resume/migration isn't important to us.
>> - Animation will be split up into one job per frame, shot would be
>> roughly 100-1000 frames.
>> - ~30min per frame
>> - For rendering a frame each node will need to load around 1-2gb over
>> a gigabit (maybe 10gigabit) network. each frame will be between
>> 20-80mb.
>> - Rendering takes up so much ram that I don't think it can be done on
>> idle workstations (maybe overnight on artists workstations)
>>
>> Does this seem like a good use for condor?
>>
>> - Is there any linux distro that is better suited for condor? (was
>> thinking centos/debian)
>> - Is the data that needs to be read for each frame likely to be a
>> problem (assuming that normal file copy isn't prohibitively slow).
>> - Has anyone here used condor for a renderfarm before? is there any
>> advice you can give?
>>
>> If I don't use condor or a similar system Ill probably hack together a
>> python script that runs the jobs over ssh, since we're not dealing
>> with 100's of jobs running at once its not so complicated.
>> just wanted to get a feel for weather this is something condor is well
>> suited for.
>>
>> Thanks!
>> --
>> - Campbell
>> _______________________________________________
>> Condor-users mailing list
>> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
>> with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/condor-users/
>
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
>