Re: [Condor-users] Condor for a renderfarm

[Campbell Barton <ideasman42@xxxxxxxxx>]

Thanks so much guys for the info,
Ill try setup a small renderfarm on 3-4 PC's at my place first and see
how it goes.

I looked into drqueue a while ago but could crash its GTK-GUI app
within 2min or so (stable and debug version), and found leaks in its
py api.
Sungrid is too 'cloud' oriented with a high administration overhead,
when I used it last (20'000 cpu hr project), I ended up having to
write my own queuing since submitting every job I needed would hang
suns queuing system and close all running jobs.

Condor seems to give enough flexibility without too much setup, am
very happy to have found it

What interests me most is the ability to manage rendering lower
priority the movie shots at the same time as artists being able to
send test renders to the farm so artists can setup lighting/materials
etc without having to render on their own systems.

The project wont start for a month or 2, so Ill lurk on this list for
a bit and do some testing.
Thanks again!

On Fri, Aug 7, 2009 at 6:37 AM, Jason Stowe<jstowe@xxxxxxxxxxxxxxxxxx> wrote:
> Campbell,
> Absolutely, Condor is a great way to manage a render farm. Entire
> films have been rendered on Condor pools:
> http://www.cs.wisc.edu/condor/images/WildLetter.pdf
>
> Condor's flexible policy configuration, ability to scavenge available
> resources, manage dependencies through DAGMan, and configurable
> prioritization (like license limits, quotas for particular projects,
> and various priority features), make it great for this purpose. There
> are also easy ways to use Condor on the cloud to run renderers like
> blender that don't have license considerations.
>
> With regard to the concerns about file transfer, if you have a shared
> file system this is a non-issue (you don't need to transfer files). If
> you don't have a shared file-system,  Condor doesn't slow down the
> transfers you'll have to do anyway.
>
> >From an advice perspective, I'll try give a few highlights. It is easy
> to get started with Condor, and as mentioned earlier, through the
> scripting that is common in 3d rendering pipelines, Condor will enable
> automated control over your pipeline and job priorities. I worked on a
> feature film that used Condor, and Cycle has customers that use Condor
> for video game/film/vfx render farm today.
>
> First, a key thing with rendering is frequently prioritization and
> dependencies. Condor's DAGMan has a great set of features for managing
> dependencies between jobs. You can create your geo's, render shadows,
> render layers, and composite final frames all in one workflow, or DAG
> (directed acyclic graph), with pre/post scripts to handle
> errors/checking.
>
> For priorities, rather than piece-meal changing
> PREEMPTION_REQUIREMENTS, RANK and other expressions, write down the
> priority policies you want to have for job quotas, job priority,
> job/machine preferences, and workstation availability. These might
> include:
> -I want to account for use of my pool based upon the project( or user
> or sequence/shot, or department ) that the job is for,
> -I want to have minimum quotas for each of the 3 projects I have so
> they are guaranteed capacity,
> -I want the first, middle, and last frames of my submissions to have
> higher priority and render first, so I can check them
> -One processor of my 4-core workstations will always render jobs on
> the queue, and other processors will only be available if the
> keyboard/mouse are idle for 30 minutes
> -I will have two types of jobs, normal renders, and low-res previews.
> Low-res previews should run immediately, but never run for more than
> 10 minutes.
> -I have 50 licenses to plug-in X, so never run more than that many
> jobs for that plug-in
> -Jobs tagged as isURGENT=True should take over the farm.
> All of these policies are easily specified in Condor, and you'll find
> it easier to implement policies if you have them written out and
> quantified ahead of time.
>
> Cycle has considerable experience with this, management tools that can
> help you with submission pipelines/priorities, and I'll touch base
> with you about other specific advice. Meanwhile, I hope this helps,
> and good luck!
>
> Best,
> Jason
>
> --
> ===================================
> Jason A. Stowe
> cell: 607.227.9686
> main: 888.292.5320
>
> Cycle Computing, LLC
> Leader in Condor Grid Solutions
> Enterprise Condor Support and Management Tools
>
> http://www.cyclecomputing.com
> http://www.cyclecloud.com
>
>
> On Thu, Aug 6, 2009 at 11:49 AM, Preston Smith<psmith@xxxxxxxxxx> wrote:
>> There's many folks who've used Condor for rendering, it's very well
>> suited for it.
>>
>> I'm sure somebody from Cycle Computing may chime in, they have a lot
>> of experience in this..
>>
>> We (Purdue) have the TeraDRE available to TeraGrid users:   http://teradre.rcac.purdue.edu/
>> Indiana University has RenderPortal that is used by their students:    http://www.avl.iu.edu/?RenderPortal
>>
>>
>> -Preston
>>
>> On Aug 5, 2009, at 7:33 PM, Campbell Barton wrote:
>>
>>> Hi there, (first post here)
>>> I'm looking at options for running a renderfarm with Blender3D.
>>>
>>> The last renderfarm I setup used sungrid but was far too complicated
>>> to manage IMHO.
>>> http://www.bigbuckbunny.org/index.php/our-renderfarm-and-how-it-works/
>>>
>>> I have seen a post that shows condor has been used with blender before
>>> but would like to know how condor goes with larger filesets.
>>> ref: http://blenderartists.org/forum/archive/index.php/t-102226.html
>>>
>>> Details
>>> - all local PCs (our own renderfarm this time :) )
>>> - Modifying blender to better work with condor is acceptable, or
>>> building against other libs.
>>> - only needs to run blender.
>>> - renderfarm will be 10-20 quad core PC's, all the same hardware.
>>> - Linux 64bit.
>>> - 4-16gig of ram
>>> - Condors suspend/resume/migration isn't important to us.
>>> - Animation will be split up into one job per frame, shot would be
>>> roughly 100-1000 frames.
>>> - ~30min per frame
>>> - For rendering a frame each node will need to load around 1-2gb over
>>> a gigabit (maybe 10gigabit) network. each frame will be between
>>> 20-80mb.
>>> - Rendering takes up so much ram that I don't think it can be done on
>>> idle workstations (maybe overnight on artists workstations)
>>>
>>> Does this seem like a good use for condor?
>>>
>>> - Is there any linux distro that is better suited for condor? (was
>>> thinking centos/debian)
>>> - Is the data that needs to be read for each frame likely to be a
>>> problem (assuming that normal file copy isn't prohibitively slow).
>>> - Has anyone here used condor for a renderfarm before? is there any
>>> advice you can give?
>>>
>>> If I don't use condor or a similar system Ill probably hack together a
>>> python script that runs the jobs over ssh, since we're not dealing
>>> with 100's of jobs running at once its not so complicated.
>>> just wanted to get a feel for weather this is something condor is well
>>> suited for.
>>>
>>> Thanks!
>>> --
>>> - Campbell
>>> _______________________________________________
>>> Condor-users mailing list
>>> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
>>> with a
>>> subject: Unsubscribe
>>> You can also unsubscribe by visiting
>>> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>>
>>> The archives can be found at:
>>> https://lists.cs.wisc.edu/archive/condor-users/
>>
>> _______________________________________________
>> Condor-users mailing list
>> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/condor-users/
>>
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
>



-- 
- Campbell