Re: [Condor-users] submitting LOTS of jobs

[Mag Gam <magawake@xxxxxxxxx>]

Do you have an example of a DAGMan job for a situation like this?

Lets say he would only like to have 10 jobs running and 10 jobs idle
while still trying to resolve 1000000000. Is that possible to do ?





On Wed, Nov 4, 2009 at 1:38 PM, Steven Timm <timm@xxxxxxxx> wrote:
> Most of the schedd submit-side limits deal with how many jobs
> are running at any given time, not how many are in the queue at all.
> DAGman or one of the newer workflow managers should be used to control
> the workflow and the job submissions.  You might want to look at
> Falkon/Swift from the U of Chicago, they have been doing a lot of work
> like this recently although things seem to be working best for them
> when they run on a BlueGene.
>
> Steve
>
>
> On Wed, 4 Nov 2009, Jonathan D. Proulx wrote:
>
>> Hi All,
>>
>> I have a user looking to submit 15million jobs this is about two
>> orders of magnitude above what weve done previously.  I don't yet know
>> too much about the calculation other than it's genome based and
>> running against the human genome (a later goal) would result in
>> 1.8 billion jobs.
>>
>> leaving aside for a monent the question of how long this will take on
>> my resources, any advise on facilitating this number of jobs?  Are
>> there submit side limits I'm going to run into?
>>
>> Thanks
>> -Jon
>> _______________________________________________
>> Condor-users mailing list
>> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/condor-users/
>>
>
> --
> ------------------------------------------------------------------
> Steven C. Timm, Ph.D  (630) 840-8525
> timm@xxxxxxxx  http://home.fnal.gov/~timm/
> Fermilab Computing Division, Scientific Computing Facilities,
> Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader.
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
>