Re: [HTCondor-users] Determine when all jobs in a cluster have finished?

["Smithies, Russell" <Russell.Smithies@xxxxxxxxxxxxxxxx>]

I agree, except s/Java/Perl/g

The Condor Perl api might be a good way of doing this?

http://research.cs.wisc.edu/htcondor/manual/v7.9/4_5Application_Program.html#SECTION00557000000000000000

 

--Russell

 

 

From: htcondor-users-bounces@xxxxxxxxxxx [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Michael Aschenbeck
Sent: Thursday, 31 January 2013 8:04 a.m.
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] Determine when all jobs in a cluster have finished?

 

DAGman is certainly an awesome tool...but if you have a java master process that is creating these submit files, I think it's over-kill.  I guess I could be missing something, but it seems like your master process should be like this:

In Java code:
1) create initial submit file and submit it (already doing this)
2) make a system call to condor_wait to wait for the initial job to finish

3) Have java check which data files came back, create submit files for post processing, and submit

4) Use condor wait to wait for those jobs (if necessary)

 

I just think that if you are using Java to create these scripts automatically and fire all this stuff off, that you might as well use that for the "dynamic" aspect.  I do this sort of thing in C++ quite a bit and, while I know DAGman and have used it, when i don't have crazy job dependencies it's much easier to just to a condor_wait as a system call in c++ and then go to setting up a post process script. Please let me know if I'm missing something or if you are familiar with condor_q and it's really just not what you want, but I think this should be considered.

 

On Wed, Jan 30, 2013 at 11:34 AM, Dimitri Maziuk <dmaziuk@xxxxxxxxxxxxx> wrote:

On 01/30/2013 12:27 PM, Dimitri Maziuk wrote:

Sorry,

Vars Workers$NUM WORKDIR="/workspace/jobs/$NUM" \
            ^^^^

--
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

 

=======================================================================
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
=======================================================================