Re: [condor-users] Dagman job submission problem

[Jaime Frey <jfrey@xxxxxxxxxxx>]

On Wed, 10 Dec 2003, Antti Eskola wrote:

> If anyone has used this initialdir with dagman jobs pls give me your
> working example, THX!
>
> (TOPIC: how to use initialdir function + dagman)
>
> I have problem when i am submitting a job using dagman.
> First my job A uses Initialdir function because i run same program
> but have 1...20000 input files.
> (before i tried adding job B to run program gatherer.sh, which gathers
> from those 1...20000 output files one file and sends it with email)
> Now i am trying to use POST function in dagman script.
> Problem: dagman sends always many,many,endlessly or something like that
> number of jobs and i think it is related to log file problem.

You can restrict the number of jobs DAGMan will submit at a time with the
-maxjobs paramter to condor_submit_dag.

> I have tried to not to use log at all, then it job fails completely and
> if i use like this define one log to all submitted A macro tasks dagman
> somehow has problems and does not know when the job A is finished.
>
> Here are my files: ( I call condor_submit_dag -l log.txt submit.dag)
> # submit.a #
> Executable              = math.sh
> Universe                = vanilla
> Arguments               = prefixPATTERNlen.m
> Requirements            = OpSys == "LINUX" && Arch == "INTEL"
> should_transfer_files   = YES
> input                   = input
> transfer_input_files    = ../prefixPATTERNlen.m,../vmmPolyTests.m
> transfer_output_files   = out,outFull
> WhenToTransferOutput    = ON_EXIT
> Getenv                  = TRUE
> Initialdir              = $(PROCESS)
> log                     = ../log.txt
> Queue 6
> # submit.dag #
> JOB     A       submit.a
> SCRIPT  POST    A gatherer.sh

DAGMan requires that each submit description file submit only one job.
However, there are tricks you can use so that you don't have to write
20000 submit description files. You can find this and more information in
the DAGMan section of the Condor manual:
http://www.cs.wisc.edu/condor/manual/v6.6/2_11DAGMan_Applications.html

+------------------------------------+-------------------------------+
|             Jaime Frey             |There are 10 types of people in|
|         jfrey@xxxxxxxxxxx          |the world: Those who understand|
|   http://www.cs.wisc.edu/~jfrey/   |  binary, and those who don't  |
+------------------------------------+-------------------------------+
Condor Support Information:
http://www.cs.wisc.edu/condor/condor-support/
To Unsubscribe, send mail to majordomo@xxxxxxxxxxx with
unsubscribe condor-users <your_email_address>