[Condor-users] [PATCH] Re: Requesting resources with condor_run

[Michael Hanke <michael.hanke@xxxxxxxxx>]

Hi again,

looking into the code of condor_run I couldn't see a way to add
additional specs. I'm attaching a patch that introduces a variable
CONDOR_SUBMIT_SPECS for this purpose. For example:

 CONDOR_SUBMIT_SPECS="RequestMemory = 1024" condor_run sleep 30

 (adding multiple specs also works)

Using another environment variable is, of course, just one way to do it.
I wasn't sure how that was decided in the past, since '-u' is a
commandline option and OPSYS and ARCH are variables. My personal
preference would be a plain commandline option interface, but the
variable was less effort.

What do you think?

Thanks,

Michael



On Mon, Dec 27, 2010 at 07:52:18PM -0500, Michael Hanke wrote:
> Dear Condor experts,
> 
> is it possible to request resources with condor_run? The specific
> problem I'm looking at is that I have a node with a partitionable
> slot. When I use condor_run like
> 
>  $ condor_run sleep 30
> 
> it schedules and runs things fine. However, if I do
> 
>  $ CONDOR_REQUIREMENTS="Memory >= 1024" condor_run sleep 30
> 
> the job stays idle forever.
> 
> While debugging the problem I think I realized what happens: In the
> latter case the job gets matched with such node and spawns a new
> partition of the slot. Since it doesn't have any resource requests it
> assigns 1 MB to the new slot and subsequently realizes that this is not
> enough given the requirements of the job. Consequently the job is
> rejected and the games repeats every minute from there on.
> 
> That would be solvable by having 'RequestMemory = 1024' in the submit
> file that is created by condor_run, but I don't see how that could be
> achieved. Am I missing something?
> 
> 
> Thanks in advance,
> 
> Michael
> 
> -- 
> Michael Hanke
> http://mih.voxindeserto.de

-- 
Michael Hanke
http://mih.voxindeserto.de

From: Michael Hanke <michael.hanke@xxxxxxxxx>
Subject: Allow arbitrary submit specs to be added by condor_run

This is useful if, e.g. jobs need a certain amount of memory. Just adding
'Memory >= X' to the requirements doesn't help in case of partitionable
slots -- wihch initially fulfil the requirements, but later on spawn partitions
might have insufficient resources, unless specified.
diff --git a/src/condor_scripts/condor_run b/src/condor_scripts/condor_run
index 7ca648f..329ea8b 100755
--- a/src/condor_scripts/condor_run
+++ b/src/condor_scripts/condor_run
@@ -112,6 +112,7 @@ if (defined($opsys)) {
     $requirements .= " && " if (defined($requirements));
     $requirements .= "OpSys == \"" . $opsys . "\"";
 }
+$submit_specs = $ENV{'CONDOR_SUBMIT_SPECS'};
 
 # create a shell script containing the user's command
 open(CMD, ">.condor_run.$$") ||
@@ -138,6 +139,9 @@ print JDF "output = .condor_out.$$\n";
 print JDF "error = .condor_error.$$\n";
 print JDF "getenv = True\n";
 print JDF "requirements = ", $requirements, "\n" if (defined($requirements));
+if (defined($submit_specs)) {
+    print JDF "$submit_specs\n";
+}
 print JDF "queue\n";
 close(JDF) ||
     &abort("Failed to write temporary (JDF) file in current directory.\n");