Re: [Condor-users] [PATCH] Re: Requesting resources with condor_run

[Matthew Farrellee <matt@xxxxxxxxxx>]

Would something more like the attached work for you?

It adds -a in the style of condor_submit's -a[ppend].

Best,


matt

On 12/28/2010 08:53 AM, Michael Hanke wrote:
> Hi again,
>
> looking into the code of condor_run I couldn't see a way to add
> additional specs. I'm attaching a patch that introduces a variable
> CONDOR_SUBMIT_SPECS for this purpose. For example:
>
>   CONDOR_SUBMIT_SPECS="RequestMemory = 1024" condor_run sleep 30
>
>   (adding multiple specs also works)
>
> Using another environment variable is, of course, just one way to do it.
> I wasn't sure how that was decided in the past, since '-u' is a
> commandline option and OPSYS and ARCH are variables. My personal
> preference would be a plain commandline option interface, but the
> variable was less effort.
>
> What do you think?
>
> Thanks,
>
> Michael
>
>
>
> On Mon, Dec 27, 2010 at 07:52:18PM -0500, Michael Hanke wrote:
> > Dear Condor experts,
> >
> > is it possible to request resources with condor_run? The specific
> > problem I'm looking at is that I have a node with a partitionable
> > slot. When I use condor_run like
> >
> >   $ condor_run sleep 30
> >
> > it schedules and runs things fine. However, if I do
> >
> >   $ CONDOR_REQUIREMENTS="Memory>= 1024" condor_run sleep 30
> >
> > the job stays idle forever.
> >
> > While debugging the problem I think I realized what happens: In the
> > latter case the job gets matched with such node and spawns a new
> > partition of the slot. Since it doesn't have any resource requests it
> > assigns 1 MB to the new slot and subsequently realizes that this is not
> > enough given the requirements of the job. Consequently the job is
> > rejected and the games repeats every minute from there on.
> >
> > That would be solvable by having 'RequestMemory = 1024' in the submit
> > file that is created by condor_run, but I don't see how that could be
> > achieved. Am I missing something?
> >
> >
> > Thanks in advance,
> >
> > Michael
> >
> > --
> > Michael Hanke
> > http://mih.voxindeserto.de
>
>
>
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/

>From 1ca095299d9be96907c4b318c998d2f8ad143003 Mon Sep 17 00:00:00 2001
From: Matthew Farrellee <matt@xxxxxxxxxx>
Date: Wed, 5 Jan 2011 10:52:29 -0500
Subject: [PATCH] Add -a to condor_run in the style of condor_submit's -append

Additionally, the shell-cmd is now pieced together from all non dash (-u/-a/-h) arguments on the command line.
---
 src/condor_scripts/condor_run |   62 +++++++++++++++++++++++++---------------
 1 files changed, 39 insertions(+), 23 deletions(-)

diff --git a/src/condor_scripts/condor_run b/src/condor_scripts/condor_run
index 7ca648f..3a9b209 100755
--- a/src/condor_scripts/condor_run
+++ b/src/condor_scripts/condor_run
@@ -40,35 +40,48 @@
 #
 ###########################################################################
 
-# possible universe argument (don't "use Getopt::Std")
-# must be first thing before shell command
-$universe = 'vanilla';
-if ( defined $ARGV[0] ) {
-    if ( $ARGV[0] eq '-u' && defined $ARGV[1] ) {
-	shift;
-	$universe = shift;
-    } elsif ( $ARGV[0] =~ /^-u(.*)/ ) {
-	$universe = $1;
-	shift;
+my @command;
+my @appends;
+my $universe = 'vanilla';
+if (!defined($ARGV[0])) {
+    goto HELP;
+}
+while ( $_ = shift( @ARGV ) ) {
+  SWITCH: {
+      if ( /^-h.*/ ) {
+	HELP:
+	  print "usage: $0 [-u <universe>] [-a command]* \"shell-cmd\"\n";
+	  print "\twhere shell-cmd is any Unix shell statement.\n";
+	  print "\t-u lets you set the universe to which the job is submitted.\n";
+	  print "\t   The default is vanilla.\n";
+	  print "\t-a lets you add additional submit commands\n";
+	  print "\t   e.g. -a request_memory=1024\n";
+	  print "\tEnvironment variables CONDOR_ARCH, CONDOR_OPSYS, and\n";
+	  print "\tCONDOR_REQUIREMENTS may specify remote machine's Arch,\n";
+	  print "\tOpSys, and any additional requirements.\n";
+	  exit 1;
+      }
+      if ( /^-u(.*)/ ) {
+          $universe = $1 ? $1 : shift(@ARGV);
+          next SWITCH;
+      }
+      if ( /^-a.*/ ) {
+          push(@appends, shift(@ARGV));
+          next SWITCH;
+      }
+
+      # Anything passed in that isn't -u or -a is part of the command to run
+      push(@command, $_);
     }
 }
+if (!@command) {
+    goto HELP;
+}
 
 # grab current working directory for initial dir in system using automounter
 $pwd = `pwd`;
 chomp $pwd;
 
-# check arguments
-if (!defined($ARGV[0]) || $ARGV[0] eq "-h" || $ARGV[0] eq "-help") {
-    print "usage: $0 [-u <universe>] \"shell-cmd\"\n";
-    print "\twhere shell-cmd is any Unix shell statement.\n";
-    print "\t-u lets you set the universe to which the job is submitted.\n";
-    print "\t   The default is vanilla.\n";
-    print "\tEnvironment variables CONDOR_ARCH, CONDOR_OPSYS, and\n";
-    print "\tCONDOR_REQUIREMENTS may specify remote machine's Arch,\n";
-    print "\tOpSys, and any additional requirements.\n";
-    exit 1;
-}
-
 # set up environment for running something in the current directory in case
 # they want to run something in the current working directory and they
 # don't specify a "./" infront of it.
@@ -117,7 +130,7 @@ if (defined($opsys)) {
 open(CMD, ">.condor_run.$$") ||
     &abort("Can't create temporary (CMD) file in current directory.\n");
 print CMD "#!", $shell, "\n";
-foreach $arg (@ARGV) {
+foreach $arg (@command) {
     print CMD $arg, " ";
 }
 print CMD "\n";
@@ -138,6 +151,9 @@ print JDF "output = .condor_out.$$\n";
 print JDF "error = .condor_error.$$\n";
 print JDF "getenv = True\n";
 print JDF "requirements = ", $requirements, "\n" if (defined($requirements));
+foreach my $append (@appends) {
+    print JDF $append . "\n";
+}
 print JDF "queue\n";
 close(JDF) ||
     &abort("Failed to write temporary (JDF) file in current directory.\n");
-- 
1.7.3.4