Re: [Condor-users] condor_submit feature request

[Steffen Grunewald <steffen.grunewald@xxxxxxxxxx>]

On Wed, Aug 06, 2008 at 09:38:54AM +0200, Steffen Grunewald wrote:
> I'd like to ask for a feature related to condor_submit.
> 
> Every now and then, I have to run a large job cluster with several 100s
> to 1000s of jobs, each using $(Process) as the argument to the executable
> (simply a counter, used as a seed for a RNG).
> Often I'd like to split the large cluster into pieces submitted separately,
> or extend an existing cluster.
> What condor_submit is lacking is a way to start counting at a process number
> > 0.

Following up myself... see below.

On Wed, Aug 06, 2008 at 07:43:02AM -0500, Nick LeRoy wrote:
> On Wed August 6 2008, Ryan Lewis wrote:
> > I would like to see support for the basic math operations ($(Process)
> > +12)*3/400
> 
> While we're discussing it, I've felt for a long time that something like this 
> could be useful (sprintf-ish):
> 
> output = %03d.out, ($(Process)+12)*3/400
> error  = %03d.err, ($(Process)+12)*3/400
> log    = %03d.log, ($(Process)+12)*3/400

Nick:

That would be nasty because it decouples the number of the corresponding file
from the number of the process. In your scheme, even multiple processes would
try to write to the same files!

What about an extension of the "Queue" syntax to
	Queue <number_of_jobs> <starting_process_number> <step_width>
which would be close to the "seq" syntax? 
No additional keywords, backwards compatibility. Nothing sacrificed, a lot gained.


And BTW, what also would be very useful: a user-imposed limit on simultaneously
running jobs from the same cluster ("I want to be nice to my colleagues, n slots
are enough even if there are more available - because I know other's work is
more important and I hate preemption")

Steffen

-- 
Steffen Grunewald * MPI Grav.Phys.(AEI) * Am Mühlenberg 1, D-14476 Potsdam
Cluster Admin * http://pandora.aei.mpg.de/merlin/ * http://www.aei.mpg.de/
* e-mail: steffen.grunewald(*)aei.mpg.de * +49-331-567-{fon:7233,fax:7298}
No Word/PPT mails - http://www.gnu.org/philosophy/no-word-attachments.html