Re: [HTCondor-users] TARGET.Runtime >= RequestRuntime

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

Hi John,

we talked with Todd about this at the condor-week and indeed it would be easiest way to have a blacklist option that we could use to exclude 'requestruntime' from being built in the requirement string on the production pool.

We will then test the new condor 'runtime-option' as a possible replacement of our current solution in the test pool :) 

Best
Chris

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

----- UrsprÃngliche Mail -----
Von: "johnkn" <johnkn@xxxxxxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Montag, 29. Juli 2019 16:41:59
Betreff: Re: [HTCondor-users] TARGET.Runtime >= RequestRuntime

For 8.9.2 and 8.9.3 your options are to rename the attribute, or to have your users add TARGET.Runtime to their submit file Requirements.

Something like this should do. 

Requirements = (TRUE || TARGET.Runtime isnât undefined)

the True ||  part will make the matchmaker prune that subclause before it does any matchmaking, and the mention of TARGET.Runtime
in your submit file Requirements will disable the automatic addition of (TARGET.Runtime >= RequestRuntime).

Or you could  rename your attribute to  RuntimeRequest or perhaps QuotaRuntime.  As the code is currently written,  *anything* that starts with the word Request is treated as a custom resource,  even +Request_Runtime.  (you would end up with a custom resource called _runtime). 
I think I should change this for 8.9.4 so at the very least it won't treat anything with an _ in then name as a valid resource name.

We have been discussing adding something like request_runtime to HTCondor,  so jobs can indicate their expected and/or maximum runtime.  
So your choice of RequestRuntime is very likely to conflict with future HTCondor.   I would consider adding a blacklist for resource names to the submit code if you need us to.    So you could configure

SUBMIT_BLACKLIST_RESOURCE_NAMES = Runtime

And submit would then ignore +RequestRuntime.   

But as I say, this particular name is likely to conflict with future HTCondor work that we are currently discussing, so I would recommend that you just choose a different name for your attribute.

-tj

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Beyer, Christoph
Sent: Monday, July 29, 2019 4:53 AM
To: htcondor-users <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] TARGET.Runtime >= RequestRuntime

Hi John,

did not recognize this e-mail at first, thanks for getting back to us :) 

As you say the behaviour changed with 8.9.2 and a lokal option that we use is now triggering the 'extended' requirement-expression. 

We use "+RequestRuntime = <time in sec>" as a custom thing to classify the job and apply the quota surplus accordingly. 

As the workernodes do not have a default runtime the created requirement-expression is not able to be fullfilled. From your mail I assume I have the following options: 

- rename "+RequestRuntime ..." into something like "+MYRuntime ..." (rather unelegant as we did educate the user to use "+RequestRuntime ...") 
- establish a workernode classadd with unlimited runtime
- put the same runtime into the jobclassadd as is requested, would that be  "RequestRuntime" or "request_runtime" ?

Maybe there is a more elegant way to tinker the requirement expression and get rid of the 'reques_runtime' part ? 

Best
Christoph

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

----- UrsprÃngliche Mail -----
Von: "johnkn" <johnkn@xxxxxxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Dienstag, 23. Juli 2019 21:13:39
Betreff: Re: [HTCondor-users] TARGET.Runtime >= RequestRuntime

In condor_submit,  any keyword that starts with request_  is presumed to be request for a custom resource.  
Everthing after the _ is presumed to be the resource type name

The resource names  Cpus, Disk, and Memory get special treatment.
The resource name Cpu is treated as an error, because that is a common mistake. 

In the examples below, assume Tag is the custom resource name.

so 
request_tag = <expression>

causes

RequestTag = <expression>

to be added to the job ClassAd, and the Requirements expression will have 

 (TARGET.Tag >= RequestTag)

  or 

 regexp(RequestTag, TARGET.TAG)

added to it.  the second expression above is used when <expression > is a string literal.

Starting with 8.9.2.  The Requirements expression will be changed when you use 

  +RequestTag = <expression>

as well as when you use 

  request_tag = <expression>

in your submit file.

if the requirements already references RequestTag,  then it will not be modified.


-tj

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Beyer, Christoph
Sent: Friday, July 19, 2019 5:24 AM
To: htcondor-users <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] TARGET.Runtime >= RequestRuntime

Hi,

any thoughts on this ? 

Best
Christoph

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

----- UrsprÃngliche Mail -----
Von: "Christoph Beyer" <christoph.beyer@xxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Mittwoch, 17. Juli 2019 14:48:18
Betreff: TARGET.Runtime >= RequestRuntime

Hi Todd et al,

this is probably rather a question for the developer or at last those who are aware of the source code I am afraid. 

I do have some user using a python framework and by doing so they trigger the requirement 'TARGET.Runtime >= RequestRuntime' in the r jon requirement which results in stuck jobs as TARGET.Runtime is not set by default on our workernodes.

I had this issue before triggered by somone who by mistake did not put '+RequestRuntime = XXX' in his submit file but 'request_runtime = XXX' which obviously was easy to fix. 

In the current case though I have no idea what triggers 'TARGET.Runtime >= RequestRuntime' to show up in the job requirement - hopefully someone can enlighten me :) (?) 

Best
Christoph

-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/