Re: [HTCondor-users] PREEMPTION_RANK undefined variables

[Jaime Frey <jfrey@xxxxxxxxxxx>]

That would explain things. When doing pslot preemption (enabled by setting ALLOW_PSLOT_PREEMPTION=True), the negotiator considers preempting one or more dslots to give enough resources to the pslot to match a candidate job. PREEMPTION_REQUIREMENTS and PREEMPTION_RANK should to be evaluated for the dslots. The negotiator does this properly for PREEMPTION_REQUIREMENTS, but it evaluates PREEMPTION_RANK against the pslot (which has no current running job and thus no RemoteUserPrio or TotalJobRunTime).

Handling PREEMPTION_RANK properly for pslot preemption is ambiguous. The negotiatorâs overall algorithm for ranking potential matches is to do a global (i.e. pool-wide) sorting. For pslot preemption matches, that would require aggregating multiple dslot PREEMPTION_RANK values into a single value for the pslot+dslots match. The current code cowardly ignores the problem.

 - Jaime

On Nov 10, 2021, at 1:31 PM, duduhandelman@xxxxxxxxxxx wrote:

Sam. 

If I recall correctly you are using pslots. 

So the negotiation is "job vs pslot".

Pslot is limited and doesn't have those attributes.

Without pslot its "job vs job" 

You cannot  use those attributes using pslots. 

I would suggest turn on full debug on the negotiator. You will be able to understand your options and its very limited.

Try to avoid preemption. Using defrag or something else. 

David

Get Outlook for Android

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Sam Copeland <SCopeland@xxxxxxxxxxxx>
Sent: Wednesday, November 10, 2021, 17:08
To: 'htcondor-users@xxxxxxxxxxx'
Subject: Re: [HTCondor-users] PREEMPTION_RANK undefined variables

Hi Jason,

Thanks for the tips on the ifThenElse() replacement.

Yes, it is always the same for all negotiator evaluations for the PREEMPTION_RANK. RemoteUserPrio and TotalJobRunTime are always undefined. However, the PREEMPTION_REQUIREMENTS works fine.

Thanks,

Sam

 

 

Hi Sam,

I'm not sure why RemoteUserPrio and TotalJobRunTime would be undefined, is this always the case for every slot that the negotiator evaluates?

FYI, there is a more compact syntax for ifThenElse(undefined(foo), bar, foo):

(foo ?: bar)

For example, the left half of your _expression_ could be:

(RemoteUserPrio ?: 0)*1000000

Finally, I doubt this solves the issue, but there are some other attributes that might be useful listed in the Priorities in Negotiation and Preemption section of the manual: https://htcondor.readthedocs.io/en/latest/admin-manual/user-priorities-negotiation.html#priorities-in-negotiation-and-preemption

Jason Patton

On 11/8/21 2:58 PM, Sam Copeland wrote:

I am trying to set the PREEMPTION_RANK in my config file but the inputs are undefined. I set in my config:

 

PREEMPTION_RANK = debug((ifThenElse(isUndefined(RemoteUserPrio), 0, RemoteUserPrio) * 1000000) - ifThenElse(isUndefined(TotalJobRunTime), 0, TotalJobRunTime))

 

In the Negotiator Log, I am getting:

 

11/08/21 12:39:14 Classad debug: RemoteUserPrio --> UNDEFINED

11/08/21 12:39:14 Classad debug: isUndefined(RemoteUserPrio) --> TRUE

11/08/21 12:39:14 Classad debug: ifThenElse(isUndefined(RemoteUserPrio),0,RemoteUserPrio) --> 0

11/08/21 12:39:14 Classad debug: TotalJobRunTime --> UNDEFINED

11/08/21 12:39:14 Classad debug: isUndefined(TotalJobRunTime) --> TRUE

11/08/21 12:39:14 Classad debug: ifThenElse(isUndefined(TotalJobRunTime),0,TotalJobRunTime) --> 0

11/08/21 12:39:14 Classad debug: (ifThenElse(isUndefined(RemoteUserPrio),0,RemoteUserPrio) * 1000000) - ifThenElse(isUndefined(TotalJobRunTime),0,TotalJobRunTime) --> 0

 

Is there a different variable I could use for RemoteUserPrio and TotalJobRuntime?

 

I am using version 8.8.12

 

Thanks,

Sam

This communication (both the message and any attachments or links) is confidential and only intended for the use of the person or persons to whom it is addressed unless we have expressly authorized otherwise. It also may contain information that is protected by solicitor-client privilege. If you are reading this communication and are not an addressee or authorized representative of an addressee, we hereby notify you that any distribution, copying or other use of it without our express authorization is strictly prohibited. If you have received this communication in error, please delete both the message and any attachments from your system and notify us immediately by e-mail or phone. In addition, we note that this communication and its transmission of data have not been secured by encryption. Therefore, we are not able to confirm or guarantee that the communication has not been intercepted, amended, or read by an unintended third party.

_______________________________________________

HTCondor-users mailing list

To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a

subject: Unsubscribe

You can also unsubscribe by visiting

https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:

https://lists.cs.wisc.edu/archive/htcondor-users/

 

This communication (both the message and any attachments or links) is confidential and only intended for the use of the person or persons to whom it is addressed unless we have expressly authorized otherwise. It also may contain information that is protected by solicitor-client privilege. If you are reading this communication and are not an addressee or authorized representative of an addressee, we hereby notify you that any distribution, copying or other use of it without our express authorization is strictly prohibited. If you have received this communication in error, please delete both the message and any attachments from your system and notify us immediately by e-mail or phone. In addition, we note that this communication and its transmission of data have not been secured by encryption. Therefore, we are not able to confirm or guarantee that the communication has not been intercepted, amended, or read by an unintended third party.

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/