RE: [Condor-users] Using JobPrio to adjust the RANK on machines

["Ian Chesal" <ICHESAL@xxxxxxxxxx>]

Hmm. So I went ahead and implemented the rank expression:

RANK = (TARGET.JobPrio * 10)

on one dual-cpu client in my pool. I submitted two experiments, both
with the same number of jobs, but one with priority=-10 and the other
with priority=10. When I left last night things were going well. The
priority=10 jobs were executing on each vm on the client.

But this morning the queue was not as I expected. I had expected to see
no p=10 jobs and only p=-10 jobs left in the queue. Instead I'm seeing
equal -10/10 jobs in the queue and a 10 and -10 job executing on the
client at the same time.

Strange. I understand that priority only guides the order in which jobs
run and it is not an absolute rank but just a guide to condor. But
combined with my RANK expression I would never have expected this
situation. The rank expression should have (I thought) ensured that
higher priority jobs were run before lower priority jobs on this machine
always.

User priority should not have any effect here (again, I thought) because
I have only one machine in the pool and I am the only user in the pool.

Did I miss something (again)?

Output from condor_q and condor_userprio -all are below.

Thanks.

Ian

--- condor_q ---

-- Submitter: ttc-ichesal-lnx.altera.com : <137.57.142.112:32772> :
ttc-ichesal-
lnx.altera.com
 ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
  68.27  ichesal         9/13 17:28   0+00:20:54 R  -10 0.0  wrapper.bat
/exper
  68.28  ichesal         9/13 17:28   0+00:00:00 I  -10 0.0  wrapper.bat
/exper
  68.29  ichesal         9/13 17:28   0+00:00:00 I  -10 0.0  wrapper.bat
/exper
  68.30  ichesal         9/13 17:28   0+00:00:00 I  -10 0.0  wrapper.bat
/exper
  68.31  ichesal         9/13 17:28   0+00:00:00 I  -10 0.0  wrapper.bat
/exper
  68.32  ichesal         9/13 17:28   0+00:00:00 I  -10 0.0  wrapper.bat
/exper
  68.33  ichesal         9/13 17:28   0+00:00:00 I  -10 0.0  wrapper.bat
/exper
  68.34  ichesal         9/13 17:28   0+00:00:00 I  -10 0.0  wrapper.bat
/exper
  68.35  ichesal         9/13 17:28   0+00:00:00 I  -10 0.0  wrapper.bat
/exper
  69.30  ichesal         9/13 17:28   0+00:11:44 R  10  0.0  wrapper.bat
/exper
  69.31  ichesal         9/13 17:28   0+00:00:00 I  10  0.0  wrapper.bat
/exper
  69.32  ichesal         9/13 17:28   0+00:00:00 I  10  0.0  wrapper.bat
/exper
  69.33  ichesal         9/13 17:28   0+00:00:00 I  10  0.0  wrapper.bat
/exper
  69.34  ichesal         9/13 17:28   0+00:00:00 I  10  0.0  wrapper.bat
/exper
  69.35  ichesal         9/13 17:28   0+00:00:00 I  10  0.0  wrapper.bat
/exper

15 jobs; 13 idle, 2 running, 0 held

--- condor_userprio -all ---

Last Priority Update:  9/14 10:17
                                    Effective   Real     Priority   Res
Accumu
lated       Usage            Last
User Name                           Priority  Priority    Factor    Used
Usage
(hrs)    Start Time       Usage Time
------------------------------      --------- -------- ------------ ----
------
----- ---------------- ----------------
ichesal@xxxxxxxxxx                       1.08     1.08         1.00    2
87.76  8/27/2004 13:12  9/14/2004 10:17
------------------------------      --------- -------- ------------ ----
------
----- ---------------- ----------------
Number of users: 1                                                     2
87.76  8/27/2004 13:12  9/13/2004 10:19

-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx
[mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Ian Chesal
Sent: September 9, 2004 10:55 AM
To: matthew hope; Condor-Users Mail List
Subject: RE: [Condor-users] Using JobPrio to adjust the RANK on machines


Okay, thanks. This was my initial plan but then I came across JobPrio
and thought I had found an almost natural way to extend the system based
on the priority users assign to their own jobs. I noticed that the
classad for the job does hold a JobPrio setting that corresponds to the
priority= setting in my condor submission ticket. Is that just chance
that it worked out like this? It just seems more intuitive to my users
if I use TARGET.JobPrio to alter rank instead of defining a new tag but
maybe I'm missing the huge pitfall here.

Ian

-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx
[mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of matthew hope
Sent: September 9, 2004 3:41 AM
To: Condor-Users Mail List
Subject: Re: [Condor-users] Using JobPrio to adjust the RANK on machines


Thomas explained the difference between the two so I'll just note:

There are a variety of solutions to achive the affect you desire. The
one that seems most common on the list is to supply an additional 'tier'
attribute which allows the users to mark the overall priority of their
jobs by putting

+TIER=whatever

in the submit file you then place this in the RANK expression as so:

RANK=((TARGET.TIER =?= "HIGHEST") * 1000) + ((TARGET.TIER =?= "HIGH")
* 100) + ((TARGET.TIER =?= "NORMAL") * 10) + ((TARGET.TIER =?= "LOW")
* 0)

This will result in the rank being directly affected by this attribute
(and that if no TIER is specified the rank is the same as LOW i.e. 0)

Common problems with this are excessive preemption (prob not a big deal
unless you are using a clipped version such as windows or cannot run in
the standard universe)

There are some (not terribly pleasant) ways to hack round this but with
unpleasant side effects (such as maintaining a claim on a machine
despite higher tier jobs existing) but 6.7.1 has added some useful
functionality in the form of retirement guarantees that provide a far
cleaner solution.

if you search the archives for these you will find a great deal of
discussion on the subject as well as examples...

Matt

On Wed, 8 Sep 2004 12:18:01 -0700, Ian Chesal <ichesal@xxxxxxxxxx>
wrote:
> The situation: my office is not a fair-share type of place. I'd like
> to exploit the RANK attribute on startd machines to achieve a non-fair

> share environment. Section 3.6.2 of the 6.6.7 manual lists "JobPrio"
> as a job ClassAd that's available when forming the machine's RANK 
> expression.
> 
> First question: I'm confused. Section 3.6.2 says that the higher the
> JobPrio number the worse the priority. But the man page for 
> condor_submit in section 9 says of the "priority = [-20:+20]" setting 
> that jobs with higher numerical priority will run before jobs with 
> lower numerical priority. This is strange, is it not? Is the 
> documentation wrong in one instance or is there some relationship 
> between the priority setting in the submission ticket at the JobPrio 
> ClassAd that I can't find?
> 
> What I'm thinking of doing is using JobPrio to adjust the rank
> expression on machines so that user priorities influence job matching 
> and I can say to my users "only change your user priority if you want 
> to trump everyone else on machines". So what I'm thinking of doing is:
> 
>        RANK = (JobPrio * -1)
> 
> I'm using -1 because the JobPrio says higher is worse, so this makes
> JobPrio = -20 jobs rank above JobPrio = 20 jobs.
> 
> Second question: Is this a good idea? Can someone comment on using
> JobPrio to help create an environment where users can unfairly gain 
> access to resources when they need them.
> 
> Thanks!
> Ian
> 
> --
> Ian R. Chesal <ichesal@xxxxxxxxxx>
> Senior Software Engineer
> Altera Corporation
> Toronto Technology Center
> _______________________________________________
> Condor-users mailing list
> Condor-users@xxxxxxxxxxx 
> http://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
http://lists.cs.wisc.edu/mailman/listinfo/condor-users
_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
http://lists.cs.wisc.edu/mailman/listinfo/condor-users