Re: [Condor-users] Per-slot rank expressions?

[Dan Bradley <dan@xxxxxxxxxxxx>]

Hi Ross,

The machine RANK expression determines which jobs the machine (i.e. 
slot) prefers to run.  To specify which slots the job prefers to run on, 
you would use the job RANK expression, which can be set by the user in 
the submit file, and which can be tweaked by the administrator using 
APPEND_RANK.

In addition, the administrator can influence which slot is preferred by 
adjusting the negotiator configuration variables 
NEGOTIATOR_POST_JOB_RANK and NEGOTIATOR_PRE_JOB_RANK.  The former takes 
precedence over the job's RANK expression; the latter has lower 
precedence than the job's RANK expression.

Hope that helps.

--Dan

On 12/3/10 12:06 PM, Ross Oldenburg wrote:
> Hi Dan,
>
> I tried this and it seemed to work:
>
> STARTD_ATTRS = $(STARTD_ATTRS) SlotRank
>
> SLOT1_SlotRank = 100
> SLOT2_SlotRank = 10
> ...
>
> RANK = MY.SlotRank
>
> Rank shows up properly defined in the machine ads as "Rank = MY.SlotRank"
>
> I set this up on 3 machines, each with 24 total slots.  12 slots on each
> machine had low rank.  So there were 36 low rank slots and 36 high rank
> slots.  But, when I tried to submit 36 jobs targeting these three
> machines, the behavior was not as I expected.  Every time I tried this,
> 12 jobs would land on Machine A, and 24 would land on machine B.
> Invariably, one of the three machines wouldn't touch my jobs, but it was
> a different machine each time (well, modulo 3).  I expected that 12 jobs
> would go to each of the three machines and would get matched to the
> high-ranking slots on each.
>
> I guess my  understanding of what RANK is capable of is a little shaky.
>   It seems to be more subtle than what I'm assuming.  Is it always the
> case that jobs will prefer higher ranked slots or are there other
> conditions that go into computing RANKs?  e.g. does user priority play a
> role?
>
> Thanks,
> Ross
>
> Dan Bradley wrote:
> > Hi Ross,
> >
> > The startd policy expressions cannot be specified on a per-slot basis.
> > However, the power of the ClassAd language allows you to make one
> > expression that acts differently on the different slots.
> >
> > Example:
> >
> > RANK = \
> >    ifThenElse( SlotID==1, $(SLOT1_RANK), \
> >    ifThenElse( SlotID==2, $(SLOT2_RANK), \
> >    0 ))
> >
> > SLOT1_RANK = X
> > SLOT2_RANK = Y
> >
> > Of course, this doesn't have to be done on a per-slot basis.  You could
> > make ifThenElse() check some attribute other than SlotID.
> >
> > --Dan
> >
> > On 12/2/10 8:51 PM, Ross Oldenburg wrote:
> > > I considered using SLOT<N>_STARTD_ATTRS = (RANK = 1000), for example.
> > > But AFAIK, startd_attrs is only for custom classads.
> > >
> > > Ross Oldenburg wrote:
> > > > Hi,
> > > >
> > > > I have machines that use Hyperthreading.  I'd like to enable one slot
> > > > per thread.  However, I would like half of them to have a higher rank
> > > > than the rest to avoid putting work other than 'nice' jobs onto the
> > > > hyperthreads unless my pool is otherwise full.
> > > >
> > > > Examples of what I've tried:
> > > >
> > > > example1 {
> > > >
> > > > SLOT_TYPE_1 = cpu=1, mem=4096
> > > > NUM_SLOTS_TYPE_1 = 8
> > > > SLOT_TYPE_1_RANK = 10000
> > > >
> > > > SLOT_TYPE_2 = cpu=1, mem=1024
> > > > NUM_SLOTS_TYPE_2 = 8
> > > > SLOT_TYPE_2_RANK = 0
> > > >
> > > > }
> > > >
> > > > example2 {
> > > >
> > > > SLOT_TYPE_1 = cpu=1, mem=4096
> > > > NUM_SLOTS_TYPE_1 = 8
> > > >
> > > > SLOT_TYPE_2 = cpu=1, mem=1024
> > > > NUM_SLOTS_TYPE_2 = 8
> > > >
> > > > SLOT1_RANK = 10000
> > > > ...
> > > > SLOT8_RANK = 10000
> > > > SLOT9_RANK = 0
> > > > ...
> > > > SLOT16_RANK = 0
> > > >
> > > > }
> > > >
> > > > Neither of these worked.  When I view the machine ads, every slot
> > > > reports that its rank is 0.
> > > >
> > > > I did find a solution that did work, but I consider it a very bad
> > > > hack...
> > > >
> > > > I run two condor_masters and two condor_startds, each with NUM_CPUS set
> > > > to |threads|/2 and with the appropriate RANK value.  One of the
> > > > master/startd sets has<SUBSYS>_NAME set to something unique.
> > > >
> > > > I would like to avoid this if it's possible and only run one master and
> > > > startd, but I can't find any hints on the condor wiki or in the manual
> > > > that indicate how this can be accomplished.  Any advice would be greatly
> > > > appreciated.
> > > >
> > > > Thanks much,
> > > > Ross Oldenburg
> > > > _______________________________________________
> > > > Condor-users mailing list
> > > > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> > > > with a
> > > > subject: Unsubscribe
> > > > You can also unsubscribe by visiting
> > > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > > >
> > > > The archives can be found at:
> > > > https://lists.cs.wisc.edu/archive/condor-users/
> > > _______________________________________________
> > > Condor-users mailing list
> > > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> > > subject: Unsubscribe
> > > You can also unsubscribe by visiting
> > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > >
> > > The archives can be found at:
> > > https://lists.cs.wisc.edu/archive/condor-users/
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/