[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Multicore Shutdown



Hi Laurence,

Looking at a local ClassAd, I see the following attributes in the parent ad:

ChildEnteredCurrentState = { 1469015894,1468984603,1469015899,1469005305,1468945896,1469003021,1468986420,1469005306 }
ChildActivity = { "Busy","Busy","Busy","Busy","Busy","Busyâ,"Idleâ,"Idle" }
ChildCpus = { 1,1,1,1,1,1,1,1 }

I think you want something like this:

>>> import classad
>>> ad = classad.ClassAd()
>>> ad['ChildEnteredCurrentState'] = [1469015894,1468984603,1469015899,1469005305,1468945896,1469003021,1468986420,1469005306]
>>> ad['ChildActivity'] = [ "Busy","Busy","Busy","Busy","Busy","Busy", "Idle", "Idle" ]
>>> ad['ChildCpus'] = [1,1,1,1,1,1,1,1]
>>> expr_str = '(ChildActivity[%(idx)s]=?="Busy") ? (time()-ChildEnteredCurrentState[%(idx)s]) : 0'
>>> ad['SumBusyTime'] = classad.ExprTree(expr_str % {'idx': 0}) + classad.ExprTree(expr_str % {'idx': 1}) + classad.ExprTree(expr_str % {'idx': 2}) + classad.ExprTree(expr_str % {'idx': 3}) + classad.ExprTree(expr_str % {'idx': 4}) + classad.ExprTree(expr_str % {'idx': 5}) + classad.ExprTree(expr_str % {'idx': 6}) + classad.ExprTree(expr_str % {'idx': 7})
>>> ad.eval('SumBusyTime')
155592L
>>> expr_str = '(ChildActivity[%(idx)s]=!="Busy") ? (time()-ChildEnteredCurrentState[%(idx)s]) : 0'
>>> ad['SumIdleTime'] = classad.ExprTree(expr_str % {'idx': 0}) + classad.ExprTree(expr_str % {'idx': 1}) + classad.ExprTree(expr_str % {'idx': 2}) + classad.ExprTree(expr_str % {'idx': 3}) + classad.ExprTree(expr_str % {'idx': 4}) + classad.ExprTree(expr_str % {'idx': 5}) + classad.ExprTree(expr_str % {'idx': 6}) + classad.ExprTree(expr_str % {'idx': 7})
>>> ad.eval('SumIdleTime')
50472L
>>> classad.ExprTree('SumIdleTime > SumBusyTime').eval(ad)
False

Offhand, I canât think of a cleaner way to do this without resorting to the custom python function interface.  The custom python function would look like this:

>>> import time
>>> def sumBusy(entered_state, activity):
...   entered_state = entered_state.eval()
...   activity = activity.eval()
...   total = 0
...   now = time.time()
...   for i in range(len(entered_state)):
...     if activity[i] == "Busy":
...       total += now - entered_state[i]
...   return total
... 
>>> classad.register(sumBusy)
>>> classad.ExprTree('sumBusy(ChildEnteredCurrentState, ChildActivity)').eval(ad)
158787.8209862709

Hope this helps,

Brian

> On Jul 19, 2016, at 2:41 PM, Laurence Field <Laurence.Field@xxxxxxx> wrote:
> 
> Hi,
> 
> We would like to optimize the shutdown of multicore startds. After the startds no longer accepts new jobs, there is a point where the time wasted by the idle slots exceeds the time lost by killing the running jobs. This is essentially shutdown startd if sum idle time > sum time of running jobs. Does anyone know how this should correctly expressed in the config?
> 
> Cheers,
> 
> Laurence
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/