I have been thinking about how to make HTCondor python bindings submit as capable as condor_submit,
The main thing missing right now is that the python bindings supports only the simplest possible queue statement. I think that the bindings needs to support the full range of queue options, but in a pythonic way, so I’m thinking this
sub = hcondor.Submit(“”” executable = /bin/echo queue 2 args in ( one, two, three ) “””)
for item in sub.itemdata() : print item which (given the submit file above) would print: {‘args’ : ‘one’} {‘args’ : ‘two’} {‘args’ : ‘three’}
with schedd.transaction() as txn : sub.queue_with_iter(txn, 2, iter(itemdata)) which would submit 2 jobs for each item returned by the iterator. The iterator could be sub.itemdata(), or any iterator that returned an equivalent type of item – either a dict, or a string. The reason this would need to be a new queue method on the submit object is that the current queue method already allows a third argument that is a list that is used to return the submitted job classads. This would conflict with adding an optional third argument to be the queue itemdata iterator. Having a new method also allows the return type to be change to be more pythonic – return a tuple of <clusterid,classad> for instance.
Thoughts? Is anyone currently using the third argument to the queue() method of the current htcondor.Submit class? Would anyone be harmed if I just removed that argument and replaced it with an optional iterator? -tj |