[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Python Binding Segmentation Fault

Apologies for the terminology confusion : where I said "Python hash(es)" in the below post, I really meant to say "Python dictionaries".

Yes, let the Perl-vs-Python terminology remarks begin :).

regards and hope everyone has a nice weekend,

On 5/12/2017 2:33 PM, Todd Tannenbaum wrote:
Hi Doga,

After some investigation, there is a simple workaround that will allow
you to use the submitMany() schedd method in v8.6.x without the
segmentation fault.  The issue is you were passing submitMany()
arguments that were Python hashes -- if you pass in ClassAds instead of
hashes, then it works.  IMHO it should work regardless, and that is
something we will investigate more, but for now below is a simple
example based on your code that works fine with HTCondor v8.6.  Note the
only significant difference is to surround the hashes for job_ad and
proc_ads with classad.ClassAd( { ....hash.... } ).

(p.s. Brian - there is no need to do any ChainToAd calls at all in the
submitMany() implementation, as the schedd will chain the proc ads to
the cluster ad submitted by submit_cluster_internal() upon committing
the transaction.... I will checkin a patch to fix this).

Hope this helps,

import htcondor
import classad
import os

proc_ads = []
logdir = os.path.join(os.environ['HOME'], 'logs')
for i in xrange(100):
    proc_ads.append(( classad.ClassAd({'Args': str(i),
                      'Iwd': logdir,
                      'Err': 'test-err.txt',
                      'Out': 'test-out.txt'}), 1))
job_ad = classad.ClassAd({
    'Cmd': '/opt/miniconda/envs/SANS/bin/python',
    'JobUniverse': 5,
    'RunAsOwner': classad.ExprTree('true'),
    'ShouldTransferFiles': "NO",
    'FileSystemDomain': "eglp.com",
    'TransferIn': classad.ExprTree('false'),
    'TransferINputSizeMB': 0,
    'OnExitHold': classad.ExprTree("(ExitBySignal == True) || (ExitCode
!= 0)"),
schedd = htcondor.Schedd()
print "Submitting {0} jobs to condor".format(len(proc_ads))
cluster_id = schedd.submitMany(job_ad, proc_ads)
print "Done!"

Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685