[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] virtualenv, pip and htcondor (python)

This looks like a bug in our handling of WANT_KRB5.  Iâm not sure when it will be patched

So you might want to add your own #ifdefs to condor_auth_ssl.cpp so you can move on to the next thing ;)




From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of L Kreczko
Sent: Saturday, December 10, 2016 10:13 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] virtualenv, pip and htcondor (python)


Hi John, hi Brian,


Thanks for the answers. After a bit of fiddling I ended up with this set:








However, since I turn off KRB5 an error occurs:

/home/vagrant/workspace/htcondor/src/condor_io/condor_auth_ssl.cpp: In static member function âstatic bool Condor_Auth_SSL::Initialize()â:

/home/vagrant/workspace/htcondor/src/condor_io/condor_auth_ssl.cpp:107:7: error: âCondor_Auth_Kerberosâ has not been declared

  if ( Condor_Auth_Kerberos::Initialize() == false ||


src/condor_utils/CMakeFiles/condor_utils.dir/build.make:7152: recipe for target 'src/condor_utils/CMakeFiles/condor_utils.dir/__/condor_io/condor_auth_ssl.cpp.o' failed.



If I allow for KRB, the compilation fails at DAG:

[ 74%] Linking CXX executable condor_submit_dag

[ 74%] Built target condor_submit_dag

Scanning dependencies of target condor_dagman_metrics_reporter

[ 74%] Building CXX object src/condor_dagman/CMakeFiles/condor_dagman_metrics_reporter.dir/condor_dagman_metrics_reporter.cpp.o

[ 74%] Building CXX object src/condor_dagman/CMakeFiles/condor_dagman_metrics_reporter.dir/__/condor_utils/condor_version.cpp.o

[ 74%] Linking CXX executable condor_dagman_metrics_reporter

//lib/x86_64-linux-gnu/libssl.so.1.0.0: undefined reference to `SHA224_Init@xxxxxxxxxxxxx'

/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `ENGINE_get_id@xxxxxxxxxxxxx'

+ many more 'undefined reference'.


Is there a way to disable DAG? Essentially, I just want the python-bindings :)


And just as I said that, I 

cd src/python-bindings/



and end up with htcondor.so and classad.so!

Well, that was easy ;)



Starting from fresh, the python-bindings also compile

 - src/classad

 - src/condor_utils



Next step is to make the compiled libraries work with the installed ones (e.g import the module):

python -c 'import htcondor'Traceback (most recent call last):

  File "<string>", line 1, in <module>

ImportError: /home/vagrant/workspace/htcondor/src/python-bindings/libcondor_utils_8_5_9.so: undefined symbol: _ZN7classad19ClassAdJsonUnParser7UnparseERNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKNS_7ClassAdERKSt3setIS6_NS_12CaseIgnLTStrESaIS6_EE


BTW: 'import classad' works out-of-the-box'


For the setup.py there are two approaches:

Not experienced with either, suggestions are welcome.






On 8 December 2016 at 18:21, Brian Bockelman <bbockelm@xxxxxxxxxxx> wrote:

> On Dec 8, 2016, at 7:29 AM, L Kreczko <L.Kreczko@xxxxxxxxxxxxx> wrote:
> Hi all,
> Since my work on an HTCondor task implementation for Spotify's Luigi [1], this topic is again becoming of interest.
> In order to do 'pip install htcondor', htcondor would need a setup.py that compiles the HTCondor Python bindings and distributes the libraries htcondor.so and classad.so the same way the condor-python RPM does.
> As I understand from the instructions [2] all I need to call in the setup.py is
> 1) match code to the installed condor version
> 2) ./configure_uw
> 3) make package
> 4) point python to the two libraries it needs to install.

I suspect you'll need to pick apart ./configure_uw and aggressively disable cmake flags.  For example, the error you reference on 16.04 is due to a failure building a unit test.  You can do without those!

> Is this a safe way? Is it possible to compile just the python bindings (I am asking because I cannot build condor on 16.04 [3])?
> Do htcondor.so and classad.so need to be compiled in the same version as the condor install or does some wiggle-room exist?

All variants of the compilation should result in a compatible protocol layer.  However, if you disable some of the security libraries (think Globus, Kerberos), the security negotiation might fail.

HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at:



  Dr Lukasz Kreczko           
  Research Associate

  Department of Physics

  Particle Physics Group

  University of Bristol
  HH Wills Physics Lab
  University of Bristol
  Tyndall Avenue
  BS8 1TL

  +44 (0)117 928 8724 


  A top 5 UK university with leading employers (2015)

  A top 5 UK university for research (2014 REF)

  A world top 40 university (QS Ranking 2015)