On 10/10/11 10:36 AM, Brian Bockelman wrote:
On Oct 10, 2011, at 10:21 AM, Tim St Clair wrote:If we hold true to shared library version-ing mantra this becomes a non-issue. We simply need to up the rev for classads.What if you want to test a bugfix that doesn't break the ABI? I think it's very useful for developers to be able to feed their own libs. Sigh - Greg pointed out to me that my suggestion is dense, and you can't do per-.so RPATH. This led me to lookup "man ld.so". To quote: The shared libraries needed by the program are searched for in various places: o (ELF only) Using the DT_RPATH dynamic section attribute of the binary if present and DT_RUNPATH attribute does not exist. Use of DT_RPATH is deprecated. o Using the environment variable LD_LIBRARY_PATH. Except if the executable is a set-user- ID/set-group-ID binary, in which case it is ignored. o (ELF only) Using the DT_RUNPATH dynamic section attribute of the binary if present. o From the cache file /etc/ld.so.cache which contains a compiled list of candidate libraries previously found in the augmented library path. If, however, the binary was linked with -z nodeflib linker option, libraries in the default library paths are skipped. o In the default path /lib, and then /usr/lib. If the binary was linked with -z nodeflib linker option, this step is skipped. Note that with RPATH that LD_LIBRARY_PATH does *not* override things.
Good point. I misunderstood this.
What if we set DT_RUNPATH to /usr/lib64:$ORIGIN/..libs? Then, 1) In glideins / testing, one can provide a custom version of the libs using trusty ol' $LD_LIBRARY_PATH. 2) If not overridden, /usr/lib64 is used. 3) If not in /usr/lib64, then $ORIGIN/..libs is used. We don't lose flexibility for developers, and it is effectively the same for users.
The flexibility seems necessary to me. The cost of flexibility is potential mixups with LD_LIBRARY_PATH.
In glideins, it seems clear that for libraries closely tied to Condor, such as libclassads, the desired option is to use the lib that comes with Condor, not the one that happens to be lying around on the system. I don't know about other libs such as openssl. I believe these two classes of libraries are segregated in case one wants to treat them differently (the external libs are in $ORIGIN/../lib/condor and the condor libs are in $ORIGIN/../lib).
This suggests to me that at least $ORIGIN/../lib could be moved to the front of the search path.
--Dan
Cheers, Tim ----- Original Message ----- From: "Dan Bradley"<dan@xxxxxxxxxxxx> To: condor-devel@xxxxxxxxxxx Sent: Monday, October 10, 2011 10:10:12 AM Subject: Re: [Condor-devel] Surprise with classad.so, new batlab and rpath: Are you linking what I'm linking? This would affect glideins as well. Good catch! I suppose LD_LIBRARY_PATH could work around this. What's the wisdom on having the system lib directories come before $ORIGIN directories? If the order had been the other way around, this particular problem would go away, right? --Dan On 10/9/11 9:48 PM, Greg Thain wrote:In the new batlab, the base Condor is installed as native packages, where available. This means that condor binaries are in /usr/bin and /usr/sbin, and the .so's are in /usr/lib64. I made some changes to the classad library, and submitted a build& test run. I was surprised to see my changes not get executed in the test suite. This is because the RPATH is set to /usr/lib/64:some_other_stuff, and the base native condor has a libclassad.so installed in /usr/lib64, and the condor-under-test, finds that one first. Ick. We can fix this by aggressive revving of the .so version number during development, but this seems like an accident waiting to happen. Thoughts? -Greg _______________________________________________ Condor-devel mailing list Condor-devel@xxxxxxxxxxx https://lists.cs.wisc.edu/mailman/listinfo/condor-devel_______________________________________________ Condor-devel mailing list Condor-devel@xxxxxxxxxxx https://lists.cs.wisc.edu/mailman/listinfo/condor-devel _______________________________________________ Condor-devel mailing list Condor-devel@xxxxxxxxxxx https://lists.cs.wisc.edu/mailman/listinfo/condor-devel