Re: [HTCondor-users] deb packages for HTCondor Python API

[Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx>]

Dear Jason,

Am 13.05.19 um 23:04 schrieb Jason Patton:
> Oliver,
> 
> I'm currently involved in packaging the bindings for PyPI. As you described, this is an unfortunate side effect of PyPI's requirement that all externals be bundled with the package. I will take a look at the external libraries, including libkeyutils, and see what we can bring up to date in the next release. Thanks for bringingÂthis to my attention!

many thanks, also for the very quick reply! 
In theory, bumping libkeyutils alone (and rebuilding libkrb5) should be sufficient to fix this issue at least, but of course this may be a good time to re-check all of them - in case you need a tester 
using Kerberos for a new version in PyPi, just let me know ;-). 

Cheers and thanks again,
	Oliver

> 
> Jason
> 
> On Mon, May 13, 2019 at 3:47 PM Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>> wrote:
> 
>     Dear Todd, dear all,
> 
>     Am 10.05.19 um 15:21 schrieb Todd Tannenbaum:
>     > If you want just the Python bindings alone, or you want them for a different Python version, you can use pip (Pythonâs package manager), ie
>     >
>     > Â Âpip install htcondorÂ
>     >
>     > Or perhaps
>     >
>     > Â Âpip3 install htcondorÂ
>     >
>     > Hope the above helps. At some point soonish it is planned for both RPM and DEB packages to include both python 2 and python 3 bindings, but not until then pip is your friend.
> 
>     that also fails completely for us ("AUTHENTICATE:1003:Failed to authenticate with any method"), albeit for a different packaging bug.
> 
>     We are using Kerberos authentication with "default_ccache_name = KEYRING:persistent:%{uid}" (the default on CentOS 7).
> 
>     The persistent keyring support was added to libkeyutils in version 1.5 (released in March 2011), but the version shipped with the pip wheel package is version 1.2 which is even older than that (early 2010),
>     compare what pip gave me:
>     $ nm -D ~/.local/lib/python2.7/site-packages/htcondor.libs/libkeyutils-1-ff31573b.2.so <http://libkeyutils-1-ff31573b.2.so> | grep persistent
>     <no output>
>     vs. what "good old" RHEL 7 gives me:
>     $ nm -D /usr/lib64/libkeyutils.so.1.5 | grep persistent
>     0000000000001b00 T keyctl_get_persistent
>     The same is true for Ubuntu Xenial or Bionic or others (albeit they ship a more recent point release of course).
> 
>     As a consequence, libkrb5 (which is also bundled with the pip package) disables persistent keyring support (c.f. https://github.com/krb5/krb5/blob/d439e370b70f7af4ed2da9c692a3be7dcf7b4ac6/src/configure.ac#L349
>     for the corresponding automake).
> 
>     So in short, it seems the version in the wheel on PyPi is broken in kerberized environments using persistent keyrings (which some major distros use as default due to the various advantages coming with that)
>     because an over eight year old version of libkeyutils is bundled...
> 
>     That also one of the major reasons why PIP will never be a friend of mine: While things often "just work", it makes users pull in a mess of library versions in such bundles
>     including possibly stone-aged library versions with functional errors or security issues inside (and admins lose control about that).
> 
>     A dirty hack for a user would be to use something like:
>     Âexport KRB5CCNAME=/tmp/krb5cc_$(id -u)
>     and fetch a new Kerberos TGT, but that's really just a hack. The best solution would be the fixed package.
> 
> 
>     TL;DR, and Leaving that aside: While I don't know how and by whom the PyPi package is built, could this be upgraded to use a somewhat more recent version of libkeyutils?
> 
>     Cheers, many thanks and all the best,
>     Â Â Â Â Oliver
> 
> 
>     >
>     > Regards
>     > ToddÂ
>     >
>     >
>     >> _______________________________________________
>     >> HTCondor-users mailing list
>     >> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx <mailto:htcondor-users-request@xxxxxxxxxxx> <mailto:htcondor-users-request@xxxxxxxxxxx <mailto:htcondor-users-request@xxxxxxxxxxx>> with a
>     >> subject: Unsubscribe
>     >> You can also unsubscribe by visiting
>     >> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>     >>
>     >> The archives can be found at:
>     >> https://lists.cs.wisc.edu/archive/htcondor-users/
>     >
>     > _______________________________________________
>     > HTCondor-users mailing list
>     > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx <mailto:htcondor-users-request@xxxxxxxxxxx> with a
>     > subject: Unsubscribe
>     > You can also unsubscribe by visiting
>     > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>     >
>     > The archives can be found at:
>     > https://lists.cs.wisc.edu/archive/htcondor-users/
>     >
> 
> 
>     -- 
>     Oliver Freyermuth
>     UniversitÃt Bonn
>     Physikalisches Institut, Raum 1.047
>     NuÃallee 12
>     53115 Bonn
>     --
>     Tel.: +49 228 73 2367
>     Fax:Â +49 228 73 7869
>     --
> 
>     _______________________________________________
>     HTCondor-users mailing list
>     To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx <mailto:htcondor-users-request@xxxxxxxxxxx> with a
>     subject: Unsubscribe
>     You can also unsubscribe by visiting
>     https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
>     The archives can be found at:
>     https://lists.cs.wisc.edu/archive/htcondor-users/
> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> 


-- 
Oliver Freyermuth
UniversitÃt Bonn
Physikalisches Institut, Raum 1.047
NuÃallee 12
53115 Bonn
--
Tel.: +49 228 73 2367
Fax:  +49 228 73 7869
--

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature