[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] deb packages for HTCondor Python API



Oliver,

I'm currently involved in packaging the bindings for PyPI. As you described, this is an unfortunate side effect of PyPI's requirement that all externals be bundled with the package. I will take a look at the external libraries, including libkeyutils, and see what we can bring up to date in the next release. Thanks for bringing this to my attention!

Jason

On Mon, May 13, 2019 at 3:47 PM Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx> wrote:
Dear Todd, dear all,

Am 10.05.19 um 15:21 schrieb Todd Tannenbaum:
> If you want just the Python bindings alone, or you want them for a different Python version, you can use pip (Pythonâs package manager), ie
>
>    pip install htcondor 
>
> Or perhaps
>
>    pip3 install htcondor 
>
> Hope the above helps. At some point soonish it is planned for both RPM and DEB packages to include both python 2 and python 3 bindings, but not until then pip is your friend.

that also fails completely for us ("AUTHENTICATE:1003:Failed to authenticate with any method"), albeit for a different packaging bug.

We are using Kerberos authentication with "default_ccache_name = KEYRING:persistent:%{uid}" (the default on CentOS 7).

The persistent keyring support was added to libkeyutils in version 1.5 (released in March 2011), but the version shipped with the pip wheel package is version 1.2 which is even older than that (early 2010),
compare what pip gave me:
$ nm -D ~/.local/lib/python2.7/site-packages/htcondor.libs/libkeyutils-1-ff31573b.2.so | grep persistent
<no output>
vs. what "good old" RHEL 7 gives me:
$ nm -D /usr/lib64/libkeyutils.so.1.5 | grep persistent
0000000000001b00 T keyctl_get_persistent
The same is true for Ubuntu Xenial or Bionic or others (albeit they ship a more recent point release of course).

As a consequence, libkrb5 (which is also bundled with the pip package) disables persistent keyring support (c.f. https://github.com/krb5/krb5/blob/d439e370b70f7af4ed2da9c692a3be7dcf7b4ac6/src/configure.ac#L349
for the corresponding automake).

So in short, it seems the version in the wheel on PyPi is broken in kerberized environments using persistent keyrings (which some major distros use as default due to the various advantages coming with that)
because an over eight year old version of libkeyutils is bundled...

That also one of the major reasons why PIP will never be a friend of mine: While things often "just work", it makes users pull in a mess of library versions in such bundles
including possibly stone-aged library versions with functional errors or security issues inside (and admins lose control about that).

A dirty hack for a user would be to use something like:
 export KRB5CCNAME=/tmp/krb5cc_$(id -u)
and fetch a new Kerberos TGT, but that's really just a hack. The best solution would be the fixed package.


TL;DR, and Leaving that aside: While I don't know how and by whom the PyPi package is built, could this be upgraded to use a somewhat more recent version of libkeyutils?

Cheers, many thanks and all the best,
        Oliver


>
> Regards
> Todd 
>
>
>> _______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx <mailto:htcondor-users-request@xxxxxxxxxxx> with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/htcondor-users/
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>


--
Oliver Freyermuth
UniversitÃt Bonn
Physikalisches Institut, Raum 1.047
NuÃallee 12
53115 Bonn
--
Tel.: +49 228 73 2367
Fax:  +49 228 73 7869
--

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/