Re: [HTCondor-users] Python 3 bindings in HTCondor 8.9.11

[Jason Patton <jpatton@xxxxxxxxxxx>]

Quentin,

Without looking too deep, I don't know why previous versions of system-packaged Python 3 bindings worked with newer versions of Python 3. Unintentionally, we may not have been using any APIs in our 8.8 bindings that have had breaking changes with newer Python versions, and we may have changed to using APIs that aren't backwards compatible at some point during 8.9 development.

I don't think we have documentation about our system packages of the Python bindings matching upstream versions of Python, but it's generally understood that, for example, packages named "python3-*" should be compatible with the version of Python installed by the "python3" package. When these library packages contain only Python code (instead of compiled binaries), yes, they do tend to also work with newer versions of Python, but the recommended practice is to use conda or virtualenvs to keep your Python environment separate from the system environment when using a different version of Python (or even the same version, considering that what's available via system package repositories is often out of date with what's available in PyPI via pip).

Jason

On Thu, Mar 11, 2021 at 4:11 AM Quentin Le Boulc'h <quentin.leboulch@xxxxxxxxxxx> wrote:

Hi Jason,

Thanks for the explanation, what I donât understand is that itâs working fine with condor-8.8.12 for any Python 3 version.
Also, is the Python version used during compilation documented somewhere for each HTCondor release?

Best regards,
Quentin


> Date: Wed, 10 Mar 2021 10:13:44 -0600
> From: Jason Patton <jpatton@xxxxxxxxxxx>
> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> Subject: Re: [HTCondor-users] Python 3 bindings in HTCondor 8.9.11
> Message-ID:
>       <CAFyOkSXFy1vuSqGxX__2kEvJ+YPUOFOLhybmCS3Ti60UnvFzNQ@xxxxxxxxxxxxxx>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Quentin,
>
> The Python bindings contain compiled libraries, and so the version of the
> Python interpreter used has to match the version of Python the bindings
> were compiled against. The versions distributed via RPM and DEB packages
> match the versions of Python available via the Linux distributions'
> standard package repositories (Python 3.6 for CentOS 7, for example). The
> workaround, as you have done, is to install via pip or conda.
>
> Jason Patton
>
>
> On Wed, Mar 10, 2021, 9:44 AM Quentin Le Boulc'h <
> quentin.leboulch@xxxxxxxxxxx> wrote:
>
>> Hi,
>>
>> With the development version 8.9.11, I can?t import the Python bindings
>> using Python > 3.6.
>> Using Python 3.9 I got for instance: "ImportError: cannot import name
>> 'htcondor' from partially initialized module 'htcondor' (most likely due to
>> a circular import)
>> (/usr/lib64/python3.6/site-packages/htcondor/__init__.py)".
>> With Python 3.6 it works. Strangely, using the same version bindings from
>> pip solves this issue. Is this a known problem?
>>
>> Best regards,
>> Quentin Le Boulc'h_______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with
>> a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/htcondor-users/