[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Unicode environment variables on Windows.



Hello,

I have a job which starts python3 application from batch or shell script. And those script needs some environment variables to work.
When I pass unicode value in the job environment job works fine on linux but fails to decode variable value string on windows.

I can overcome this by replacing environment variable with file, but approach with variable seems more clear. Is it possible to decode such value?
I tried to start python with '-X utf8' option but it does not help.

HTCondor version is 8.9.6.

Example of submit code:

import random
import htcondor
import classad

task = {
    "initialdir": "/Users/3",
    "transfer_input_files": "check_vars.py", # Just prints variables.
    "environment": "\"VAR1='ÑÐÑÑ' VAR2='\U0001F3E0'\"",
    "output": "test.stdout",
    "error": "test.stderr",
    "requirements": 'OpSys == "WINDOWS"',
}

sub = htcondor.Submit(
    {
        # Start script, sets PYTHONIOENCODING=utf8 and runs 'python3 check_vars.py'
        "executable": "start.$$(OpSys).sh.bat",
        "universe": "11",  # Parallel
        "machine_count": "1",
        "should_transfer_files": "IF_NEEDED",
        "when_to_transfer_output": "ON_EXIT",
        "transfer_executable": "True",
        "+ParallelShutdownPolicy": classad.quote("WAIT_FOR_NODE0"),
        "+WantIOProxy": classad.quote("True"),
        "+NTDomain": classad.quote("p7"),
    }
)

coll = htcondor.Collector()
schedd_ad = random.choice(coll.locateAll(htcondor.DaemonTypes.Schedd))
schedd = htcondor.Schedd(schedd_ad)
with schedd.transaction() as transaction:
    sub_result = sub.queue_with_itemdata(transaction, 1, iter([task]))

print(f"CLUSTER ID={sub_result.cluster()}")


PS: I noticed another strange thing on windows nodes, job PATH environment variable have HTCondor binaries
directory in the front. But the delimeter after this directory is colon. So condor binaries and the next
directory in the PATH are unavailable in the job:
PATH=C:\pSevenExtension\HTCondor\bin:C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH

----------
Sergey Komissarov
Senior Software Developer
DATADVANCE

This message may contain confidential information
constituting a trade secret of DATADVANCE. Any distribution,
use or copying of the information contained in this
message is ineligible except under the internal
regulations of DATADVANCE and may entail liability in
accordance with the current legislation of the Russian
Federation. If you have received this message by mistake
please immediately inform me of it. Thank you!