Re: [HTCondor-users] Unicode environment variables on Windows.

[Thomas Hartmann <thomas.hartmann@xxxxxxx>]

Hi Sergey,

can you maybe base64 encode these values before assigning them to the
environment variables? And decode them in the python scripts (or in the
job wrapper). Might not be very elegant but hopefully encoding safe.

Cheers,
  Thomas


> FOOVAR=$(echo "ââ" | base64)
> echo $FOOVAR
4omw4oufCg==

> echo $FOOVAR | base64 -d
ââ
> NEWFOOVAR=$(echo $FOOVAR | base64 -d)



On 24/08/2020 11.15, Sergey A. Komissarov via HTCondor-users wrote:
> Hello Todd,
> 
> Thank you for the explanation. Actually I run several parallel tasks via python API in one job and only some of them uses windows. 
> But all tasks needs those unicode environment variables. I will use workaround with files for the windows tasks, for now it is the simplest solution.
> Hope this will be fixed someday.
> 
> ----------
> Sergey Komissarov
> Senior Software Developer
> DATADVANCE
> 
> This message may contain confidential information
> constituting a trade secret of DATADVANCE. Any distribution,
> use or copying of the information contained in this
> message is ineligible except under the internal
> regulations of DATADVANCE and may entail liability in
> accordance with the current legislation of the Russian
> Federation. If you have received this message by mistake
> please immediately inform me of it. Thank you!
> 
> 
> 
> ----- Original Message -----
> From: "Todd L Miller" <tlmiller@xxxxxxxxxxx>
> To: "Sergey A. Komissarov via HTCondor-users" <htcondor-users@xxxxxxxxxxx>
> Cc: "Sergey A. Komissarov" <sergey.komissarov@xxxxxxxxxxxxxx>
> Sent: Friday, August 21, 2020 7:45:38 PM
> Subject: Re: [HTCondor-users] Unicode environment variables on Windows.
> 
>> When I pass unicode value in the job environment job works fine on linux 
>> but fails to decode variable value string on windows.
> 
>  	On Windows, the API to set environment variables requires that you 
> specify if the variables are ANSI or Unicode.  HTCondor currently uses the 
> ANSI version, which does locale-aware conversion into the UTF-16 encoding 
> that Windows actually stores in the environment.  It seems like this 
> conversion doesn't know what to do with UTF-8 encoded Unicode.  It may be 
> possible to construct a work-around by encoding the variable values in 
> UTF-16 and adjusting the locale, but that would make your submit file 
> (more) Windows-specific.
> 
>  	I would recommend that you use your existing work-around, instead.
> 
> - ToddM
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature