Re: [HTCondor-users] interpolation using the python bindings

[Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

Hi Jose,

It looks like this indicates that the Python bindings don't parse and expand submit variables it finds in the dictionary. This probably has to do with differences in the way in which condor_submit handles its communication with the scheduler as compared to the bindings and the dict. Do the bindings handle other submit-file macros like $(ProcID) and $(OPSYSMAJORVER)?

To make your example work, you'd need to do:

      "+foo" : ' string(ClusterId)'

This should work also:

      "MY.foo" : ' string(ClusterId)'

This expression can be evaluated after the job submission is complete regardless of the route by which it arrives at the scheduler. Good luck!

Michael V. Pelletier
Information Technology
Digital Transformation & Innovation
Integrated Defense Systems
Raytheon Company


-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Jose Caballero
Sent: Friday, September 7, 2018 9:39 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>; Todd Tannenbaum <tannenba@xxxxxxxxxxx>; Brian Bockelman <bbockelm@xxxxxxxxxxx>
Cc: Condor-Users Mail List <condor-users@xxxxxxxxxxx>
Subject: [External] Re: [HTCondor-users] interpolation using the python bindings

Hi again

this is really confusing to me.
So, I have a classad like this in the dictionary

      "+foo" : ' "$(Cluster)" '

After submission, if I run  submit.expand('+foo'), indeed, I get 12345, as expected.
However, condor_q -l 12345 shows

     foo = "$(Cluster)"

unexpanded.
I cannot do the expansion before submission, because then $(Cluster) is resolved as 1.
I have played trying to pass

      "+foo": ExprTree(' "$(Cluster)" ')

but then I get

     TypeError: No registered converter was able to produce a C++ rvalue of type std::string from this Python object of type ExprTree

If I try w/o the single quotes and only the double ones, then

     SyntaxError: Unable to parse string into a ClassAd.

What am I missing here?

Thanks
Jose




2018-09-06 10:24 GMT-04:00 Jose Caballero <jcaballero.hep@xxxxxxxxx>:
> Hi Tony
>
> I already tried, and this is the best I got
>
>>>> print classad.ExprTree("'$ENV(HOSTNAME)'").eval()
> Undefined
>
> which indeed doesn't really help too much.
> Or is it resolved at submission time?
> I guess it has to be, otherwise, not sure how it would work with 
> custom classads that depends on others like $(Cluster), which only get 
> a value after submission.
>
> Thanks a lot.
> Jose
>
>
>
>
> 2018-09-06 10:14 GMT-04:00 Anthony R Tiradani <tiradani@xxxxxxxx>:
>> Hi Jose,
>>
>> I think you have to wrap the expressions in an ExprTree class.  See 
>> https://htcondor-python.readthedocs.io/en/latest/classad.html#classad
>> .ExprTree
>>
>> Thanks,
>>
>> --
>> Anthony Tiradani
>>
>> ï-----Original Message-----
>> From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf 
>> of Jose Caballero <jcaballero.hep@xxxxxxxxx>
>> Reply-To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
>> Date: Wednesday, September 5, 2018 at 2:04 PM
>> To: Condor-Users Mail List <condor-users@xxxxxxxxxxx>
>> Subject: [HTCondor-users] interpolation using the python bindings
>>
>>     Hi,
>>
>>     I am submitting jobs using the python bindings.
>>     Some of the submit file items in the dictionary for htcondor.Submit( )
>>     object are like these:
>>
>>     'foo' : '   "aaa $(Cluster) bbb"   '
>>     'bar' : '   "$ENV(HOSTNAME)"  '
>>
>>     [no white spaces between single and double quotes, I just included for
>>     readability]
>>
>>     but the interpolation does not seem to happen. condor_q -l shows them
>>     unmodified:
>>
>>     foo = "aaa $(Cluster) bbb"
>>     bar = "$ENV(HOSTNAME)"
>>
>>     what would be the correct way to get them resolved?
>>     Is that something I must do myself before feeding them to Submit( )?
>>
>>     Thanks a lot in advance,
>>     Jose
>>     _______________________________________________
>>     HTCondor-users mailing list
>>     To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
>>     subject: Unsubscribe
>>     You can also unsubscribe by visiting
>>     
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.ed
>> u_mailman_listinfo_htcondor-2Dusers&d=DwICAg&c=gRgGjJ3BkIsb5y6s49QqsA
>> &r=5PsfAQ_fLSL7f1osCT9L8r9FZaZ9NGnhiuijyN6ZNfM&m=f-OnwFKOEYHvntjf24Ey
>> _tKbyqAHQZZVjqZ_wRlNZII&s=UBVpIQh4ADAEPaJMSZPXeIF-Fzj3dknSwnVTMO4IOmo
>> &e=
>>
>>     The archives can be found at:
>>     
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.ed
>> u_archive_htcondor-2Dusers_&d=DwICAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=5PsfA
>> Q_fLSL7f1osCT9L8r9FZaZ9NGnhiuijyN6ZNfM&m=f-OnwFKOEYHvntjf24Ey_tKbyqAH
>> QZZVjqZ_wRlNZII&s=r-r5iGOEFpbYW2H1LkKrUJgLM8amKbQAJ2hi8aHxirE&e=
>>
>>
>>
>> _______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx 
>> with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/