[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Dynamically changing HTCondor configuration when using python binding



Hi Josh,
the software, GWMS, pre-dates the bindings, uses the bindings if available, falls back to the wrappers parsing XML output if the bindings are not installed.

CONDOR_CONFIG is indeed picked from os.environ
_CONDOR_ variables are not

Thank you,
Marco


> On Dec 6, 2019, at 11:46 AM, Josh Karpel via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:
> 
> Quick addendum...
> 
> os.environ is not affecting the current environment (of the running interpreter) and the python bindings are executed there, therefore ignoring these changes. I'd like to avoid to have to re-invoke the interpreter in a subprocess.  
> 
> Maybe I'm not understanding exactly what you're doing, but this doesn't sound right to me. Changing os.environ should indeed effect what the running interpreter sees in the environment (althogh obviously not in a different process). Are you making sure to call htcondor.reload_config() after changing the environment? If you do that, you should get a consistent config in the interpreter for the bindings to run against.
> 
> For a uniform experience, I would recommend always doing things in this order:
> 
> 1. set env
> 2. reload config
> 3. use bindings OR run command in subprocess (and inheriting env)
> 4. unset env
> 
> That should make everything behave the same, whether it's a bindings call or a wrapped binary.
> 
> (Incidentally, I'd be interested in knowing which binaries you had to wrap because we don't provide an equivalent binding!)
> 
> 
> Josh Karpel
> karpel@xxxxxxxx
> 
> 
> On Fri, Dec 6, 2019 at 11:30 AM John M Knoeller <johnkn@xxxxxxxxxxx> wrote:
> 1) sorry no.   htcondor.reload_config() will always use the file pointed to by the CONDOR_CONFIG environment variable if it is set. 
> 
> 2) yes.  But keep in mind that changes made this way will not be visible to sub-processes (if you should use any)
> 
> 3) no.  what you are doing is what we intended.
> 
> -tj
> 
> 
> -----Original Message-----
> From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Marco Mambelli
> Sent: Thursday, December 5, 2019 5:58 PM
> To: HTCondor-users <htcondor-users@xxxxxxxxxxx>
> Subject: [HTCondor-users] Dynamically changing HTCondor configuration when using python binding
> 
> Greetings,
> I'm using some HTCondor commands from a python program either wrapping the binaries or using the bindings (when available).
> 
> I need to change dynamically and programmatically some configuration parameters.
> When wrapping the commands I'm using os.environ[] to set variables like CONDOR_CONFIG or _CONDOR_... 
> This works because the commands are in a subshell.
> 
> os.environ is not affecting the current environment (of the running interpreter) and the python bindings are executed there, therefore ignoring these changes. I'd like to avoid to have to re-invoke the interpreter in a subprocess.
> 
> 
> 1. Is it there a way to specify a config file to htcondor.reload_config(), e.g the path in CONDOR_CONFIG from os.environ, not from the environment at the invocation of the python interpreter?
> 
> 2. And then should I modify the content of htcondor.param (_Param object) with what is in the _CONDOR_... variables?
> 
> 3. Is it there another better way to change the configuration when using the bindings?
> 
> 
> Thank you,
> Marco
> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_mailman_listinfo_htcondor-2Dusers&d=DwICAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=EF06-Wh4L9CNLgD8bnIjNQ&m=YdxM3XdAxkvixsShAbzF6vpl92MwInoPY9I9xxaG_CE&s=tQ-iX9lSPayEAZCgOMnXI_vJA8c1XJgbeJ8naSlHxMA&e= 
> 
> The archives can be found at:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_archive_htcondor-2Dusers_&d=DwICAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=EF06-Wh4L9CNLgD8bnIjNQ&m=YdxM3XdAxkvixsShAbzF6vpl92MwInoPY9I9xxaG_CE&s=x3DhDRlOwow7eIUwS0ZvCzJtDa1sZtHDPIdccU7y-Mw&e=