Re: [HTCondor-users] Use execute machine environment variable

[Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

Ivoâs approach is definitely what youâll want to use. An alternative is a USER_JOB_WRAPPER, but that runs around every job submitted in the entire pool, which may be more extensive than youâd want.

 

I ran into the same question very early in my HTCondor adoption back in 2013, and it took a while to get my mind around whatâs going on.

 

The submit description is telling the scheduler what kind of machine is needed to run the job, and it has no connection to the execute machine at the point when itâs submitted. So $ENV() in the submit description is using the environment variables of the job submitter - the person running condor_submit â not anything on the exec side.

 

The submission tells the exec node exactly how to build the environment for the job. You want it this way, because you wouldnât want someone with ill intent to be able to control the environment your job uses without your express knowledge.

 

The $$() technique is used to expand a ClassAd attribute of the machine where the job is finally assigned to execute. By using $$(), youâre expressly acknowledging that you trust what that value will be on the exec node.

 

To detect and set the exec path, you can use a OneShot STARTD_CRON job to run a command or script to set the ExecutablePath attribute for the machine, so that a submission with $$(ExecutePath) will be able to use it.

 

                -Michael Pelletier.

 

From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Ivo
Sent: Wednesday, May 17, 2017 11:52 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Use execute machine environment variable

 

There are some solutions to your problem. I'm employing, nowadays, STARTD variables that I can put in the submit files, for example:

Add to HTCondor configuration (execute nodes):

EXECUTABLE_PATH = "c:\xpto\asd.exe"
STARTD_ATTRS = $(STARTD_ATTRS), EXECUTABLE_PATH

And you'll be able to put, on the submit file:

executable = $$(EXECUTABLE_PATH)

Var will be resolved on executing machine. As I said, this is only one way to address the problem, but it works.

Cheers,
Ivo Cavalcante

 

Em qua, 17 de mai de 2017 12:10, Fernando Nellmeldin <f.nellmeldin@xxxxxxxxxxxxxxxxxxxx> escreveu:

Hello!

I'm new to HTCondor (less than 1 week!) and I'm trying to set our network to run it. 

So far, we have the Executor nodes in Linux, and most of the Submit machines are Windows 8. 

Everything is working as expected and I can send jobs from Windows and Linux without a problem.

 

What I would like to know is if it is possible to pick the executable from the environment variables of the Executor node. 

So far, our submit file starts with:

Universe = vanilla

Executable = /path/to/executable.sh

transfer_executable = false

copy_to_spool = false

 

But the path to the executable could change between Executor nodes, so it would be nice to say:

executable = $EXECUTABLE_PATH

and every node set would set this variable to the correct location. Is it possible to do this? 

 

I do know that you can define and pass some environment variables with:

environment = "var=value"

 

And it is even possible to pass the complete environment of the Submit machine with:

getenv = True

 

But I would like to have access to the environment of the Executor node... As I am very new to HTCondor, maybe I am missing something obvious here. Sorry for that!

 

Thank you very much.

 

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/