Re: [HTCondor-users] How to submit jobs to both Windows and Linux with a single .sub file

[John M Knoeller <johnkn@xxxxxxxxxxx>]

Yes,  $(OpSys)  and config if/else are evaluated at submit time, which is too soon for this use case.

 

This is why there is $$(OpSys),  the two dollar macro expansion does not expand at submit time,

it expands at match time to the OpSys of the machine that the job is going to run on.  

 

There is an example on the submit man page

https://htcondor.readthedocs.io/en/v8_9_7/man-pages/condor_submit.html

 

executable = povray.$$(OpSys).$$(Arch)

 

Hope this helps

-tj

 

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of jefhque@xxxxxxxxx
Sent: Tuesday, August 4, 2020 5:02 AM
To: htcondor-users@xxxxxxxxxxx
Subject: [HTCondor-users] How to submit jobs to both Windows and Linux with a single .sub file

 

Hello everyone, 

 

I'm working on HTCondor v8.7.9 with a condor pool containing both Windows and Linux machines, and my submit node is a Windows machine. As the executable of Windows cannot execute on Linux, and the executable of Linux cannot execute on Windows either, I need to submit two kinds of executables to run on both Windows and Linux,  but I don't know how to write the condor job description (.sub file) to let condor choose the correct executable to execute. I have tried several ways to figure out this, but they don't work:

  I specified the executable with $(OpSys) macro (e.g. foo.$(OpSys).exe), but I find that the value of the macro is always "WINDOWS" when I submit with my Windows machine, regardless of the target machine is Windows or Linux. My thought is that the $(OpSys) macro is determined as the operating system of the submit machine, and thus it cannot resolve my issue.

  I also tried to use the condition description (if/else), but it fails to resolve macros like $(OpSys) and the target machine's operating system and cause the parsing error when I submit the job. It seems that the conditions in the submit description file are only supported for simple static conditions, and dynamic conditions like choosing executables according to the target machine's operating system are not supported.

  Currently, I can statically split the jobs into Windows jobs and Linux jobs and determine the job numbers of each platform according to the slot numbers of each platform, but it suffers from undesired load balance problem, as I cannot perfectly predict the execution time of each job on each machine.

Is there any way to make condor automatically choose the right executable for the target machine's operating system and schedule the jobs for both Windows and Linux without any manual scheduling?