[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] How to submit jobs to both Windows and Linux with a single .sub file



Hello,

Not sure about the condor way of choosing the right executable. You may use python wrapper instead of having two submit files to run the right executable depending upon OS.Â

Thanks & Regards,
Vikrant Aggarwal


On Tue, Aug 4, 2020 at 3:33 PM <jefhque@xxxxxxxxx> wrote:
Hello everyone,Â

I'm working on HTCondor v8.7.9 with a condor pool containing both Windows and Linux machines, and my submit node is a Windows machine. As the executable of Windows cannot execute on Linux, and the executable of Linux cannot execute on Windows either, I need to submit two kinds of executables to run on both Windows and Linux, but I don't know how to write the condor job description (.sub file) to let condor choose the correct executable to execute. I have tried several ways to figure out this, but they don't work:
  • I specified the executable with $(OpSys) macro (e.g. foo.$(OpSys).exe), but I find that the value of the macro is always "WINDOWS" when I submit with my Windows machine, regardless of the target machine is Windows or Linux. My thought is that the $(OpSys) macro is determined as the operating system of the submit machine, and thus it cannot resolve my issue.
  • I also tried to use the condition description (if/else), but it fails to resolve macros like $(OpSys) and the target machine's operating system and cause the parsing error when I submit the job. It seems that the conditions in the submit description file are only supported for simple static conditions, and dynamic conditions like choosing executables according to the target machine'sÂoperating system are not supported.
  • Currently, I can statically split the jobs into Windows jobs and Linux jobs and determine the job numbers of each platform according to the slot numbers of each platform, but it suffers from undesired load balance problem, as I cannot perfectly predict the execution time of each job on each machine.
Is there any way to make condor automatically choose the right executable for the target machine's operating system and schedule the jobs for both Windows and Linux without any manual scheduling?Â
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/