[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Condor and Matlab: BASICS

> The central manager then farms out the jobs to workers. Each worker
> has a fixed IP address.
> The machine running my code, is receiving data in real time, and
> needs to process that (LAPACK style calculations) and hence the
> requirement for CPU.

You may have looked at this as it's akin to the Liverpool Uni
notes you refer to but, just in case ...
... one way to do what you want in your environment might be
as follows:

it assumes that you do not need to have Matlab running interactively
with the GUI and all the blah, blah, blah.

The upside of that is you are not using a licence when running the
executables as you would be for each instance of the interactive
Matlab "environment" and you do not need to install the full Matlab
on each node, just the MCR, which is about a 500MB payload for R2010a.

You'ld need to purchase a licence for the Matlab Compiler so as to
be able to compile your Matlab code and/or install the Matlab Compiler
Runtime (MCR) on each of the machines you want to run the codes on,
though I suppose you could get someone with a Matlab Compiler licence
to compile it for you and send you the executable.

That latter approach would seem to be what Matlab are trying to get
folk doing: develop in Matlab but ship a stand-alone exe. The compiler
even alows the developer to package the MCR with the exe they are
looking to distribute.

You would then treat the executable you created as a SIMD program
and invoke it with different inputs using Condor.

Submitting through Condor from windows compute nodes is simply a
matter of either passing a DOS .BAT file that sets the %PATH% to
include the MCR and then invokes your executable or setting that
%PATH% in the environment and just passing the executable.

For example to run 10 invocations of mcr.exe:

$ cat mcr.cmd
universe = vanilla
environment = "path=c:\WINDOWS\SYSTEM32 CONDOR_CLUSTER=$(Cluster)
executable = mcr.bat
arguments  = one two three
output     = mcr.out.$(Cluster).$(Process)
error      = mcr.err.$(Cluster).$(Process)
log        = mcr.log.$(Cluster).$(Process)
Requirements = (OpSys == "WINNT51")
TransferInputFiles  = magicsquare.exe
ShouldTransferFiles  = YES
WhenToTransferOutput = ON_EXIT
queue 10

$ cat mcr.bat
set PATH=C:\Program Files\MATLAB\MATLAB Compiler

The thing to note is that by defining CONDOR_CLUSTER and CONDOR_PROCESS
in the submission script environment you get access to those numbers
in the .BAT file using the DOS %ENV_VAR% syntax and so can branch each
invocation of the executable internally on the CONDOR_PROCESS (task
array id) number which, in the above example would go from 0 to 9.

Let's say your Condor cluster (job id) number is 1234, the "first"
mcr.exe (process/task array) gets invoked as

 mcr.exe 1234 0 one two three

the second as

 mcr.exe 1234 1 one two three

and so on.


I've had this working here for Condor/windows and SGE/UNIX grids.


I am aware that there are some toolboxes within the Matlab suite
that you can't compile and I guess there may be the odd function
in other toolboxes as well, but if your code doesn't use those
features, then you should be good to go.

Hope this helps,

Kevin M. Buckley                                  Room:  CO327
School of Engineering and                         Phone: +64 4 463 5971
 Computer Science
Victoria University of Wellington
New Zealand