[Condor-users] Substituting values in a submit file

[Ben Cotton <bcotton@xxxxxxxxxx>]

Greetings, Condorland,

Our clusters have grown so large that we can no longer maintain a
single scratch file system across all of them.  Since many of our
Condor users run jobs out of scratch, we had to split the clusters
into separate FILESYSTEM_DOMAINs to prevent jobs from landing on
machines where the scratch file system doesn't exist.  For some users,
we are advising the use of file transfer, but that doesn't work in all
cases.

Specifically, we still have a common NFS file system for home
directories, and when users run jobs out of their $HOME, the
FILESYSTEM_DOMAIN distinction is less important.

I know that if you specify a TARGET.FileSystemDomain in the submit
file, Condor will not use the schedd's FileSystemDomain.  So what we
need is a way to get an appropriate FileSystemDomain expression into
the submit file without the users having to know or care about our
infrastructure.

Ideally, something like

   Requirements = IS_RCAC

would be expanded to

   Requirements = (TARGET.FileSystemDomain ==
"bluearc.rcac.purdue.edu" ) || (TARGET.FileSystemDomain ==
"lustrea.rcac.purdue.edu") || (TARGET.FileSystemDomain ==
"miner.rcac.purdue.edu")

or, preferably,

  Requirements = regexp("*.rcac.purdue.edu", TARGET.FileSystemDomain)

However, I haven't been able to find a way to do this.  If we can't
make this simple for the users, then less Science gets done, and that
makes management sad.  I suppose the other option is to write a
wrapper for condor_submit that will do this for us, but that sounds
like work.  Any suggestions?


Thanks,
BC

-- 
Ben Cotton
Systems Research Engineer
IT Research Systems
Purdue University