Re: [Condor-users] jobs with dependencies

[matthew hope <matthew.hope@xxxxxxxxx>]

1) I do not believe dag man can force a job to run on the same node.

2) Note that running rsync from the job itself may require some nasty
permission hacks

some alternates - all hacky

a) submit a cluster which sends the rsync job to every machine (by
updating it's requirements after each call to queue to state the
machines one  by one.
use condor_wait to wait on this job completing ok then release all the others.

b) potentially interesting method

Expose an additional attribute on your calc machines sayLastBlastDBSyncDay,
Get a cron job on each machine that after syncing updates the condor
config local with this value. then execs condor reconfig

your blast jobs can then add this to their requirements.

there are all sorts of nasties with this but it keeps them away from
the real jobs.

c) If the machines are only for runnig these jobs stop the startd
every night, do not start it untill your sync job has run...

A wrapper job is the most flexible...

On Mon, 1 Nov 2004 06:51:03 +0000, Michael Thon <mthon@xxxxxxxx> wrote:
> Greetings -
> I am setting up a small condor pool of linux machines for life science
> computing and I need some configuration advice.  I need to keep a directory
> of files (blast databases for the blastall app) on each execution node in
> sync with the directory on the master node.  Previous advice from others in
> the mailing list suggested that I use local copies of these files rather than
> a shared file system for performance reasons.  I can use rsync to ensure that
> the local copy is syncronized with the master.  The master directory could
> change on a daily basis so I need to ensure that it is synced before a
> blastall job is run.  therefore, I need to run rsync before every blastall
> job.  I see two ways of doing this:
> a) submit an rsync job and create a dependency somehow that prevents blastall
> job from running on the node unless the rsync job completes successfully.
> Don't know how to do this but I'm sure it must be supported and documented
> somewhere.  This would also require that rsync job and blastall job run on
> the same node.
> 
> b) create a job that actually runs two applications.  This would require some
> kind of wrapper script around the two apps.
> Suggestions/recommendations anyone?
> thanks
> Mike
> _______________________________________________
> Condor-users mailing list
> Condor-users@xxxxxxxxxxx
> http://lists.cs.wisc.edu/mailman/listinfo/condor-users
>