Re: [Condor-users] VMware universe

[Andrew Howard <ahoward@xxxxxxxxxx>]

The way you've specified it would spawn 20 different jobs, each with
different CD images, but also using different instances of the VMware
image. I understand how you'd like the workflow to go, but I'm not
really sure how this could be accomplished. Again, I'd think the
easiest way to achieve this would be to run a scheduler inside of the
VM itself, then you wouldn't have to worry about CD images, you'd just
submit a series of jobs to the VM.

-- 
Andrew Howard
ahoward@xxxxxxxxxx
Assistant Research Programmer
Rosen Center for Advanced Computing, Purdue University

On Wed, Jun 9, 2010 at 6:15 PM, Dave STREET <davey_street@xxxxxxxxxxx> wrote:
> To be honest I haven't yet tested how to get the start up files in to the
> vmware machine, but I thought there was a command in the submit file you
> could use? vm_cdrom_files = a.txt,b.txt,c.txt where this would create a CD
> image taht is loaded on to the VMware machine when it runs?
> so I could have a sumit file such as
>
> universe                     = vm
> executable                   = MATJOBS  (this is simple a name isn't it is
> has no effect on the job running?)
> log                          = simple.vm.log.txt
> vm_type                      = vmware
> vm_memory                    = 64
> vmware_dir                   = C:\condor-test
> vm_cdrom_files = a$(PROCESS).txt,b$(PROCESS).txt
> vmware_should_transfer_files = True
> queue =20
> so each run of the job uses the same vmmware image but different inputs as
> specified by the cdrom files? or is this not how it works?
>
> As for the pre-staging its nice to know its in the pipe line, my idea was
> that the first time a client recives a VM job it coopies over the VMware
> images. Condor then tags this image with the queue number(unique to that
> specifice queue) and master server ID.
> when the job has completed and the results are returned/schedule updated on
> the master, the client/master checks if there are still any more jobs in
> that queue, if so the image is kept, if not it is removed from the client.
> The client could then take on a new job as normal, after ever run it cheks
> if the vm que still has jobs and once it is empty deleted the image.
> because you may want to have other jobs run in-between VM ware jobs (some
> one submits a high priority task), idealy you would want
> wipe vm-ware ware image on job completion,
> wipe vm-ware image if job queue changes (so if it dosn't follow on with a
> second job from the same queue, the image is deleted and re-downloaded if
> needed later on)
> wipe vm-ware image if queue empty
> never wipe VM-ware image (ie if it is a image you are going to come back to
> again and again but with possible long delays between jobs)
> Just my ideas but look forward to seeing how this will work, Wish I had a
> bit more developmment background and not just networking as I would love to
> be able to get stuck in and help out on this project more.