[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] condor and AWS

Yes, Fermilab has done this as part of the HEPCloud project.  See Burt Holzman's talk at last year's Condor Week for details.  http://research.cs.wisc.edu/htcondor//HTCondorWeek2016/presentations/ThuHolzman_HepCloud.pdf  There are others who have done it as well.  We have the machines call back to the pool at Fermilab.  There are other people who have set up a separate pool and flock to it.  See my talk at Condor Week this year for details on how we did the same in Google Compute Engine.


From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Taha Vasi <tvasi@xxxxxxxxxxxxx>
Sent: Friday, March 10, 2017 10:16:42 AM
To: htcondor-users@xxxxxxxxxxx
Subject: [HTCondor-users] condor and AWS




Has anyone implemented a condor pool, with bursting into AWS to eliminate peaks of jobs?


So the idea is to spin up machines in AWS when the job Q internally gets above a certain amount.. till the Q drops below some threshold.


Would it make sense to setup a separate condor pool in the cloud and then flock to it? Or just spin up AWS instances and have them join the existing pool whenever  you determine?





Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorized and regulated by the Financial Conduct Authority.