[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Antwort: Re: can jobs be executed while the condor submitter isturned off ?

A simple solution is to install the schedd on a remote, stable server (you can use the same computer as the collector/negotiator but there are security considerations involved) and submit to that machine by one of the following options:


 1. shh/remote into this box, submit the job, remote back in to get the results

   Simple, effective, security issues involved, would involve transferring the required executable/submit files manually to the submit machine


 2. use the -remote submission argument but submit from your machine.

  Even simpler than 1 but does have some foibles and is ill suited to tasks where many jobs are part of a single submission since the input files and executable are duplicated for every job.

  You must use the condor_transfer_data command to retrieve the results.


 3. Write your own custom front end which runs on the scheduling server and deals with submitting the jobs and getting you back the data one the original machine is reconnected.

  Much more up front effort, gives you a platform configured for your needs though.


In all these cases your scheduling machine must have sufficient space available to it to handle all the data involved and it becomes a single point of failure unless you have several such machines. It also becomes a choke point, but this is likely to make a difference only if you have > 100 execute nodes and/or your data requirements for transfer in/out of the execute nodes on start/finish are high.


From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Tao.3.Chen@xxxxxxxxxxxxxxxxxxxxxxxxxxx
Sent: 05 August 2009 08:03
To: Condor-Users Mail List
Subject: [Condor-users] Antwort: Re: can jobs be executed while the condor submitter isturned off ?


Hi, Alex,
       Thank you so much!
        I also checked the condor workshop presentation, in it the job startup steps are  as follows:
1. Startd sends collector ClassAd describing itself.  (The Schedd does as well, but it has nothing interesting to say yet.)
2. The user calls condor_submit to submit a job.  The job is handed off to the schedd and condor_submit returns.
3. The schedd alerts the collector that it now has a job waiting.
4. The negotiator asks the collector for a list machines able to run jobs and schedd queues with waiting jobs.
5. The negotiator contacts the schedd to learn about the waiting job.
6. The negotiator matches the waiting job with the waiting machine.
7. The negotiator alerts the schedd and the startd that there is a match.
8. The schedd contacts the startd to claim the match.
9. The schedd starts a shadow to monitor the job.
10. The startd starts a starter to start the job.
11. The starter and the shadow contact each other.
11. The starter starts the job.
12. If the job is using the Condor syscall library (typically through being condor_compiled), it will contact the shadow to access necessary files.
      hmm....... So I think you are right, the whole condor system architecture is designed as the above, it is impossible to change. But I am not sure if there is any special design which is adapted to the submitter off?Also, for my work now ,I  need this kind of system which would allow the submitter unplug or shutdown, and the jobs work well, any ideas?
 Good Luck!

"Alas, Alex [FEDI]" <aalas@xxxxxxxxxxxxx>
Gesendet von: condor-users-bounces@xxxxxxxxxxx

08/04/2009 09:12 PM

Bitte antworten an
Condor-Users Mail List <condor-users@xxxxxxxxxxx>


"Condor-Users Mail List" <condor-users@xxxxxxxxxxx>



Re: [Condor-users] can jobs be executed while the condor submitter        isturned off ?


No, In my experience the submitter needs to be online until the last jobs’ execution is completed. That is using the default configuration after installing condor in the execute node. Every time I tried either to shutdown or unplug the computer after submitting a set of jobs, the jobs go immediately into the hold state. The reason why I leave it to some more experienced condor user.
I hope this statement at helps a little in your quest of answers,
From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Tao.3.Chen@xxxxxxxxxxxxxxxxxxxxxxxxxxx
Tuesday, August 04, 2009 5:06 AM
[Condor-users] can jobs be executed while the condor submitter isturned off ?



        I have read the manual and the some presentations carefully, found that during the execution of job, the schedd daemon should watch the progress of executing machine. And for the jobs in the queue, the schedd daemon will maintain the persistent queue, contact the available machines and match the job with executing machines.
       does that mean if I submit several jobs in the queue, the submitter should keep online and contact with the central manager and the executing machine?

       Is that possible after submitting the jobs, the submitter could be shut down, and the central manager takes the response for the whole jobs?  Or are there any other kinds of software could achieve this target?
Any ideas would be appreciated! Thanks in advance!

Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at:


Gloucester Research Limited believes the information provided herein is reliable. While every care has been taken to ensure accuracy, the information is furnished to the recipients with no warranty as to the completeness and accuracy of its contents and on condition that any errors or omissions shall not be made the basis for any claim, demand or cause for action.

The information in this email is intended only for the named recipient.  If you are not the intended recipient please notify us immediately and do not copy, distribute or take action based on this e-mail.

All messages sent to and from this email address will be logged by Gloucester Research Ltd and are subject to archival storage, monitoring, review and disclosure.

Gloucester Research Limited, 5th Floor, Whittington House, 19-30 Alfred Place, London WC1E 7EA.

Gloucester Research Limited is a company registered in England and Wales with company number 04267560.