[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Managing dynamic services

On 11/16/18 6:57 PM, Stuart Anderson wrote:
Is there a standard way for Condor to manage local privileged dynamic services on an execute machine depending on whether any currently matched jobs have requested it?

Put another way, instead of matching a job to a machine that already has a service running, what is the best way to handle individual jobs optionally requesting a service that is started by condor when a match occurs and before the condor job starts, along with Condor stopping the service after the last job requesting it has finished running?

The context of this question is Linux systems running systemd if there is a natural hook there.


This is an interesting question, and I think there's at least one way to do this with HTCondor, but I have a couple of questions to put a finer point on potential solutions

Usually, pools would be configured so this service would be started at boot time on all machine, but HTCondor would limit the number of running jobs that use the service using some mechanism. I assume that you don't want to have the service always running because it uses a non-trivial amount of resources, even when idle. If so, we may want to reflect this additional machine load in the startd. Also, it seems that many jobs on the same machine from different users can use this service without too much additional load on the service. Do you want the service to get killed when the last job finishes, or is it ok to have some "grace time", when no jobs are running, in order to encourage a subsequent job to start?

Also, is it sufficient for Condor to start the service before starting the first job, or do we need to block the job from starting until the service is "ready"?