[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] schedd on every submit node VS single schedd

On 11/19/18 5:35 AM, David Cohen wrote:
I'm starting an HTCondor system for the first time, to replace a torque/maui cluster, and trying to figure out the basic topology before I start. What is the benefit of having a schedd on every submit node vs single schedd.

I am glad you asked this question, as it touches on fundamental aspects of HTCondor.

The first reason to have multiple schedds it to scale up. HTCondor is designed to have a lightweight central manager that merely provisions resources to users, with most of the expense of job management happening at the edge, in the schedds. If you have a single submit machine with enough power to handle all of your idle and running jobs, there is less reason to have more than one schedd.

Another reason to have multiple schedds is data locality. If you are transferring large input sandboxes to run your jobs, perhaps you want to put your schedds near the source of the input files.

A common reason to have multiple schedds is administrative control. There are a lot of policy knobs that an administrator can set in the schedd, and if your organization contains sub-organizations, each with different policy goals, sometimes it is easier and clearer for each sub-domain to have their own scheduler to work with.