Re: [HTCondor-users] htcondor interactions with software firewalls

[Matthew T West <m.t.west@xxxxxxxxxxxx>]

Hi Todd,

I presume these things are in the docs somewhere...

* Which daemon actually runs the remote file-transfer plugins? My guess 
would be the starter.
* The standard htcondor file transfer from the AP's local FS runs via 
the shadow to the starter, correct? As in both at the start of a job but 
also the other way during clean-up?
* If one has configured for a shared file system, the job itself should 
have direct access to files from that NFS/Lustre/BeeGFS/whatever system 
therefore doesn't need any daemon process mediator.
* Which daemon destroys the sandbox, the starter or the startd?

Beyond my own desire to understand the architecture better because its 
interesting, I am trying to flesh out slides 18-25 from Todd's talk 
https://indico.cern.ch/event/1274213/contributions/5571128/attachments/2716862/4719087/intro_and_architecture.pdf 
in a maybe vain hope to demystify the doings of a scheduler for cluster 
users.

Cheers,
Matt

Matthew T. West
DevOps & HPC SysAdmin
University of Exeter, Research IT
http://www.exeter.ac.uk/research/researchcomputing/support/researchit
57 Laver Building, North Park Road, Exeter, EX4 4QE, United Kingdom

On 26/11/2023 21:59, Matthew T West via HTCondor-users wrote:
> CAUTION: This email originated from outside of the organisation. Do 
> not click links or open attachments unless you recognise the sender 
> and know the content is safe.
>
>
> Hi Todd,
>
> Thanks for the reply and clear answers.
>
> * Presume the CCB is htCondor Connection Broker, right?
>
> * Sorry for the vague last question, which was mean to be asked from an
> admin perspective. For ease of management, should the CM be accessible
> by a SysAdmin directly or by exclusively using the AP as a jump-host? I
> can see the benefits of either choice.
>
> Cheers,
> Matt
>
> Matthew T. West
> DevOps & HPC SysAdmin
> University of Exeter, Research IT
> http://www.exeter.ac.uk/research/researchcomputing/support/researchit 
>
> 57 Laver Building, North Park Road, Exeter, EX4 4QE, United Kingdom
>
> On 26/11/2023 20:22, Todd L Miller wrote:
> > CAUTION: This email originated from outside of the organisation. Do
> > not click links or open attachments unless you recognise the sender
> > and know the content is safe.
> >
> >
> > > * Which port does ssh_to_job use to make a remote connection to the
> > > Â execution point where the job is running? Is it 9618 or the default
> > > Â ssh 22?
> >
> > ÂÂÂÂÂÂ It's port 9618.
> >
> > > * Is there any extra configuration necessary to make sure ssh_to_job
> > > Â works beyond the default setup? Particularly if I limit traffic to
> > > Â just a few open ports.
> >
> > ÂÂÂÂÂÂ If you allow inbound port 9618 on your execution points (that is,
> > the EPs are _not_ using CCB), then no; HTCondor tunnels the traffic over
> > its own connection.
> >
> > > In order for the various file transfer plugins to work, can I just
> > > have the
> > > relevant ports open on the access point or do I need to make sure the
> > > execution points also have firewalld rules configured for https, 
> > > ftp, or
> > > what-have-you as well?
> >
> > ÂÂÂÂÂÂ The file-transfer plug-ins should (generally) only ever want to
> > make outbound connections, but they will be made from the EPs.
> >
> > > For an single htcondor pool, should one be able to directly ssh to the
> > > central manager or should it be only accessible through an AP?
> >
> > ÂÂÂÂÂÂ Submitters should not (generally) need to ssh to the central
> > manager.
> >
> > -- ToddM
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx 
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users 
>
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/