[HTCondor-users] Relay unix socket from remote node Re: ssh_to_job via JumpHost from local machine for port forwarding

["Till Riedel (TM)" <riedel@xxxxxxx>]

Hi,

I give up and have one remaining simpler question:Â Is there any command to bind a remote inetd style pipe (like ssh_to_job does) or a remote unix socket inside a job to a unix socket on the submitting node in the vanilla universe?

TLDR;

I officially give up because the workings of this are pretty hardcoded and would require a custom condor installation (I don't even get the admins to update it to a newer one).

The problem is that the key is actually generated on the remote node. You can only send ATTR_SSH_KEYGEN_ARGS which is appended to SSH_TO_JOB_SSH_KEYGEN_ARGS (fun fact: the default is "-N '' -C '' -q -f %f -t rsa" as far as I understand the only arg is typicall "-b" for the paranoid).

So the solution on infrastructure level are:

• The most straightforward alternative seems to ask the admins to support the parallel universe which seems to provide actual ssh servers by default (don't quite find decent documentation actually)
  
• Another straight-forward solution that would work now already seems to be to completely exchange the keygen in the condor config with a more flexible wrapper (I guess this cannot be done via passing a SSH_TO_JOB_SSH_KEYGEN argument specific to the job) accepting more arguments (and passing a bogus private key in my case since this is checked explicitly).

Other solutions that should work without changing the infrastructure:

• simply try to get another job run on the same machine as the original job that starts my sshd
  
• tunnel another sshd inside of ssh-to-job :)

I will try the latter if I have time and nag the admins about the parallel universe.

IMHO, the most simple solution would be to allow modifying SSH_TO_JOB_SSH_KEYGEN by the user as I do not see any security implication if the user knows what they are doing.

It leaves me with my remaining question, how to do all this without opening more random network ports. I looked through the Code and could not quite understand all the magic being done by condor_starter.

Till

Am 18.04.2020 um 09:18 schrieb Till Riedel (TM):

> Hi,
>       
>       
>       thanks for the reply.
>       
>       
>       No key pair is needed in this case: only a single public key from
>       .authorized_keys, so there is no security risk.
>       
>       
>       The only problem seems that there is no option in condor to use an
>       existing key or any other way to spawn inetd style servers on the
>       remote node. Seems that the SSH-Servert part is pretty hardcoded
>       into starter (but I don't quite understand the code).
>       
>       
>       I now will try to rebuild the binary to allow using the existing
>       authorized keys, seems the easiest way around the problem.
>       
>       
>       Till
>       
>       
>       Am 18.04.2020 um 04:28 schrieb Bockelman, Brian:
>       
> > Hi Till,
> >         
> >         
> >         I have very little SSH-foo, but since no one else answered...
> >         
> >         
> >         When the sshd is launched on the worker node, thereâs a sshd
> >         config template file which is used to generate the config.
> >         
> >         
> >         Is it possible what youâre trying is forbidden by the generated
> >         server config? Maybe thereâs a clever way to punch a hole with
> >         a known key pair when starting the server?
> >         
> >         
> >         Brian
> >         
> >         
> >         Sent from my iPhone
> >         
> >         
> > > On Apr 14, 2020, at 5:09 PM, Till Riedel
> > >           (TM) <riedel@xxxxxxx> wrote:
> > >           
> > >           
> > >           ïHi,
> > >           
> > >           
> > >           (I have been thinking about a solution to a problem that I
> > >           have the feeling could be solved much simpler. I thought maybe
> > >           this list can safe me from doing to stupid stuff. I still
> > >           don't get the HTCondor internal communication part
> > >           completely...)
> > >           
> > >           
> > >           
> > >           I am accessing htcondor via a login node . What I want to do
> > >           is to define a ProxyCommand line in my ssh config to directly
> > >           connect to the job like:
> > >           
> > >           
> > >           Host *%condor-job
> > >           
> > >            ProxyCommand ssh login.example.com ssh_to_job_tunnel.sh
> > >           $(echo %h | cut -d%% -f1)
> > >           
> > >           
> > >           
> > >           ssh_to_job_tunnel.sh only contains a line to call the
> > >           ProxyCommand directly:
> > >           
> > >           
> > >           eval `echo $@|sed -n
> > >           "s/.*-oProxyCommand=\(.*\)condor-job.*/\1/p"`
> > >           
> > >           
> > >           
> > >           But now I have a problem I don't seem to get around:
> > >           ssh_to_job always generates new keypairs and I cannot add them
> > >           easily (maybe I try could to do agent-forwarding, which I
> > >           don't like). So I was thinking to fork condor_ssh_to_job to
> > >           add the keys as parameters. But before I start digging further
> > >           into the source I thought I first join this mailing-list to
> > >           not solve problems that have been solved already!
> > >           
> > >           
> > >           
> > >           Maybe I should also should also explain what I want to
> > >           achieve: I want to securely forward a unix socket from the job
> > >           via ssh to my local machine. (Actually somewhat similar to
> > >           what the ssh_to_job stuff does, but it seems pretty hardcoded
> > >           into the starter class or is there a command to open sockets
> > >           or bidirectional pipes to the remote job)
> > >           
> > >           
> > >           
> > >           Hope I don't sound to mad for a first post.
> > >           
> > >           
> > >           
> > >           Thanks in advance for the help!
> > >           
> > >           
> > >           Till
> > >           
> > >           
> > >           
> > >           
> > >           
> > >           
> > >           --Â
> > >           KIT - The Research University in the Helmholtz Association
> > >           
> > >           
> > >           Dr. Till Riedel (AkadR)
> > >           
> > >           Lab Leader TECO
> > >           
> > >           Vincenz PrieÃnitz Str. 1
> > >           
> > >           76131 Kalrlsruhe
> > >           
> > >           
> > >           Tel: 0721 608 41706 (forwarded!)
> > >           
> > >           
> > >           Web: https://www.teco.kit.edu/~riedel
> > >           
> > >           Email/XMPP: till.riedel@xxxxxxx
> > >           
> > >           Skype: till.riedel
> > >           
> > >           
> > >           _______________________________________________
> > >           
> > >           HTCondor-users mailing list
> > >           
> > >           To unsubscribe, send a message to
> > >           htcondor-users-request@xxxxxxxxxxx with a
> > >           
> > >           subject: Unsubscribe
> > >           
> > >           You can also unsubscribe by visiting
> > >           
> > >           https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> > >           
> > >           
> > >           The archives can be found at:
> > >           
> > >           https://lists.cs.wisc.edu/archive/htcondor-users/
> > >           
> > >         
> >         _______________________________________________
> >         
> >         HTCondor-users mailing list
> >         
> >         To unsubscribe, send a message to
> >         htcondor-users-request@xxxxxxxxxxx with a
> >         
> >         subject: Unsubscribe
> >         
> >         You can also unsubscribe by visiting
> >         
> >         https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> >         
> >         
> >         The archives can be found at:
> >         
> >         https://lists.cs.wisc.edu/archive/htcondor-users/
> >         
> >       
>       
>     
-- 
KIT - The Research University in the Helmholtz Association

Dr. Till Riedel (AkadR)
Lab Leader TECO
Vincenz PrieÃnitz Str. 1
76131 Kalrlsruhe

Tel: 0721 608 41706 (forwarded!)

Web: https://www.teco.kit.edu/~riedel
Email/XMPP: till.riedel@xxxxxxx
Skype: till.riedel