Re: [HTCondor-users] Submitting and refreshing X509_USER_PROXY

[Zach Miller <zmiller@xxxxxxxxxxx>]

Hi Tom,

I actually have been working on just such a mechanism... and I'll be giving a talk about it at HTCondor Week.  (Credential Management in HTCondor)

It's not yet documented since it isn't all finished yet... we can chat off-list if you are curious.  The general gist of it is that HTCondor maintains a private, root-owned directory of credentials associated with jobs and provides hooks for renewing/transforming them on the submit side as well as transferring them to the execute node, which can then periodically call back to the submit node for fresh versions if needed.


Cheers,
-zach


> -----Original Message-----
> From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of
> Thomas Patrick Downes
> Sent: Wednesday, April 25, 2018 6:54 PM
> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> Subject: Re: [HTCondor-users] Submitting and refreshing X509_USER_PROXY
> 
> I don't think there's a lot of client/server control over where the proxy
> ends up. Just whether it exists or not. Suppose you created a mechanism
> that facilitated copying to a safe location on the submit node for
> subsequent delegation. I believe the main import of "safe" is "able to be
> delegated but not read by anyone except condor/root and maybe, for
> convenience, the user."
> 
> If such a mechanism were off by default, but able to be automated by a job
> transform or some other admin setting, that would work pretty well for me.
> 
> I would describe this as not being terribly different from a submit node
> reboot since the proxies are often in /tmp. So might be worth thinking
> about from that perspective.
> 
> Tom
> 
> ïOn 4/25/18, 3:46 PM, "HTCondor-users on behalf of Zach Miller" <htcondor-
> users-bounces@xxxxxxxxxxx on behalf of zmiller@xxxxxxxxxxx> wrote:
> 
>     If by "handle the deletion of the proxy" you mean "does HTCondor make a
> copy of the proxy and associate it with the job" then the answer is no.
> 
>     What would happen:
> 
>     1) If the job is idle, and you logout, the proxy will go away.  The job
> will not be able to start because it will presumably fail during file
> transfer when trying to delegate the non-existant proxy.
> 
>     2) If the job was already running, and you logout, the proxy will go
> away.  Further attempts to update the proxy on the execute machine will
> fail, but I expect the job will continue running with the proxy it has.
> 
> 
>     I can think of plenty of workarounds (hacks) as I'm sure you can as
> well.  But I'm a little uneasy having HTCondor make copies of people's
> proxies without their explicit instruction to do so.  I guess the question
> is what do you want to happen, and how should that look?
> 
> 
>     Cheers,
>     -zach
> 
> 
>     > -----Original Message-----
>     > From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf
> Of
>     > Thomas Patrick Downes
>     > Sent: Wednesday, April 25, 2018 1:06 PM
>     > To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
>     > Subject: Re: [HTCondor-users] Submitting and refreshing
> X509_USER_PROXY
>     >
>     > Zach:
>     >
>     > Within LIGO, we use gsissh to access our HTCondor submit nodes. The
> default
>     > behavior of the client is to create an impersonation proxy on the
> submit
>     > node and to set the X509 environment variables appropriately. The
>     > impersonation proxy is deleted upon logout. Does HTCondor handle the
>     > deletion of the proxy on the submit node?
>     >
>     > Tom
>     >
>     > On 4/25/18, 12:14 PM, "HTCondor-users on behalf of Zach Miller"
> <htcondor-
>     > users-bounces@xxxxxxxxxxx on behalf of zmiller@xxxxxxxxxxx> wrote:
>     >
>     >     Hi Lukas,
>     >
>     >
>     >     HTCondor can automatically delegate GSI credentials to the
> execute
>     > node.  In your submit file just specify:
>     >        x509UserProxy = /tmp/x509up_u<uid>
>     >
>     >     Delegating is better than transferring because the proxy is then
> not
>     > transmitted over the wire.  HTCondor will also then update the proxy
> on the
>     > execute machine if it changes on the submit machine.  No need for a
> wrapper
>     > script.  Details on that here:
>     >
>     >
> http://research.cs.wisc.edu/htcondor/manual/v8.7/condor_submit.html#97602
>     >
>     >
>     >     Also, a VERY important issue is that if you are running all jobs
> as
>     > nobody, one job would be able to see another jobs sandbox (and
> therefore
>     > potentially steal the proxy from that job, which may be from a
> different
>     > user).
>     >
>     >     Instead of "nobody", you should use "slot users".  Details on
> that can
>     > be found here:
>     >
>     >
> http://research.cs.wisc.edu/htcondor/manual/v8.7/3_8Security.html#sec:RunAs
>     > Nobody
>     >
>     >     This will isolate each job with its own UID so they cannot
> examine or
>     > interfere with each other.
>     >
>     >
>     >     Please let me know if you have any further questions!
>     >
>     >
>     >     Cheers,
>     >     -zach
>     >
>     >
>     >     > -----Original Message-----
>     >     > From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On
> Behalf
>     > Of
>     >     > Koschmieder, Lukas
>     >     > Sent: Wednesday, April 25, 2018 11:57 AM
>     >     > To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
>     >     > Subject: [HTCondor-users] Submitting and refreshing
> X509_USER_PROXY
>     >     >
>     >     > Hi,
>     >     >
>     >     > My scheduler and execute nodes are located in different
> networks.
>     > Therefore
>     >     > there is neither a shared filesystem nor a common UID domain
>     > available. All
>     >     > jobs have to run as nobody user.
>     >     >
>     >     > I've enable GSI auth in Condor and set up two file servers that
>     > provide GSI
>     >     > auth support (Globus-GridFTP and XRootD). Now I'd like to
> enable
>     > Condor
>     >     > jobs to use the job owner's GSI credentials to access the GSI
> file
>     > servers.
>     >     > (The final goal is to dynamically auto-mount a user's XRootD
> working
>     >     > directory (input/output folder) on the execute nodes when a job
>     > starts -
>     >     > preferably inside the scratch directory.) I could use
>     > TRANSFER_INPUT_FILES
>     >     > to manually copy a user's local X509_USER_PROXY to the execute
> nodes
>     > and
>     >     > then use USER_JOB_WRAPPER to refresh that X509_USER_PROXY. I
> was
>     > wondering
>     >     > if there is a better / less hacky way to do that.
>     >     >
>     >     > Best regards,
>     >     > Lukas
>     >     >
>     >     >
>     >     >
>     >     > --
>     >     > Lukas Koschmieder
>     >     > Steel Institute IEHK
>     >     > RWTH Aachen University
>     >     > IntzestraÃe 1
>     >     > 52072 Aachen
>     >     > Germany
>     >
>     >
>     >     _______________________________________________
>     >     HTCondor-users mailing list
>     >     To unsubscribe, send a message to htcondor-users-
> request@xxxxxxxxxxx
>     > with a
>     >     subject: Unsubscribe
>     >     You can also unsubscribe by visiting
>     >     https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>     >
>     >     The archives can be found at:
>     >     https://lists.cs.wisc.edu/archive/htcondor-users/
>     >
>     >
>     >
>     > _______________________________________________
>     > HTCondor-users mailing list
>     > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
> with a
>     > subject: Unsubscribe
>     > You can also unsubscribe by visiting
>     > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>     >
>     > The archives can be found at:
>     > https://lists.cs.wisc.edu/archive/htcondor-users/
> 
>     _______________________________________________
>     HTCondor-users mailing list
>     To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
> with a
>     subject: Unsubscribe
>     You can also unsubscribe by visiting
>     https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
>     The archives can be found at:
>     https://lists.cs.wisc.edu/archive/htcondor-users/
> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/