Re: [HTCondor-users] Problems with condor_ssh_to_job

[Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx>]

Dear Tim and fellow users,

I managed to understand the issue myself. 

The problem is that our /pool directory is on a separate partition, and not in "/var/lib/condor/pool".
Hence, our "/pool/condor" is only equipped with default_t context. 
Since that's the default location as configured by: 
https://github.com/HEP-Puppet/htcondor
we thought this would be taken care of. 

Now, manually executing:
semanage fcontext -a -t condor_var_lib_t "/pool/condor(/.*)?"
followed by:
restorecon -v /pool/condor
is needed to actually make the fix work. 

I will look into incorporating that into puppet and open a PR against the mentioned puppet module. 

Cheers,
Oliver


Am 02.11.2017 um 17:38 schrieb Oliver Freyermuth:
> Dear Tim,
> 
> sorry, but I only around to actually test the fix with the release. 
> 
> Sadly, it still does not seem to work for me. Using 8.6.7 on CentOS 7.4, in the audit.log, I get:
> 
> type=AVC msg=audit(1509640279.152:330): avc:  denied  { write } for  pid=52177 comm="ssh-keygen" path="/pool/condor/dir_52146/.condor_ssh_to_job_1/keygen.log" dev="sda8" ino=42729489 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:default_t:s0 tclass=file
> type=AVC msg=audit(1509640279.152:330): avc:  denied  { write } for  pid=52177 comm="ssh-keygen" path="/pool/condor/dir_52146/.condor_ssh_to_job_1/keygen.log" dev="sda8" ino=42729489 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:default_t:s0 tclass=file
> type=SYSCALL msg=audit(1509640279.152:330): arch=c000003e syscall=59 success=yes exit=0 a0=cc8700 a1=cc76e0 a2=cc3430 a3=7ffcdd442700 items=0 ppid=52172 pid=52177 auid=4294967295 uid=67803 gid=513 euid=67803 suid=67803 fsuid=67803 egid=513 sgid=513 fsgid=513 tty=(none) ses=4294967295 comm="ssh-keygen" exe="/usr/bin/ssh-keygen" subj=system_u:system_r:ssh_keygen_t:s0 key=(null)
> type=PROCTITLE msg=audit(1509640279.152:330): proctitle=2F7573722F62696E2F7373682D6B657967656E002D4E00002D4300002D71002D66002F706F6F6C2F636F6E646F722F6469725F35323134362F2E636F6E646F725F7373685F746F5F6A6F625F312F7373686B6579002D7400727361
> type=AVC msg=audit(1509640279.351:331): avc:  denied  { write } for  pid=52177 comm="ssh-keygen" name=".condor_ssh_to_job_1" dev="sda8" ino=42729487 scontext=system_u:system_r:ssh_keygen_t:s0 tcontext=system_u:object_r:default_t:s0 tclass=dir
> type=SYSCALL msg=audit(1509640279.351:331): arch=c000003e syscall=2 success=no exit=-13 a0=55b70f811da0 a1=241 a2=180 a3=0 items=0 ppid=52172 pid=52177 auid=4294967295 uid=67803 gid=513 euid=67803 suid=67803 fsuid=67803 egid=513 sgid=513 fsgid=513 tty=(none) ses=4294967295 comm="ssh-keygen" exe="/usr/bin/ssh-keygen" subj=system_u:system_r:ssh_keygen_t:s0 key=(null)
> 
> condor_submit -interactive just tells me "Failed to create ssh key ..." as before,
> and checking the selinux contexts of the /pool/dir_XXXXX directory and also /pool/dir_XXXXX/.condor_ssh_to_job/,
> I still get:
> system_u:object_r:default_t:s0
> for all directories and files within. 
> 
> Am I missing something? 
> Condor 8.6.7 is installed from the official HTCondor repos, and I even reinstalled the full worker node to remove any potential remnants of the old installation. 
> 
> Cheers,
> Oliver
> 
> 
> Am 20.10.2017 um 23:37 schrieb Tim Theisen:
>> Hi Oliver,
>>
>> I have fixed up the problem with condor_ssh_to_job and SELinux. It will
>> be in the upcoming 8.6.7 release.
>>
>> https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=6362
>>
>> ...Tim
>>
>>
>> On 08/03/2017 12:24 PM, Oliver Freyermuth wrote:
>>> Hi Todd, 
>>>
>>> Am 03.08.2017 um 19:02 schrieb Todd Tannenbaum:
>>>> Yes, you can use shared_port on your submit (schedd) machine, setup your firewall to only allow port 9618, and still have condor_ssh_to_job work with CCB.  To make it all work together this way requires that you change the permissions on the directory used by shared_port like so:
>>>>
>>>>    chmod 1777 /var/lock/condor/daemon_sock
>>>>
>>>> (/var/lock/condor is the default, you may need to do "condor_config_val LOCK" to get the actual path HTCondor lock directory).
>>>>
>>>> To understand why this is needed and why it works, read the wisdom for knob DAEMON_SOCKET_DIR in the HTCondor Manual, available at URL http://bit.ly/2u4Jt1F
>>> Wow - thanks for that link, and also for the quick reply!
>>> I totally missed that in the documentation, I was looking around in the more general parts (CCB and network configuration, and condor_ssh_to_job options and manpage),
>>> but never checked there. Really good news we can keep the firewall up with just this (known and needed) hole for usage by HTCondor. 
>>>
>>>>> 2) SELinux policies prevent running ssh-keygen on the startd machine. SELinux denies permission to write the generated keys to /pool/condor/dir_<PID>/.condor_ssh_to_job_1/ .
>>>>>     Is this already fixed in a new version of HTCondor?
>>>>>     This breaks on CentOS 7 out of the box.
>>>>>
>>>> Ugh.  Thank you for investigating and reporting this. We will make a ticket on the wiki.htcondor.org to address this for the next release, and include a work-around in the ticket until the new version is released.   Tim T will post the ticket URL here shortly.
>>> Thanks for taking care of that! 
>>>
>>> Best regards, 
>>> Oliver
>>>
>>>
>>
> 
> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature