Re: [HTCondor-users] Problem with line endings in docker universe

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 8/11/2015 9:12 AM, Matthew Hinton wrote:
> Hi,
> We've started to use the docker universe to handle running jobs on our
> cluster and are experiencing problems with the output.
>
> For testing we've run the following job on a personal condor
> installation (ubuntu 14.04) (version 8.3.6) with docker version 1.7.1
> installed.
>
> /universe = docker/
> /executable = cat/
> /arguments = ./hello.in <http://hello.in>/
> /docker_image = ubuntu/
> /should_transfer_files = YES/
> /# contains hello/
> /transfer_input_files = ./hello.in <http://hello.in>/
> /when_to_transfer_output= ON_EXIT/
> /output = hello.out/
> /error = hello.err/
> /stream_output = True/
> /queue/
>
> where hello.in <http://hello.in> only contains "hello". We get output
> files as expected (hello.out, hello.err). However, running *file
> hello.out* gives:
>
> hello.out: ASCII text, with CRLF line terminators
>
> whereas normally there would not be CRLF terminators. Running an
> interactive docker session doesn't give CRLF terminators, so I can't see
> where these are coming from.
>
> We can deal with these when the stdout for the process is a string, but
> we're dealing with some binary outputs, which are therefore erroring out
> the next step in the pipeline.
>
> Thanks in advance, let me know if you need more information.
>
> Matt

Hi Matt,

Wild guess here...  You say above that you do not get the CRLF line 
terminators when you run your container outside of HTCondor via docker 
run.  Could you please try again but use the "--tty" argument to docker 
run this time?  Looks like HTCondor uses --tty with docker run, and I 
suspect that may be related to the problems you are having above.

Thanks!
Todd