[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Problem with line endings in docker universe

On 8/11/2015 9:12 AM, Matthew Hinton wrote:
We've started to use the docker universe to handle running jobs on our
cluster and are experiencing problems with the output.

For testing we've run the following job on a personal condor
installation (ubuntu 14.04) (version 8.3.6) with docker version 1.7.1

/universe = docker/
/executable = cat/
/arguments = ./hello.in <http://hello.in>/
/docker_image = ubuntu/
/should_transfer_files = YES/
/# contains hello/
/transfer_input_files = ./hello.in <http://hello.in>/
/when_to_transfer_output= ON_EXIT/
/output = hello.out/
/error = hello.err/
/stream_output = True/

where hello.in <http://hello.in> only contains "hello". We get output
files as expected (hello.out, hello.err). However, running *file
hello.out* gives:

hello.out: ASCII text, with CRLF line terminators

whereas normally there would not be CRLF terminators. Running an
interactive docker session doesn't give CRLF terminators, so I can't see
where these are coming from.

We can deal with these when the stdout for the process is a string, but
we're dealing with some binary outputs, which are therefore erroring out
the next step in the pipeline.

Thanks in advance, let me know if you need more information.


Hi Matt,

Wild guess here... You say above that you do not get the CRLF line terminators when you run your container outside of HTCondor via docker run. Could you please try again but use the "--tty" argument to docker run this time? Looks like HTCondor uses --tty with docker run, and I suspect that may be related to the problems you are having above.