Re: [Condor-users] Compressing large files on Condor

[Frederic Magnard <magnard@xxxxxx>]

Hi Alexandra,

On Thu, 19 Jul 2007, Alexandra  Lupu wrote:
> My Condor jobs produce huge (64 MB) .fits files in large numbers, and they
> can take about 4 hours to download to my desktop.  Does anyone know of a
> way to compress these files while they're on the Condor network, so as to
> reduce the download time?

My answer is not really related to condor, but it might help...

64MB is quite small, we have FITS files of around 10GB.
imcopy, from the cfitsio distribution, supports RICE compression for
images, which is quite efficient and fast. But beware that it is lossless
for floating points (bitpix<0) images.

Syntax: imcopy toto.fits 'toto.fits.fz[COMPRESS]'

Cheers,
Fred.