[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] Compressing large files on Condor
- Date: Thu, 19 Jul 2007 19:20:15 +0100
- From: Si Hammond <simon.hammond@xxxxxxxxx>
- Subject: Re: [Condor-users] Compressing large files on Condor
Can you not make one large compressed file say a zip and then use a
script to decompress these when the job is running?
On Thu, 19 Jul 2007, Alexandra Lupu wrote:
My Condor jobs produce huge (64 MB) .fits files in large numbers,
can take about 4 hours to download to my desktop. Does anyone
know of a
way to compress these files while they're on the Condor network,
so as to
reduce the download time?
My answer is not really related to condor, but it might help...
64MB is quite small, we have FITS files of around 10GB.
imcopy, from the cfitsio distribution, supports RICE compression for
images, which is quite efficient and fast. But beware that it is
for floating points (bitpix<0) images.
Syntax: imcopy toto.fits 'toto.fits.fz[COMPRESS]'
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
You can also unsubscribe by visiting
The archives can be found at: