[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] Condor feature suggestion: automatically compressed output files
I recently asked for a similar feature.
For now I just wrote a wrapper script which compresses the output before
it is returned to the submitting machine.
Matthias Röhm, University Ulm,
DaimlerChrysler AG, Research Center Ulm,
Department for Inline Inspection and
Data Mining Solutions, REI/DM
P.O. Box 23 60, 89013 Ulm, Germany
Phone: +49 (0)731 505 4864
Fax: +49 (0)711 3052183085
> Hi all,
> Recently I find myself using Condor in the following way: I have a
> executable that produces large standard output, which I need stored
> further analysis. In each execution, I need to run that executable
> of different parameters, generating hundreds of jobs - and I would
like to be
> able to store these outputs for further reference.
> The problem is, the outputs take a lot of place, and I need to use
> compress them after they execute. However, I cannot just run bzip2
> want, since some of the jobs can be still running, and so compressing
> output file would result in broken output.
> A solution would be to have Condor automatically compress the file
> contents if
> the output (and .err) file name ends with ".bz2" or ".gz".
This should be
> easy to implement as bzip2 library is publicly available.
> I think this feature would be useful not only to me, but to other
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> subject: Unsubscribe
> You can also unsubscribe by visiting
> The archives can be found at either