[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-users] condor_hdfs wrapper moving to contrib section

condor_hdfs is a condor daemon which manages the running of the Hadoop distributed filesystems daemon. We've shipped it with Condor since Condor version 7.4, and also included the jar files from the Apache Hadoop project which implement the HDFS. However, Apache HDFS is a fast moving project, and frequently, the version of hdfs jar files we ship has not been the one which Condor users want. This is particularly problematic because of the tight coupling between the APIs in the filesystem and those of tools above it, like Hadoop Map Reduce.

So, in order to increase flexibility for Condor sites, as of Condor 7.8, we will be moving the condor_hdfs daemon to the contrib section, and we will stop distributing hdfs jar files. Instead, we want to encourage sites to download themselves the particular version of hdfs they need.


The Condor Team