[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Hadoop on Condor



Oddly enough Map-Reduce is what I currently have the most interest in from users.

On Wed, Nov 9, 2011 at 3:15 AM, Rita <rmorgan466@xxxxxxxxx> wrote:
I have started to use it. I think map reduce is irrelevant when you use condor. However, hdfs is extemly useful for streaming large data (filer bigger than 300mb).


On Tue, Nov 8, 2011 at 10:46 PM, Matthew Farrellee <matt@xxxxxxxxxx> wrote:
On 10/24/2011 03:42 PM, David Brodbeck wrote:
I have some users who are interested in running Hadoop jobs on our
Condor cluster.  Googling found me this, which sounds promising but is
thin on implementation details:
http://www.cs.wisc.edu/condor/CondorWeek2010/condor-presentations/thain-condor-hadoop.pdf

I'm wondering if anyone can point me to some more detailed information
on how to get this going?  I don't have any previous experience with
Hadoop, so I'm probably missing something obvious.

--
David Brodbeck
System Administrator, Linguistics
University of Washington

Re HDFS, see http://www.cs.wisc.edu/condor/manual/v7.6/3_13Setting_Up.html#36386

Re MapReduce, see https://condor-wiki.cs.wisc.edu/index.cgi/wiki?p=MapReduce

I've not walked through these, but they're likely your best starting points. Both are maintained by people who read condor-users.

Best,


matt
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxedu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/



--
--- Get your facts first, then you can distort them as you please.--

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/




--
David Brodbeck
System Administrator, Linguistics
University of Washington