[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] 2 questions about the condor scheduler
- Date: Thu, 6 Nov 2008 13:48:24 -0600
- From: Alain Roy <roy@xxxxxxxxxxx>
- Subject: Re: [Condor-users] 2 questions about the condor scheduler
On Nov 6, 2008, at 1:17 PM, Mohamed Salah Meddeb wrote:
I will work on real time HD video coding; I’ll implement an
algorithm of video coding on grid computing because of the
computational needed by the algorithms of video coding such as H264
(MPEG 4 part 10).I've chosen Condor because it is dedicated for the
high throughput computing
Real-time is very different from high-throughput. What exactly do you
mean by real-time? What are your constraints?
Please, I have 2 questions:
1- Is condor recommended for the multimedia application especially
when we work with HD resolution i.e. a tremendous amount of data
required for coding
Think of it like this: you write a program to do something--in your
case HD video coding--and you have a bunch of computer. Condor will
run your program on a bunch of computers. While your program is
running, the overhead of running Condor is fairly small. So Condor may
or may not be recommended. The question is: can the computers you have
run your program?
2- I’ll work on real time HD video coding, after some
researches I realized that condor is not suitable for constraint of
Is it possible to make some modification on condor’s scheduler to
make it support the real time constraint?
Define what you need for real-time. How strict is your schedule?
In general, Condor is not optimized for quick job startup, but once
the job is running Condor adds almost no overhead. The way we ship
Condor, it may take many seconds, perhaps a minute, to start up a job.
In some cases, you can speed up the job startup time, but it will
still be on the order of a few seconds.
That said, if you are writing new code and you are willing to fit into
an existing framework, you might be interested Condor's Master Worker
(MW) framework. Instead of jobs, you write code that thinks about
"tasks". MW runs worker jobs that start up, then grab tasks and run
them. They can run many tasks, so the overhead of stating up a task is
much smaller than starting up a job. Depending on what you need for
real-time, it may be good enough for you, but we don't think of it as
a system that can meet strict real-time demands, it's just a fairly
Alain Roy roy@xxxxxxxxxxx
Condor Project http://www.cs.wisc.edu/condor