[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] 2 questions about the condor scheduler

On Nov 6, 2008, at 1:17 PM, Mohamed Salah Meddeb wrote:
I will work on real time HD video coding; I’ll implement an algorithm of video coding on grid computing because of the computational needed by the algorithms of video coding such as H264 (MPEG 4 part 10).I've chosen Condor because it is dedicated for the high throughput computing

Real-time is very different from high-throughput. What exactly do you mean by real-time? What are your constraints?

Please, I have 2 questions:
1- Is condor recommended for the multimedia application especially when we work with HD resolution i.e. a tremendous amount of data required for coding

Think of it like this: you write a program to do something--in your case HD video coding--and you have a bunch of computer. Condor will run your program on a bunch of computers. While your program is running, the overhead of running Condor is fairly small. So Condor may or may not be recommended. The question is: can the computers you have run your program?

2- I’ll work on real time HD video coding, after some researches I realized that condor is not suitable for constraint of real time. Is it possible to make some modification on condor’s scheduler to make it support the real time constraint?

Define what you need for real-time. How strict is your schedule?

In general, Condor is not optimized for quick job startup, but once the job is running Condor adds almost no overhead. The way we ship Condor, it may take many seconds, perhaps a minute, to start up a job. In some cases, you can speed up the job startup time, but it will still be on the order of a few seconds.

That said, if you are writing new code and you are willing to fit into an existing framework, you might be interested Condor's Master Worker (MW) framework. Instead of jobs, you write code that thinks about "tasks". MW runs worker jobs that start up, then grab tasks and run them. They can run many tasks, so the overhead of stating up a task is much smaller than starting up a job. Depending on what you need for real-time, it may be good enough for you, but we don't think of it as a system that can meet strict real-time demands, it's just a fairly fast system.



Alain Roy                                         roy@xxxxxxxxxxx
Condor Project                      http://www.cs.wisc.edu/condor