[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Bulk submission of Jobs



Hi all,
 
I added those runtime statements sent by Matt but after adding I get the exception when I am submitting the 5592 nd job.  :(
 
-Esh

 
On 7/4/07, Matthew Farrellee <matt@xxxxxxxxxxx> wrote:
Esh,

Did Matt's suggestion reveal anything?

Best,


matt

On Jul 3, 2007, at 6:53 AM, Matt Hope wrote:

> On 7/3/07, Esh Esh <eshforcondor@xxxxxxxxx> wrote:
>> Hi,
>>
>> I am trying to submit 50000 job to condor system (Condor 6.8.2) in
>> loop
>> using webservices. Every time I run this program I get a error after
>> submitting 32600 jobs.
>>
>>  Stack Trace gives:
>>
>> "java.net.connectionexception : Connection Refused"
>>
>> Has any body faced this problem earlier?
>>  Is this specific to condor 6.8.2? Or Is there any limit on the
>> number of
>> jobs that can be submitted?
>
> 32600 sounds suspiciously like you are running out of file handles or
> running out of sockets...
>
> Are you holding open a file or socket on each submission?
>
> If you pop a:
>
> Runtime rt = Runtime.getRunTime();
> rt.gc();
> rt.runFinalization();
>
> after each submission and it then works then it is likely you are
> failing to tidy up your handles. (you could isolate that by seeing how
> long it normally takes to do the gc/finalize and just sleeping for
> that time instead)
>
> If it doesn't then it gets more complex... there were previous
> conversations here about keep-alive on the http connection. That may
> be a factor
>
> Matt
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/