RE: [Condor-users] Greetings, and rtgen over Condor

["Miguel Dilaj" <mdilaj@xxxxxxxxxxxxx>]

Hi Matt et all,

Thanks for the comments. I think that I'm not going to be affected by
evictions because the nodes have been configured to stop and continue later,
not to migrate.
I already took care of indicating an external (big) drive as the
destination, so the files should be transferred back there. BUT, I run into
an unexpected problem...

When I was piloting with 3 dedicated machines, I launched my .sub file, and
got the *.rt (the rainbow tables) back in the central manager's disk, but
now I'm not getting the files back, and I don't see the reason for that...

The .sub is simple, only minor mods since the first version I used for
piloting, mainly due to the fact that I'm using E:\ as the destination now:

	Universe = vanilla
	Executable = rtgen.exe
	Log = rtgeneration.log
	Arguments = lm alpha 1 5 $(Process) 1000 4000
	Initialdir = E:\
	
	# the extra line above is to avoid problems with the backslash and
command continuation!
	Should_transfer_files = YES
	When_to_transfer_output = ON_EXIT
	Transfer_input_files = charset.txt
	Nice_user = True
	Notification = Never
	Requirements = (OpSys == "WINNT50") || (OpSys == "WINNT51")
	Queue 5

Inspecting the log file shows that the jobs finished OK. It can also be
monitored using condor_q during the few seconds it is running. However, the
logs show "0 - Run bytes sent by job", when I expect the table to be
transferred back.

For information to people without experience in rainbow table generation:
Rtgen will run with the arguments provided, and generate an output file with
.rt extension. The file name is dynamically constructed from the arguments,
and I like the idea of AVOID specifying it (in fact it worked OK during my
pilot test!).

What am I doing wrong now???

TIA!
Regards,

Miguel



-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx
[mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Matt Hope
Sent: 02 July 2005 16:07
To: Condor-Users Mail List
Subject: Re: [Condor-users] Greetings, and rtgen over Condor


On 7/1/05, Miguel Dilaj <mdilaj@xxxxxxxxxxxxx> wrote:
> Hi all,
> 
> I'm an absolutely new starter with Condor (5 hours!).

fresh meat :)

> I had Condor 6.6.10 for Win up and running in a few pilot boxes in no 
> time, and run rtgen using a .sub adapted from the rc5.sub in the 
> examples.

This is a Good Idea - I think anyone with a problem on an initial pool setup
should try to submit printname and cpusoak before asking for help :)

> I'm particularly interested in knowing if there're any drawbacks of 
> using Condor in this scenario, where biiiiig files are generated on 
> each node.

There are a few. For starters you should ensure any large files which aren't
needed back on the submitter are deleted or created in a sub directory of
the initial dir since these are not transferred automatically.

Second if you are supporting manual checkpointing it is quite possible to
blitz your schedd with a mass eviction (I have seen a dual Xeon machine with
scsi disks and 100M network annihilated by 40 evictions each with about 1/2
Gig of data). There are mechanisms to slow the starting of jobs but not the
pace of eviction. IF you want the evictions to succeed with significant
amounts of data I strongly suggest increasing the KILL setting to allow up
to 30minutes!

The space for the execute directory would ideally be allocated on a sizable
disk separate from the main OS install. The significant extra load on the
disk may well increase the chance of failure as well as potentially annoying
the provider if it fills up.

Best of luck!

Matt

_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/condor-users


***********************************************************************************************************
DISCLAIMER:                                                                                                
This e-mail contains proprietary information, some or all of which may be legally privileged.              
It is for the intended recipient only. If an addressing or transmission error has misdirected this e-mail, 
please notify the author by replying to this e-mail. If you are not the intended recipient you may not use,
disclose, distribute, copy, print or rely on this e-mail.                                                  
***********************************************************************************************************