Re: [Condor-users] Python and Condor

[simon kagwe <simonkagwe@xxxxxxxxx>]

Hi,

Thank you Nick for your answers.

 

So does it mean the machines will have to share a filesystem.? I was trying to avoid using NFS or AFS. The whole reason I was going to implement it on Linux was to take advantage of checkpointing and remote system calls.

Nick LeRoy <nleroy@xxxxxxxxxxx> wrote:

> On Wed March 28 2007, simon kagwe wrote:
> > Hi,
> Hello,
>
> > I am supposed to write a program to run on a Condor grid on x86 Linux
> > (Fedora Core 1 and Ubuntu) platform. The task is to show change in
> > computation time when running a compute-intensive program on a grid as
> > opposed to a single computer. Python is my programming language of choice,
> > but there are a few things I need to know before I can start off:
> >
> > 1)Is it possible to
>  run a Python program in standard universe? ( i want
> > to take advantage of checkpointing and remote system calls)
>
> Well, yes and no... There is no native support for Python in Condor, and 
> there is no way to run a python program (indeed, any interpreted program) in 
> the standard universe.
>
> That being said, if you're willing to do some extra work, you can still do it.
>
> The first option is to use Python's built in pickle to checkpoint, and to use 
> chirp for remote I/O.
>
> Option #2: Build your own python interpreter as a Condor standard universe 
> executable, then submit your job with "executable = mypython" and "arguments 
> = myscript.py" ...
>
> > 2)Will I need 
> > to use any parallel programming techniques like message-passing or BSP for
> > the python program to run on the grid in such a way as to reduce
> > computation time?
>
> Depends on your program more than anything. Does your program naturally split
>  
> up so that multiple instances run independently of each other? Look at 
> section 2.4 of the Condor manual "Road-map for Running Jobs" 
> http://www.cs.wisc.edu/condor/manual/v6.8.4/2_4Road_map_Running.html for more 
> advice.
>
> > 3)Will there be need to relink seeing as Python is an 
> > interpreted language?
>
> See above. You can't relink your python program, but you'd have to relink 
> your python interpreter itself.
>
> > 4)Are there interfaces/bindings that I will need to 
> > enable a python script to run on the grid (like Pyglobus for Globus
> > toolkit).
>
> I don't think that you'd need any of these.
>
> > If you can help me with the above questions or if you have any useful
> > information about running python scripts on a Condor grid, please share.
>
> Hope this helps
>
> -Nick
>
> -- 
> <<< The Matrix is everywhere. >>>
> /`-_ Nicholas R. LeRoy The Condor Project
> { }/
>  http://www.cs.wisc.edu/~nleroy http://www.cs.wisc.edu/condor
> \ / nleroy@xxxxxxxxxxx The University of Wisconsin
> |_*_| 608-265-5761 Department of Computer Sciences
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at either
> https://lists.cs.wisc.edu/archive/condor-users/
> http://www.opencondor.org/spaces/viewmailarchive.action?key=CONDOR

---

Be a PS3 game guru.
Get your game face on with the latest PS3 news and previews at Yahoo! Games.