[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] Condor and Python
- Date: Mon, 8 Nov 2010 22:37:33 -0600
- From: Nick LeRoy <nleroy@xxxxxxxxxxx>
- Subject: Re: [Condor-users] Condor and Python
> > I write a python script which takes too much time to run in my Mac... I
> > therefore turn to Condor for help. I've learned the way to run Matlab
> > script in Condor so I guess running Python should be similar.
> It depends on how you plan to speed things up.
> I assume the speed up with Matlab comes from a discretisation of
> the computation and using the Matlab Compiler Runtime to process
> those, then smaller run time, computational units in an embarassingly
> parallel batch fashion.
> You can do the same for the "loops" in your python scripts and just
> submit them into your Condor pool as you would any other script,
> assuming python is somewhere on the path available for you on the
> resources uderlying your Condor pool.
> I guess you could compile the python too, should it not be on the
> remote boxes, thereby paralleling the Matlab use case even further.
> I am less familiar with the supporting environment needed for the
> running of compiled python though.
> If, however, your Matlab speed up comes from utilising one their
> parallel toolboxes, you'll need to find similar packages for your
> python environment which would then require some tailoring of the
> underlying Condor "universe" too, to do the explicit, message passing,
I recently learned of matplotlib (http://matplotlib.sourceforge.net/) which is
a Python library that provides much of Matlab's functionality in a Python