[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Usage of udp/tcp ports is a little confusing...



Rob wrote:
Hello,

I want to introduce Condor to our university library.

Fantastic!

In that system, I will use one Linux central master for all the
administration and job submissions; the pool consists of all
the public Windows PCs.


* On central master I need to open the Linux firewall for
  incoming UDP port 9618, if I only want to collect availability
  information, without job submissions.

and TCP 9618 - although updates are sent via UDP by default, queries to the collector (e.g. condor_status) will use TCP to port 9618.

  Submitting jobs from the central master, I also have to open
  incoming TCP ports in the 9600-9700 range.
  (The linux firewall allows all outgoing communication!).


Sounds good. BTW, how large is your pool? Note that currently in Condor each running job uses X number of ports on your submitting machine. Back of the napkin, I'd say with the above configuration of giving your central manager+submit machine 100 ports, figure on safely running ~15 jobs at once. So I would suggest adding the following to your central manager condor_config:
  MAX_JOBS_RUNNING = 15

For formulas computing port usage on the submit machine, see section 3.7.1.4 of the Condor Manual:
http://www.cs.wisc.edu/condor/manual/v7.2/3_7Networking_includes.html#SECTION00471000000000000000

BTW, reducing the number of ports used by Condor is something we are thinking about addressing over the next year. The issue is UDP & TCP only support 32k ports max, and we have users that want to have more than 20,000 jobs running from a *single* submission machine (!).

* On the pool PCs, the Windows firewall must allow
  condor_startd  incoming/outgoing communication with the
  central master Linux PC, in order to give Ads info (UDP)
  and to communicate job submissions (TCP).


Sounds fine.



2) If I'm right with the Windows firewall, then I'm confused
    about the firewall exceptions modifications by installing
    the Condor msi package:
        condor_dagman.exe allowed with any computer
        condor_master.exe allowed with any computer
        condor_startd.exe allowed with any computer

    I'm inclined to completely remove the condor_master.exe
    and the condor_dagman.exe firewall exceptions.
    But why are they there in the first place???


The condor_master will need to receive incoming network connections from localhost, the IP address of the windows box itself, and whatever machine(s) are listed in HOSTALLOW_ADMINISTRATOR (by default, the central manager). condor_dagman will need to receive incoming network connectinos from localhost and the IP address of the windows box itself.

Also, I do not think it is the MSI installer package that is adding these exceptions. I think it is the condor_master.exe itself adding these firewall rules by default when the Condor service is started. To disable this behavior, in condor_config add:
   ADD_WINDOWS_FIREWALL_EXCEPTION = False


Thank you!

Rob.


Hope the above is helpful (albeit it is off the top of my head...)

regards,
Todd

--
Todd Tannenbaum                       University of Wisconsin-Madison
Condor Project Research               Department of Computer Sciences
tannenba@xxxxxxxxxxx                  1210 W. Dayton St. Rm #4257