[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Multiple Collectors + CCB & Shared Port

# Ability for multiple sub-collectors on a single machine
COLLECTOR.NumProcesses = 4

I don't see this in the manual, and as far as I know is unnecessary for what you're doing. There's COLLECTOR_QUERY_WORKERS, which is how many times the collector will fork to answer a query, but that's not really the same thing.

COLLECTOR_HOST                  = $(CONDOR_HOST):20000

This is a little weird, because it will make the collector at port 20000 the 'main' collector, and everything will query the collector behind the shared port by default, which won't know everything.

If all you want is some extra CCBs, you don't need to change COLLECTOR_HOST at all. You need to set CCB_ADDRESS in the clients, but that's all you need to do.

Can somebody point me in a direction what I'm doing wrong regarding the combination of CCB + Shared Port together with multiple sub-collectors?

If you /want/ to have multiple ports open, you need to turn off shared port in the sub-collectors. The standard way to do this is to add _CONDOR_USE_SHARED_PORT=FALSE to the environment line you already have defined.

If you /can't/ have multiple ports open, then using -sock in the _ARGS configuration should set the shared port socket name for those collectors. If you're just using the sub-collectors for CCB, you should be able to set an execute node CCB_ADDRESS to a string like without any further worries. (This may have to be '<>' -- with the <brackets> -- or could be different, depending on other details of your networking configuration; the easiest way to find it is probably to look up another daemon (not the collector) running on the same machine and replace the 'sock=value' appropriately.) If you want the sub-collectors to help deal with expensive security sessions, you'll need to set the execute node's COLLECTOR_HOST instead, which may cause further problems (IIRC, the default security configuration uses COLLECTOR_HOST, but strings of the form above don't work in security).

If you go for multiple collectors behind a single shared port, you may have some trouble configuring your execute nodes to actually do the load balancing. RANDOM_CHOICE() may approximate what you want sufficiently well to get started.

I'm not sure there's a way to support both arbitrary socket numbers and shared port (in the same daemon), although you'd have to explain to me why you'd want to do that.

- ToddM