Mailing List Archives Public Access	UW Madison Computer Sciences Department Computer Systems Lab
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] How to filter out hyperthreading CPU cores

Date: Thu, 10 Nov 2011 14:10:29 +0100
From: Hermann Fuchs <hermann.fuchs@xxxxxxxxxxxxxxxx>
Subject: Re: [Condor-users] How to filter out hyperthreading CPU cores
Hi

Thank you very much for your answer, it really helped.
I was told this machine has only one CPU. Seems that was wrong.

Thanks again,
Hermann

On Wed, 2011-11-09 at 15:15 +0100, Lukas Slebodnik wrote:
> I will try to answer next questions:
> 
> 1. How many physical processors are there?
>     grep 'physical id' /proc/cpuinfo | sort | uniq | wc -l
> 
>     In your case result is 2.
> 2.Are the processors dual-core (quad-core or multi-core)?
>     grep 'cpu cores' /proc/cpuinfo
>     
>     Output from yours previous mail:
>     cpu cores       : 4
>     cpu cores       : 4
>     cpu cores       : 4
>     cpu cores       : 4
>     cpu cores       : 4
>     cpu cores       : 4
>     cpu cores       : 4
>     cpu cores       : 4
> 
> "4" indicates the two physical processors (physical id) are quad-core,
>  resulting in 8 virtual processors.
> 
> 
> And last helpfull information: If the number of cores == the number of
> siblings for a given physical processor, then hyperthreading is OFF. 
> 
> More details about hyperthreading and /proc/cpuinfo you can find on page
> http://www.richweb.com/cpu_info
> 
> Regards,
> Lukas
> 
> On Wed, Nov 09, 2011 at 08:25:58AM +0100, Hermann Fuchs wrote:
> > Additionally I can offer the /proc/cpu output
> > 
> > To me it is pretty clear that every second cpu mentionned here is a
> > hyperthreading core, if you look at the core id.
> > 
> > cat /proc/cpuinfo
> > processor	: 0
> > vendor_id	: GenuineIntel
> > cpu family	: 6
> > model		: 15
> > model name	: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz
> > stepping	: 11
> > cpu MHz		: 2327.439
> > cache size	: 4096 KB
> > physical id	: 0
> > siblings	: 4
> > core id		: 0
> > cpu cores	: 4
> > apicid		: 0
> > initial apicid	: 0
> > fpu		: yes
> > fpu_exception	: yes
> > cpuid level	: 10
> > wp		: yes
> > flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
> > constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64
> > monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts
> > tpr_shadow vnmi flexpriority
> > bogomips	: 4654.87
> > clflush size	: 64
> > cache_alignment	: 64
> > address sizes	: 38 bits physical, 48 bits virtual
> > power management:
> > 
> > processor	: 1
> > vendor_id	: GenuineIntel
> > cpu family	: 6
> > model		: 15
> > model name	: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz
> > stepping	: 11
> > cpu MHz		: 2327.439
> > cache size	: 4096 KB
> > physical id	: 1
> > siblings	: 4
> > core id		: 0
> > cpu cores	: 4
> > apicid		: 4
> > initial apicid	: 4
> > fpu		: yes
> > fpu_exception	: yes
> > cpuid level	: 10
> > wp		: yes
> > flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
> > constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64
> > monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts
> > tpr_shadow vnmi flexpriority
> > bogomips	: 4655.06
> > clflush size	: 64
> > cache_alignment	: 64
> > address sizes	: 38 bits physical, 48 bits virtual
> > power management:
> > 
> > processor	: 2
> > vendor_id	: GenuineIntel
> > cpu family	: 6
> > model		: 15
> > model name	: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz
> > stepping	: 11
> > cpu MHz		: 2327.439
> > cache size	: 4096 KB
> > physical id	: 0
> > siblings	: 4
> > core id		: 1
> > cpu cores	: 4
> > apicid		: 1
> > initial apicid	: 1
> > fpu		: yes
> > fpu_exception	: yes
> > cpuid level	: 10
> > wp		: yes
> > flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
> > constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64
> > monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts
> > tpr_shadow vnmi flexpriority
> > bogomips	: 4655.05
> > clflush size	: 64
> > cache_alignment	: 64
> > address sizes	: 38 bits physical, 48 bits virtual
> > power management:
> > 
> > processor	: 3
> > vendor_id	: GenuineIntel
> > cpu family	: 6
> > model		: 15
> > model name	: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz
> > stepping	: 11
> > cpu MHz		: 2327.439
> > cache size	: 4096 KB
> > physical id	: 1
> > siblings	: 4
> > core id		: 1
> > cpu cores	: 4
> > apicid		: 5
> > initial apicid	: 5
> > fpu		: yes
> > fpu_exception	: yes
> > cpuid level	: 10
> > wp		: yes
> > flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
> > constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64
> > monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts
> > tpr_shadow vnmi flexpriority
> > bogomips	: 4655.05
> > clflush size	: 64
> > cache_alignment	: 64
> > address sizes	: 38 bits physical, 48 bits virtual
> > power management:
> > 
> > processor	: 4
> > vendor_id	: GenuineIntel
> > cpu family	: 6
> > model		: 15
> > model name	: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz
> > stepping	: 11
> > cpu MHz		: 2327.439
> > cache size	: 4096 KB
> > physical id	: 0
> > siblings	: 4
> > core id		: 2
> > cpu cores	: 4
> > apicid		: 2
> > initial apicid	: 2
> > fpu		: yes
> > fpu_exception	: yes
> > cpuid level	: 10
> > wp		: yes
> > flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
> > constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64
> > monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts
> > tpr_shadow vnmi flexpriority
> > bogomips	: 4655.02
> > clflush size	: 64
> > cache_alignment	: 64
> > address sizes	: 38 bits physical, 48 bits virtual
> > power management:
> > 
> > processor	: 5
> > vendor_id	: GenuineIntel
> > cpu family	: 6
> > model		: 15
> > model name	: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz
> > stepping	: 11
> > cpu MHz		: 2327.439
> > cache size	: 4096 KB
> > physical id	: 1
> > siblings	: 4
> > core id		: 2
> > cpu cores	: 4
> > apicid		: 6
> > initial apicid	: 6
> > fpu		: yes
> > fpu_exception	: yes
> > cpuid level	: 10
> > wp		: yes
> > flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
> > constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64
> > monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts
> > tpr_shadow vnmi flexpriority
> > bogomips	: 4655.05
> > clflush size	: 64
> > cache_alignment	: 64
> > address sizes	: 38 bits physical, 48 bits virtual
> > power management:
> > 
> > processor	: 6
> > vendor_id	: GenuineIntel
> > cpu family	: 6
> > model		: 15
> > model name	: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz
> > stepping	: 11
> > cpu MHz		: 2327.439
> > cache size	: 4096 KB
> > physical id	: 0
> > siblings	: 4
> > core id		: 3
> > cpu cores	: 4
> > apicid		: 3
> > initial apicid	: 3
> > fpu		: yes
> > fpu_exception	: yes
> > cpuid level	: 10
> > wp		: yes
> > flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
> > constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64
> > monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts
> > tpr_shadow vnmi flexpriority
> > bogomips	: 4655.05
> > clflush size	: 64
> > cache_alignment	: 64
> > address sizes	: 38 bits physical, 48 bits virtual
> > power management:
> > 
> > processor	: 7
> > vendor_id	: GenuineIntel
> > cpu family	: 6
> > model		: 15
> > model name	: Intel(R) Xeon(R) CPU           E5345  @ 2.33GHz
> > stepping	: 11
> > cpu MHz		: 2327.439
> > cache size	: 4096 KB
> > physical id	: 1
> > siblings	: 4
> > core id		: 3
> > cpu cores	: 4
> > apicid		: 7
> > initial apicid	: 7
> > fpu		: yes
> > fpu_exception	: yes
> > cpuid level	: 10
> > wp		: yes
> > flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
> > constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64
> > monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca lahf_lm dts
> > tpr_shadow vnmi flexpriority
> > bogomips	: 4655.06
> > clflush size	: 64
> > cache_alignment	: 64
> > address sizes	: 38 bits physical, 48 bits virtual
> > power management:
> > 
> > Regards,
> > Hermann
> > 
> > 
> > On Tue, 2011-11-08 at 22:48 -0500, Matthew Farrellee wrote:
> > > I don't know specifically, but you can run "env 
> > > _CONDOR_TOOL_DEBUG=D_FULLDEBUG condor_config_val -debug NUM_CPUS" to get 
> > > a look at how condor is detecting the number of CPUs.
> > > 
> > > Best,
> > > 
> > > 
> > > matt
> > > 
> > > On 10/21/2011 08:21 AM, Hermann Fuchs wrote:
> > > > Hello
> > > >
> > > > It seems
> > > > COUNT_HYPERTHREAD_CPUS=FALSE
> > > > has no effect. Still all virtual hyperthreaded cores show up.
> > > > Using NUM_CPUS=4
> > > > works as expected.
> > > > However I would prefer a more automated way of detection.
> > > >
> > > > Do you have any idea why COUNT_HYPERTHREAD_CPUS=FALSE could not work?
> > > > On this machine Ubuntu 11.04 64bit is employed.
> > > >
> > > > Yours,
> > > > Hermann
> > > >
> > > > On Fri, 2011-10-21 at 07:31 -0400, Matthew Farrellee wrote:
> > > >> On 10/21/2011 04:38 AM, Hermann Fuchs wrote:
> > > >>> Hello
> > > >>>
> > > >>> In our condor cluster we have several workstations configured to use
> > > >>> hyperthreading (e.g. virtual CPU cores from INTEL CPUs).
> > > >>> Condor seems to recognise virtual hyperthreaded CPU cores as real
> > > > ones
> > > >>> and evenly divides recources.
> > > >>>
> > > >>> For example a workstation with a four core Intel CPU would then
> > > > appear
> > > >>> as an eight core workstation, which we would like to avoid.
> > > >>>
> > > >>> We do not want to use such virtual CPU cores but can not enforce
> > > > their
> > > >>> hardware sided (e.g in the BIOS) deactivation. Is there a way that
> > > >>> condor somehow "filters" them out and announces only real CPU cores?
> > > >>>
> > > >>> Can condor distinguish real cores from virtual ones?
> > > >>> I suppose manually declaring 4 CPU cores may not work because all 4
> > > >>> declared cores might be running on virtual cores...
> > > >>>
> > > >>> Yours,
> > > >>> Hermann
> > > >>
> > > >> You could configure NUM_CPUS=4, but try COUNT_HYPERTHREAD_CPUS=FALSE
> > > > first.
> > > >>
> > > >> http://www.cs.wisc.edu/condor/manual/v7.6/3_3Configuration.html#17736
> > > >>
> > > >> COUNT_HYPERTHREAD_CPUS
> > > >>       This macro controls how Condor sees hyper threaded processors.
> > > > When
> > > >> set to True (the default), it includes virtual CPUs in the default
> > > > value
> > > >> of NUM_CPUS. On dedicated cluster nodes, counting virtual CPUs can
> > > >> sometimes improve total throughput at the expense of individual job
> > > >> speed. However, counting them on desktop workstations can interfere
> > > > with
> > > >> interactive job performance.
> > > >>
> > > >> Best,
> > > >>
> > > >>
> > > >> matt
> > > >>
> > > >
> > > 
> > > 
> > 
> > -- 
> > -------------
> > DI Hermann Fuchs
> > Div. Medical Radiation Physics
> > Department of Radiotherapy
> > Med. Univ. Vienna / AKH Vienna
> > Währinger Gürtel 18-20
> > A-1090 Wien
> > 
> > Tel.  + 43 / 1 / 40 400 7271
> > Mail. hermann.fuchs@xxxxxxxxxxxxxxxx
> > 
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > 
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/

-- 
-------------
DI Hermann Fuchs
Div. Medical Radiation Physics
Department of Radiotherapy
Med. Univ. Vienna / AKH Vienna
Währinger Gürtel 18-20
A-1090 Wien

Tel.  + 43 / 1 / 40 400 7271
Mail. hermann.fuchs@xxxxxxxxxxxxxxxx
References:
- Re: [Condor-users] How to filter out hyperthreading CPU cores
  - From: Matthew Farrellee
- Re: [Condor-users] How to filter out hyperthreading CPU cores
  - From: Hermann Fuchs
- Re: [Condor-users] How to filter out hyperthreading CPU cores
  - From: Lukas Slebodnik
Prev by Date: Re: [Condor-users] Can't add machines into my condor pool
Next by Date: Re: [Condor-users] Quick Start Vanilla Condor on Ubuntu 10.04
Previous by thread: Re: [Condor-users] How to filter out hyperthreading CPU cores
Next by thread: [Condor-users] Why isn't the negotiation cycle finding this job?
Index(es):
- Date
- Thread
Mailing List Archives

Public Access

Re: [Condor-users] How to filter out hyperthreading CPU cores
