Re: [Gems-users] Time spent in synchronization

["soohong p kim" <spkim@xxxxxxxxxx>]

Hemayet,

I have not see any significant increase in synchronization overhead in my
Simics+Ruby (x86+Tango-based Linux target) for SPEC OMP (OpenMP) benchmarks.


BTW, based on my observation, threads migrated from one physical CPU core to
another in a single-CMP Simics target.  How did you handle thread-processor
affinity?  Could you tell us how did you bind a thread to a specific CPU or
CPU core?  And did thread-CPU affinity impact synchronization overhead?

Soohong

Hemayet Hossain wrote:
> ... I have binded each thread to a specific processor (one-to-one)...


-----Original Message-----
From: gems-users-bounces@xxxxxxxxxxx [mailto:gems-users-bounces@xxxxxxxxxxx]
On Behalf Of Dan Gibson
Sent: Friday, December 14, 2007 3:58 AM
To: Gems Users
Subject: Re: [Gems-users] Time spent in synchronization

Its really hard to say if it is reasonable or not without lots of 
details of the real machine and the workloads. However, if the workload 
1) fits in the cache (16MB seems like it might hold a lot of SPLASH-2) 
and 2) Sees a lot of multicycle ALU operations then its reasonably 
likely that synchronization will start to dominate by effectively 
shortening non-synchronization time.

Unfortunately, its rather hard to determine how much increase is really 
reasonable. You can try playing around with the target's cache sizes 
(perhaps sizing the shared cache to the size of a single private cache 
in your SunFire -- /usr/platform/sun4u/sbin/prtdiag -v should do the 
trick), but it is rather hard to make up for the IPC=1 core assumption.

Regards,
Dan

Hemayet Hossain wrote:
> Hi Dan,
> Thanks a lot. Yes, my real system is a 16 processors Sun-FireSystem. I 
> agree with your explanation; but I was wondering whether the 
> difference can be that  much. Like, I am getting  12% -->42%, 
> 28%->75%, 54%->82%, 13%->60% for four different applications (from 
> RUBY_CYCLES). Do you think this is reasonable?
> Thanks once again,
> Hemayet
>
>
> Dan Gibson wrote:
>> I think that is the right approach for measuring simulation time. But is 
>> the increased synchronization time all that surprising? I see from your 
>> response to Mike that your real machine is a 16-processor v9-based 
>> (sun4u -- is it a SunFire of some sort?), and your target is a 
>> 16-processor CMP with a big shared L2.
>>
>> Depending on the performance disparity between real and target machines, 
>> it might be correct to show increased synchronization time under 
>> simulation. (Recall that Simics+Ruby uses a very simple processor model 
>> (IPC=1 for all non-memory operations). This can artificially inflate the 
>> apparent performance of a processor by abstracting away pipeline
details.)
>>
>> Regards,
>> Dan
>>
>> Hemayet Hossain wrote:
>>   
>>> Hi Dan,
>>> I have collected the time through RUBY_CYCLES also. For that I have 
>>> passed the binded proc id (kept in an array in program) with Magic 
>>> calls for start and finish of lock/barriers calls and used that proc 
>>> id for keeping track of which proc is in synchronization and which one 
>>> is not. Do you think my approach is wrong?
>>>
>>> Thanks,
>>> Hemayet
>>>
>>> Dan Gibson wrote:
>>>     
>>>> gethrtime() is bogus under simulation. Solaris's view of time is 
>>>> horribly skewed under Simics alone or Simics+Ruby. Try measuring using 
>>>> RUBY_CYCLES instead.
>>>>
>>>> Hemayet Hossain wrote:
>>>>   
>>>>       
>>>>> Hi All,
>>>>> I am simulating some splash2 benchmarks by using ruby with simics
2.2.19 
>>>>> (Solaris 10) and to characterize the time spent in synchronization, I 
>>>>> have instrumented the synchronization calls like locks and barrier. I 
>>>>> have binded each thread to a specific processor (one-to-one) and 
>>>>> collecting the time by calling high resolution timer gethrtime().  In 
>>>>> real machine run (having 16 processors) for 16 threads I get around
19% 
>>>>> time spent on synchronization for a program. If I run the same program

>>>>> in simics without ruby, I also get similar percentage of time spent in

>>>>> synchronization.
>>>>>
>>>>> But If I run the same program in simics with ruby, the time spent in 
>>>>> synchronization is much higher (goes around 75% of total).  I have 
>>>>> collected the time from both programs and from ruby. Both are getting 
>>>>> almost same percentage number. I am using MESI_SCMP_directory like 
>>>>> protocol having 2 cycles for L1 and 14 cycles for L2 access.
>>>>>
>>>>> Does anyone have any idea what's going on? What wrong with my setup? I

>>>>> would really appreciate your reply.
>>>>> Thanks,
>>>>> Hemayet
>>>>>
>>>>> _______________________________________________
>>>>> Gems-users mailing list
>>>>> Gems-users@xxxxxxxxxxx
>>>>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>>>>> Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>>>>
>>>>>
>>>>>   
>>>>>     
>>>>>         
>>>> _______________________________________________
>>>> Gems-users mailing list
>>>> Gems-users@xxxxxxxxxxx
>>>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>>>> Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>>>   
>>>>       
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Gems-users mailing list
>>> Gems-users@xxxxxxxxxxx
>>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>>> Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>>
>>>   
>>>     
>>
>>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Gems-users mailing list
> Gems-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
> Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>
>   

-- 
http://www.cs.wisc.edu/~gibson [esc]:wq!

_______________________________________________
Gems-users mailing list
Gems-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/gems-users
Use Google to search the GEMS Users mailing list by adding
"site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.