[Gems-users] Token Coherence Protocol Messages

[Ragavendra <natar@xxxxxxxxxx>]

Hello all,

I am investigating the network power consumed by different protocols and 
I have a question regarding the number of messages generated by the 
token coherence protocol.
For my simulations is am using a 16-core machine and using 
MOESI_SMP_token and MOESI_SMP_directory protocols. I am using a 2D torus 
interconnection network.
I am intercepting messages before they are converted to flits 
("flitisized") by the garnet code and printing some statistics such as 
the source, destination, cache miss that lead to this message. I 
consider only data request and response messages. Consider a miss at the 
L1 cache of processor 3 (L1Cache-3) leading to a GETS request. Here are 
the relevant messages from the dump using MOESI_SMP_directory protocol. 
Here 3_0 is a cache miss ID that is assigned by me to uniquely identify 
misses.
Flitisizing 3_0 from L1Cache-3 Dests: 16
Flitisizing 3_0 from Directory-0 Dests: 3
Flitisizing 3_0 from L1Cache-3 Dests: 16

We can see the GETS request to the directory, data sent back, and the 
acknowledgement message. The number indicated in Dest is different since 
it uses a different mapping.
Here are the messages produced for the same miss when using 
MOESI_SMP_token protocol. There are many more messages from L1Cache-3 
with different number of destinations which are not shown here. Many 
messages are duplicates with the same destinations.
Flitisizing 3_0 from L1Cache-3 Dests: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 
15 16
Flitisizing 3_0 from L1Cache-3 Dests: 4 5 6 7 8 9 10 11 12 13 14 15 16
Flitisizing 3_0 from L1Cache-3 Dests: 4 5 6 7 8 9 10 11 12 13 14 15 16
Flitisizing 3_0 from L1Cache-3 Dests: 4 5 6 7 8 9 10 11 12 13 14 15 16
Flitisizing 3_0 from L1Cache-3 Dests: 4 5 6 7 8 9 10 11 12 13 14 15 16
....
....
....
Flitisizing 3_0 from Directory-0 Dests: 3

From my understanding I should see a broadcast message to all the 
caches (plus the corresponding directory) from L1Cache-3. Since there 
are no sharers the directory must respond with the data and the owner 
token. I do not understand why I see the other multicast messages from 
L1Cache-3. Am I missing something?
Any help is greatly appreciated.

Thanks,
Raga