Re: [Gems-users] Please help to check my scripts for cache warming up

[ruzhu kao <kaoruzhu@xxxxxxxxx>]

Hi Faraz:
   Thanks for your suggestion.

I have tested cold simics script without setting
instruction-fetch-mode and i/dstc-disable. It seems no difference at
least the result dump from ruby about the XACT_BREAKDOWN_CYCLE maybe
it just affect simics running environment.

I use the similar way  to gen-scripts.py to generate a sh script to
run the workload. I create a new sh to call simics like this way:
#############################################################
cd /opt/develop/gems/gems-2.1/simics/home/MESI_CMP_filter_directory
./simics -echo -verbose -no-log -no-win -stall -x
../../../gen-scripts/microbench-cold.simics
###############################################################################
I changed the microbench-cold.simics from microbench.simics just make
it call ssca2-cold.simics. The warm scripts are use the same calling
pattern.

  In microbench.start_ruby I used it to test ssca in my clean
checkpoint and there is no problem to count the XACT_BREAKDOWN_CYCLE,
so I assume it would have no problem to load and set ruby correctly.
  Do I miss any other point. Please point my mistake.

Thanks in advanced.

Sincerely your:
Ruzhu

2009/10/27 Faraz Ahmad <faraz@xxxxxxxxxx>:
> The following steps may help:
> 1. In cold simics script, you do not need to execute instruction-fetch-mode,
> instruction-fetch-trace, istc-disable, dstc-disable commands because you are
> not loading ruby.
> 2. In warm simics script, you should execute above commands because now you
> want to load ruby. Also make sure that you start simics with the -stall
> flag.
> 3. In microbench.start_ruby, make sure that it loads the ruby module, sets
> memory and cache parameters correctly and executes ruby0.init.
>
>
> ----- Original Message -----
> From: "ruzhu kao" <kaoruzhu@xxxxxxxxx>
> To: "Gems Users" <gems-users@xxxxxxxxxxx>
> Sent: Monday, October 26, 2009 9:44 PM
> Subject: [Gems-users] Please help to check my scripts for cache warming up
>
>
>> Hi all:
>>     I have read the relating topics in ISCA-tutorial and this list. I
>> assume it needs three steps and then I write these scripts for cold,
>> warm and run respectively. However, it seems that I should miss some
>> points which lead to some failure.  The warm and run steps take a long
>> time and show strange behavior. Can you help to check my scripts and
>> point out the problem.
>>
>> #***************below is the cold simics script******************#
>> @import microbench, workloads, mfacet
>>
>> @processors = 16
>> @ruby_proc_map = (1, 2, 2, 4, 4, 8, 8, 8, 8, 16, 16, 16, 16, 16, 16, 16,
>> 16)
>> @if(processors <= 16):
>>    ruby_procs = ruby_proc_map[processors]
>>    proc_no = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
>> elif (processors <= 32):
>>    ruby_procs = 32
>>    proc_no = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
>> 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47]
>> else:
>>    processors = -1
>>
>> @transactions = 1
>> @dump_interval = 1
>>
>> @print "running simics with %d processors" % ruby_procs
>>
>> #############################################
>> # Load a clean checkpoint                   #
>> #############################################
>> @mfacet.run_sim_command('read-configuration
>> "../../../checkpoints/conf_files/2G_16p/step_2.conf"')
>> @mfacet.run_sim_command('magic-break-enable')
>>
>>
>> @sys.path.append("../../gen-scripts")
>>
>> @hostpath = "/host" + os.getcwd() +
>> "/../../../microbenchmarks/transactional/stamp/ssca2/"
>>
>>
>> @command_lines = [
>>    "bash\n",
>>    "cp " + hostpath + "ssca2" + " ./\n",
>>    "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib\n",
>>    "export PATH=$PATH:/usr/local/bin\n",
>>    "./ssca2 -s13 -i1.0 -u1.0 -l3 -p3 -t16\n" ]
>>
>> @mfacet.console_commands(command_lines,"#")
>>
>> ############################################################################
>> # Copy the binary and execute it in vm  till the first magic breakpoint
>> #
>> ############################################################################
>> @mfacet.run_sim_command('instruction-fetch-mode instruction-fetch-trace')
>> @mfacet.run_sim_command('istc-disable')
>> @mfacet.run_sim_command('dstc-disable')
>> @mfacet.run_sim_command('cpu-switch-time 1')
>> @mfacet.run_sim_command('c')
>> @print "encounter the first magic breakpoint"
>>
>> ######################################################################
>> # Continue the executing because it is just for a cold checkpoint    #
>> ######################################################################
>> @mfacet.setup_run_for_n_transactions(transactions, dump_interval)
>> @print "running ssca2 on simics + ruby with %d transactions" %
>> transactions
>> @mfacet.run_sim_command('c')
>> @print "encounter the second magic breakpoint"
>>
>> #############################################################
>> # Execute till the third breakpoint                         #
>> #############################################################
>> @mfacet.run_sim_command('c')
>> @print "encounter the third magic breakpoint"
>>
>> ##############################################################
>> # Save a cold checkpoint [App]-cold-[clean-checkpoint].conf  #
>> ##############################################################
>> @mfacet.run_sim_command('write-configuration
>> "../../../checkpoints/conf_files/2G_16p/ssca2-cold-step_2.conf"')
>>
>>
>> quit
>> #-----------------
>> #***************above is the cold simics script******************#
>>
>> -------------------------------------------------------------------------------------------------------
>>
>>
>> #***************below is the warm simics script******************#
>> @import microbench, workloads, mfacet
>>
>> @env_dict = workloads.prepare_env_dictionary(simics = 1)
>> @processors = 16
>> @ruby_proc_map = (1, 2, 2, 4, 4, 8, 8, 8, 8, 16, 16, 16, 16, 16, 16, 16,
>> 16)
>> @if(processors <= 16):
>>    ruby_procs = ruby_proc_map[processors]
>>    proc_no = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
>> elif (processors <= 32):
>>    ruby_procs = 32
>>    proc_no = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
>> 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47]
>> else:
>>    processors = -1
>>
>> @print "running simics with %d processors" % ruby_procs
>>
>>
>> @lock_type = 'TM'
>> @transactions = 1
>> @dump_interval = 1
>> @env_dict = workloads.prepare_env_dictionary(simics = 1)
>> @visualizer = 0
>>
>> #############################################
>> # Load the cold checkpoint                  #
>> #############################################
>> @mfacet.run_sim_command('read-configuration
>> "../../../checkpoints/conf_files/2G_16p/ssca2-cold-step_2.conf"')
>> @mfacet.run_sim_command('magic-break-enable')
>>
>>
>> @sys.path.append("../../gen-scripts")
>>
>> @hostpath = "/host" + os.getcwd() +
>> "/../../../microbenchmarks/transactional/stamp/ssca2/"
>>
>>
>> @command_lines = [
>>    "bash\n",
>>    "cp " + hostpath + "ssca2" + " ./\n",
>>    "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib\n",
>>    "export PATH=$PATH:/usr/local/bin\n",
>>    "psrset -i\n",
>>    "./ssca2 -s13 -i1.0 -u1.0 -l3 -p3 -t16\n" ]
>>
>> @mfacet.console_commands(command_lines,"#")
>>
>> ############################################################################
>> # Copy the binary and execute it in vm  till the first magic breakpoint
>> #
>> ############################################################################
>> @mfacet.run_sim_command('c')
>> @print "encounter the first magic breakpoint"
>>
>> #################################################
>> # Configure simics and load ruby                #
>> #################################################
>> @microbench.start_ruby(debug=0)
>>
>> ######################################################################
>> # Warmup the cache to run the parallel part of the application       #
>> ######################################################################
>> @mfacet.setup_run_for_n_transactions(transactions, dump_interval)
>> @print "running ssca2 on simics + ruby with %d transactions" %
>> transactions
>> @mfacet.run_sim_command('c')
>> @print "encounter the second magic breakpoint"
>>
>> #########################################
>> # Save the cache file                   #
>> #########################################
>> @cache_filename_prefix = os.getcwd() +
>> "/../../../microbenchmarks/transactional/stamp/ssca2/" + "ssca2-warm"
>> @mfacet.run_sim_command('ruby0.save-caches %s.cache.save' %
>> cache_filename_prefix)
>>
>> @filename_prefix = os.getcwd() +
>> "/../../../microbenchmarks/transactional/stamp/ssca2/" +
>> "ssca2-warm-dump"
>> @if visualizer == 1:
>>  mfacet.run_sim_command('ruby0.xact-visualizer-file %s.visual' %
>> filename_prefix)
>> @mfacet.run_sim_command('ruby0.dump-stats %s.stats' % filename_prefix)
>>
>> ##############################################################
>> # Save a warm checkpoint [App]-warm-[clean-checkpoint].conf  #
>> ##############################################################
>> @mfacet.run_sim_command('write-configuration
>> "../../../checkpoints/conf_files/2G_16p/ssca2-warm-step_2.conf"')
>>
>>
>> quit
>> #-----------------
>> #***************above is the warm simics script******************#
>>
>> ---------------------------------------------------------------------------------------------------
>>
>>
>> #***************below is the run simics script******************#
>> @import microbench, workloads, mfacet
>>
>> @env_dict = workloads.prepare_env_dictionary(simics = 1)
>> #@processors = int(workloads.get_var(env_dict, "PROCESSORS"))
>> @processors = 16
>> #                 0  1  2  3  4  5  6  7  8   9  10  11  12  13  14  15
>> @ruby_proc_map = (1, 2, 2, 4, 4, 8, 8, 8, 8, 16, 16, 16, 16, 16, 16, 16,
>> 16)
>> @if(processors <= 16):
>>    ruby_procs = ruby_proc_map[processors]
>>    proc_no = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
>> elif (processors <= 32):
>>    ruby_procs = 32
>>    proc_no = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
>> 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47]
>> else:
>>    processors = -1
>>
>> @print "running simics with %d processors" % ruby_procs
>>
>>
>> @lock_type = 'TM'
>> @transactions = 1
>> @dump_interval = 1
>> @env_dict = workloads.prepare_env_dictionary(simics = 1)
>> @visualizer = 0
>>
>>
>>
>> @mfacet.run_sim_command('read-configuration
>> "../../../checkpoints/conf_files/2G_16p/ssca2-warm-step_2.conf"')
>> @mfacet.run_sim_command('magic-break-enable')
>>
>>
>> @sys.path.append("../../gen-scripts")
>>
>> @hostpath = "/host" + os.getcwd() +
>> "/../../../microbenchmarks/transactional/stamp/ssca2/"
>>
>>
>> @command_lines = [
>>    "bash\n",
>>    "cp " + hostpath + "ssca2" + " ./\n",
>>    "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib\n",
>>    "export PATH=$PATH:/usr/local/bin\n",
>>    "psrset -i\n",
>>    "./ssca2 -s13 -i1.0 -u1.0 -l3 -p3 -t16\n" ]
>>
>> @mfacet.console_commands(command_lines,"#")
>> @mfacet.run_sim_command('c')
>> @print "first magic break point"
>>
>> @microbench.start_ruby(debug=0)
>>
>> ###############################
>> # Load cache data             #
>> ###############################
>> @warmed_cache_file = os.getcwd() +
>> "/../../../microbenchmarks/transactional/stamp/ssca2/" +
>> "ssca2-warm.cache.save"
>> @mfacet.run_sim_command('ruby0.load-caches %s' % warmed_cache_file)
>> @mfacet.run_sim_command('ruby0.clear-stats')
>> @mfacet.setup_run_for_n_transactions(transactions, dump_interval)
>> @print "running ssca2 on simics + ruby with %d transactions" %
>> transactions
>> @mfacet.run_sim_command('c')
>>
>> @print "second magic break point"
>> @filename_prefix = os.getcwd() +
>> "/../../../microbenchmarks/transactional/stamp/ssca2/" +
>> "ssca2-warmed-result"
>> @if visualizer == 1:
>>  mfacet.run_sim_command('ruby0.xact-visualizer-file %s.visual' %
>> filename_prefix)
>> @mfacet.run_sim_command('ruby0.dump-stats %s.stats' % filename_prefix)
>>
>>
>> quit
>> #-----------------
>> #***************above is the run simics script******************#
>>
>> -----------------------------------------------------------------------------------------------------
>>
>>  When I use run simics scripts load step_2.conf, the stats file
>> generated shows counters work well like this.
>> XACT CYCLE BREAKDOWN
>> XACT_BREAKDOWN_NON_TRANS_CYCLES:  48557592
>> XACT_BREAKDOWN_TRANS_CYCLES:      3398260
>> XACT_BREAKDOWN_GOOD_TRANS_CYCLES: 3357808
>> XACT_BREAKDOWN_ABORTING_CYCLES:   89996
>> XACT_BREAKDOWN_COMMITING_CYCLES:  0
>> XACT_BREAKDOWN_BACKOFF_CYCLES:    5051407
>> XACT_BREAKDOWN_BARRIER_CYCLES:    22622226
>> XACT_BREAKDOWN_STALL_CYCLES:      110658
>> However, when I check the stats file from warm scripts, these counters
>> look strange like this:
>> XACT CYCLE BREAKDOWN
>> XACT_BREAKDOWN_NON_TRANS_CYCLES:  122136
>> XACT_BREAKDOWN_TRANS_CYCLES:      0
>> XACT_BREAKDOWN_GOOD_TRANS_CYCLES: 0
>> XACT_BREAKDOWN_ABORTING_CYCLES:   0
>> XACT_BREAKDOWN_COMMITING_CYCLES:  0
>> XACT_BREAKDOWN_BACKOFF_CYCLES:    0
>> XACT_BREAKDOWN_BARRIER_CYCLES:    672
>> XACT_BREAKDOWN_STALL_CYCLES:      0
>>
>> Any suggestions would be welcomed.
>>    Thanks in advance.
>>
>>
>> Sincerely yours:
>> Ruzhu
>> _______________________________________________
>> Gems-users mailing list
>> Gems-users@xxxxxxxxxxx
>> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
>> Use Google to search the GEMS Users mailing list by adding
>> "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>>
>>
>
> _______________________________________________
> Gems-users mailing list
> Gems-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/gems-users
> Use Google to search the GEMS Users mailing list by adding "site:https://lists.cs.wisc.edu/archive/gems-users/"; to your search.
>
>