Calculating actual/effective CPI for 3 level cache

Question

(a) You are given a memory system that has two levels of cache (L1 and L2). Following are the specifications:

• Hit time of L1 cache: 2 clock cycles
• Hit rate of L1 cache: 92%
• Miss penalty to L2 cache (hit time of L2): 8 clock cycles
• Hit rate of L2 cache: 86%
• Miss penalty to main memory: 37 clock cycles
• Assume for the moment that hit rate of main memory is 100%.

Given a 2000 instruction program with 37% data transfer instructions (loads/stores), calculate the CPI (Clock Cycles per Instruction) for this scenario.

For this part, I calculated it like this (am I doing this right?):

(m1: miss rate of L1, m2: miss rate of L2)

AMAT = HitTime_L1 + m1*(HitTime_L2 + m2*MissPenalty_L2)
CPI(actual) = CPI(ideal) + (AMAT - CPI(ideal))*AverageMemoryAccess

(b) Now lets add another level of cache, i.e., L3 cache between the L2 cache and the main memory. Consider the following:

• Miss penalty to L3 cache (hit time of L3 cache): 13 clock cycles
• Hit rate of L3 cache: 81%
• Miss penalty to main memory: 37 clock cycles
• Other specifications remain as part (a)

For the same 2000 instruction program (which has 37% data transfer instructions), calculate the CPI.

(m1: miss rate of L1, m2: miss rate of L2, m3: miss rate of L3)

AMAT = HitTime_L1 
         + m1*(HitTime_L2 + m2*MissPenalty_L2)
           + m2*(HitTime_L3 + m3*MissPenalty_L3)

Is this formula correct and where do I add the miss penalty to main memory in this formula? It should probably be added with the miss penalty of L3 but I am not sure.

Answer 1

(a) The AMAT calculation is correct if you notice that the MissPenalty_L2 parameter is what you called Miss penalty to main memory.

The CPI is a bit more difficult. First of all, let's assume that the CPU is not pipelined (sequential processor).

There are 1.37 memory accesses per instruction (one access to fetch the instruction and 0.37 due to data transfer instructions). The ideal case is that all memory acceses hit in the L1 cache. So, knowing that:

CPI(ideal) = CPI(computation) + CPI(mem) = 
             CPI(computation) + Memory_Accesses_per_Instruction*HitTime_L1 =
             CPI(computation) + 1.37*HitTime_L1

With real memory, the average memory access time is AMAT, so:

CPI(actual) = CPI(computation) + Memory_Accesses_per_Instruction*AMAT =
              CPI(ideal) + Memory_Accesses_per_Instruction*(AMAT - HitTime_L1) =
              CPI(ideal) + 1.37*(AMAT - HitTime_L1)

(b) Your AMAT calculation is wrong. After a miss at L2, it follows a L3 access that can be a hit or a miss. Try to finish the exercise yourself.