How to switch CPU models in gem5 after restoring a checkpoint and then observe the difference?

Question

I want to boot the Linux kernel in full system (FS) mode with a lightweight CPU to save time, make a checkpoint after boot finishes, and then restore the checkpoint with a m

Answer 1

First, for your question, I don't see how cycle count being an indication of the restoration result. The cycle being restored should be the same regardless of what CPU you want to switch. Switching does not change the past cycles. When creating a checkpoint, you basically freeze the simulation at that state. And switching CPU simply changes all the parameter of the CPU while keeping the ticks unchanged. It is like hot swapping a CPU.

To correctly verify the restoration, you should keep a copy of config.json before restoration and compare it with the new one after restoration. For X86 case, I could find string AtomicSimpleCPU there only before restore.

Furthermore, only --cpu-type will determine the CPU being switched. But it does not make --restore-with-cpu useless. In fact, --restore-with-cpu should only be used when you boot up the system with a CPU other than AtomicSimpleCPU. Most people want to boot up the system with AtomicSimpleCPU and make a checkpoint since it is faster. But if you mistakenly boot up using DerivO3CPU, to restore this particular checkpoint, you have to configure --restore-with-cpu to DerivO3CPU. Otherwise, it will fail.

Answer 2

From reading through some of the code I believe that --restore-with-cpu is specifically for the case when your checkpoint was created using a CPU model that isn't the AtomicCPU. The scripts assume that AtomicCPU was used to create the checkpoint. I think when restoring it's important to have the same cpu model as the system was checkpointed with, if you give another model with --cpu-type then it switches to that model after the restore operation as completed.

http://gem5.org/Checkpoints#Sampling has some (small) detail on switching cpu models

Answer 3

--cpu-type= affected the restore, but --restore-with-cpu= did not

I am not sure why that is, but I have empirically verified that if I do:

-r 1 --cpu-type=HPI

then as expected the cache size options start to affect cycle counts: larger caches leads to less cycles.

Also keep in mind that caches don't affect AtomicSimpleCPU much, and there is not much point in having them.

TODO so what is the point of --restore-with-cpu= vs --cpu-type if it didn't seem to do anything on my tests?

Except confuse me, since if --cpu-type != --restore-with-cpu, then the cycle count appears under system.switch_cpus.numCycles instead of system.cpu.numCycles.

I believe this is what is going on (yet untested):

• switch_cpu contains stats for the CPU you switched to
• when you set --restore-with-cpu= != --cpu-type, it thinks you have already switched CPUs from the start
• --restore-with-cpu has no effect on the initial CPU. It only matters for options that switch the CPU during the run itself, e.g. --fast-forward and --repeat_switch. This is where you will see both cpu and switch_cpu data get filled up.

TODO: also, if I use or remove --restore-with-cpu=, there is a small 1% cycle difference. But why is there a difference at all? AtomicSimpleCPU cycle count is completely different, so it must not be that it is falling back to it.

--cpu-type= vs --restore-with-cpu= showed up in fs.py --fast-forward: https://www.mail-archive.com/gem5-users@gem5.org/msg17418.html

Confirm what is happening with logging

One good sanity that the CPU want want is being used, is to enable some logging as shown at: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/bab029f60656913b5dea629a220ae593cc16147d#gem5-restore-checkpoint-with-a-different-cpu e.g.:

--debug-flags ExecAll,FmtFlag,O3CPU,SimpleCPU

and shen see if you start to get O3 messages rather than SimpleCPU ones.