ARM: Start/Wakeup/Bringup the other CPU cores/APs and pass execution start address?

Question

I\'ve been banging my head with this for the last 3-4 days and I can\'t find a DECENT explanatory documentation (from ARM or unofficial) to help me. I\'ve got an ODR

Answer 1

QEMU uses PSCI

The ARM Power State Coordination Interface (PSCI) is documented at: https://developer.arm.com/docs/den0022/latest/arm-power-state-coordination-interface-platform-design-document and controls things such as powering on and off of cores.

TL;DR this is the aarch64 snippet to wake up CPU 1 on QEMU v3.0.0 ARMv8 aarch64:

/* PSCI function identifier: CPU_ON. */
ldr w0, =0xc4000003
/* Argument 1: target_cpu */
mov x1, 1
/* Argument 2: entry_point_address */
ldr x2, =cpu1_entry_address
/* Argument 3: context_id */
mov x3, 0
/* Unused hvc args: the Linux kernel zeroes them,
 * but I don't think it is required.
 */
hvc 0

and for ARMv7:

ldr r0, =0x84000003
mov r1, #1
ldr r2, =cpu1_entry_address
mov r3, #0
hvc 0

A full runnable example with a spinlock is available on the ARM section of this answer: What does multicore assembly language look like?

The hvc instruction then gets handled by an EL2 handler, see also: the ARM section of: What are Ring 0 and Ring 3 in the context of operating systems?

Linux kernel

In Linux v4.19, that address is informed to the Linux kernel through the device tree, QEMU for example auto-generates an entry of form:

    psci {
            method = "hvc";
            compatible = "arm,psci-0.2", "arm,psci";
            cpu_on = <0xc4000003>;
            migrate = <0xc4000005>;
            cpu_suspend = <0xc4000001>;
            cpu_off = <0x84000002>;
    };

The hvc instruction is called from: https://github.com/torvalds/linux/blob/v4.19/drivers/firmware/psci.c#L178

static int psci_cpu_on(unsigned long cpuid, unsigned long entry_point)

which ends up going to: https://github.com/torvalds/linux/blob/v4.19/arch/arm64/kernel/smccc-call.S#L51