In Verilog you can't use a variable (i.e. bitsEnd) as the end of range. You can use +:/-: operator to solve your issue:
assign leadingBits = magnitude[bitsEnd+3 -: 4];
In the first case you only calculate single index (it's not a range). That's why the compiler is not complaining about it.