Why does numpy mixed basic / advanced indexing depend on slice adjacency?

Question

I know similar questions have been asked before (e.g.), but AFAIK nobody has answered my specific question...

My question is about the numpy mixed advanced / basic indexing described here:

... Two cases of index combination need to be distinguished:

• The advanced indexes are separated by a slice, ellipsis or newaxis. For example x[arr1,:,arr2].
• The advanced indexes are all next to each other. For example x[...,arr1,arr2,:] but not x[arr1,:,1] since 1 is an advanced index in this regard.

In the first case, the dimensions resulting from the advanced indexing operation come first in the result array, and the subspace dimensions after that. In the second case, the dimensions from the advanced indexing operations are inserted into the result array at the same spot as they were in the initial array (the latter logic is what makes simple advanced indexing behave just like slicing).

Why is this distinction necessary?

I was expecting the behaviour described for case 2 to be used in all cases. Why does it matter whether indexes are next to each other?

I understand you may want the behaviour of case 1 in some situations; for example, "vectorization" of index results along new dimensions. But this behaviour can and should be defined by the user. That is, if case 2 behaviour was the default, case 1 behaviour would be possible using only: x[arr1,:,arr2].reshape((len(arr1),x.shape[1]))

I know you can achieve the behaviour described in case 2 using np.ix_(), but this inconsistency in default indexing behaviour is unexpected and unjustified, in my opinion. Can someone justify it?

Thanks,

Answer 1

The behavior for case 2 isn't well-defined for case 1. There's a subtlety you're probably missing in the following sentence:

In the second case, the dimensions from the advanced indexing operations are inserted into the result array at the same spot as they were in the initial array

You're probably imagining a one-to-one correspondence between input and output dimensions, perhaps because you're imagining Matlab-style indexing. NumPy doesn't work like that. If you have four arrays with the following shapes:

a.shape == (2, 3, 4, 5, 6)
b.shape == (20, 30)
c.shape == (20, 30)
d.shape == (20, 30)

then a[b, :, c, :, d] has four dimensions, with lengths 3, 5, 20, and 30. There is no unambiguous place to put the 20 and the 30. NumPy defaults to sticking them in front.

On the other hand, with a[:, b, c, d, :], the 20 and 30 can go where the 3, 4, and 5 were, because the 3, 4, and 5 were next to each other. The whole block of new dimensions goes where the whole block of original dimensions was, which only works if the original dimensions were in a single block in the original shape.