unstable result from scipy.cluster.kmeans

Question

The following code gives different results at every runtime while clustering the data into 3 parts using the k means method:

from numpy import array
from scipy.cluster.vq import kmeans,vq

data = array([1,1,1,1,1,1,3,3,3,3,3,3,7,7,7,7,7,7])
centroids = kmeans(data,3,100) #with 100 iterations
print (centroids)

Three possible results obtained were:

(array([1, 3, 7]), 0.0)
(array([3, 7, 1]), 0.0)
(array([7, 3, 1]), 0.0)

Actually, the order of the calculated k means are different. But, does not it unstable to assign which k means point belongs to which cluster? Any idea??

Answer 1

That's because if you pass an integer as the k_or_guess parameter, k initial centroids are chosen randomly from the set of input observations (this is known as the Forgy method).

From the docs:

k_or_guess : int or ndarray

The number of centroids to generate. A code is assigned to each centroid, which is also the row index of the centroid in the code_book matrix generated.

The initial k centroids are chosen by randomly selecting observations from the observation matrix. Alternatively, passing a k by N array specifies the initial k centroids.

Try handing it a guess instead:

kmeans(data,np.array([1,3,7]),100)

# (array([1, 3, 7]), 0.0)
# (array([1, 3, 7]), 0.0)
# (array([1, 3, 7]), 0.0)

Answer 2

From the docs:

k_or_guess: int or ndarray

The number of centroids to generate. A code is assigned to each centroid, which is also the row index of the centroid in the code_book matrix generated.

The initial k centroids are chosen by randomly selecting observations

So resulting order of clusters is random. If you want more control with this, you can specify

Alternatively, passing a k by N array specifies the initial k centroids

I would not reccomend latter in general case, as different starting clusters [may] lead to different clustering, and predefined initial centroids can lead to suboptimal solution.

In your simple case resulting clustering is always the same (optimal) modulo clusters order:

>>> centroids, _ = kmeans(data,3,100)
>>> idx, _  = vq(data, centroids)
>>> centroids, idx
array([1, 7, 3]), array([0, 0, 0, 0, 0, 0, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1])
>>> centroids, _ = kmeans(data,3,100)
>>> idx, _  = vq(data, centroids)
>>> centroids, idx
array([3, 7, 1]), array([2, 2, 2, 2, 2, 2, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1])