Python: Rank order correlation for categorical data

问题

I am somewhat new to programming and statistics, so please help me improve this question if it is formally not correct.

I have a lot of parameters and a couple of result vectors I produced in a MonteCarlo simulation. Now I want to test the influence of each parameter for the result. I already got a script working with Kendall's Tau. Now I would like to compare with Spearman and Pearson rho. An example:

from scipy.stats import spearmanr, kendalltau, pearsonr
result = [106, 86, 100, 101, 99, 103, 97, 113, 112, 110]
parameter = ['A', 'B', 'A', 'B', 'A', 'B', 'A', 'B', 'A', 'B']
kendalltau(parameter, result)

>> (0.14907119849998596, 0.54850624613917143)

However if I try the same for spearmanr or pearsonr I get errors. Apparently this feature was not implemented in Scipy. Do you know of a simple way to obtain correlation coefficients for categorical data?

回答1:

Actually spearmanr works, however pearsonr will not as it needs to calculate the mean of the array, dtype is not correct for string. See below:

from scipy.stats import spearmanr, kendalltau, pearsonr

result = [106, 86, 100, 101, 99, 103, 97, 113, 112, 110]

parameter = ['A', 'B', 'A', 'B', 'A', 'B', 'A', 'B', 'A', 'B']

spearmanr(result, parameter)

(0.17407765595569782, 0.63053607555697644)

help(pearsonr)

Help on function pearsonr in module scipy.stats.stats:

pearsonr(x, y)
    Calculates a Pearson correlation coefficient and the p-value for testing
    non-correlation.

    The Pearson correlation coefficient measures the linear relationship
    between two datasets. Strictly speaking, Pearson's correlation requires
    that each dataset be normally distributed. Like other correlation
    coefficients, this one varies between -1 and +1 with 0 implying no
    correlation. Correlations of -1 or +1 imply an exact linear
    relationship. Positive correlations imply that as x increases, so does
    y. Negative correlations imply that as x increases, y decreases.

    The p-value roughly indicates the probability of an uncorrelated system
    producing datasets that have a Pearson correlation at least as extreme
    as the one computed from these datasets. The p-values are not entirely
    reliable but are probably reasonable for datasets larger than 500 or so.

    Parameters
    ----------
    x : 1D array
    y : 1D array the same length as x

    Returns
    -------
    (Pearson's correlation coefficient,
     2-tailed p-value)

    References
    ----------
    http://www.statsoft.com/textbook/glosp.html#Pearson%20Correlation

convert 'A' to 1, 'B' to 2, for example

params = [1 if el == 'A' else 2 for el in parameter]

print params

[1, 2, 1, 2, 1, 2, 1, 2, 1, 2]

pearsonr(params, result)

(-0.012995783552244984, 0.97157652425566488)

hope this helps.

来源：https://stackoverflow.com/questions/26489961/python-rank-order-correlation-for-categorical-data

标签

python

statistics

scipy

correlation