correlation

I have a DataFrame df with a non-numerical column CatColumn . A B CatColumn 0 381.1396 7.343921 Medium 1 481.3268 6.786945 Medium 2 263.3766 7.628746 High 3 177.2400 5.225647 Medium-High I want to include CatColumn in the correlation analysis with other columns in the Dataframe. I tried DataFrame.corr but it does not include columns with nominal values in the correlation analysis. I am going to strongly disagree with the other comments. They miss the main point of correlation: How much does variable 1 increase or decrease as variable 2 increases or decreases. So in the very first place, order

I have two very large matrices (60x25000) and I'd like to compute the correlation between the columns only between the two matrices. For example: corrVal(1) = corr(mat1(:,1), mat2(:,1); corrVal(2) = corr(mat1(:,2), mat2(:,2); ... corrVal(i) = corr(mat1(:,i), mat2(:,i); For smaller matrices I can simply use: colCorr = diag( corr( mat1, mat2 ) ); but this doesn't work for very large matrices as I run out of memory. I've considered slicing up the matrices to compute the correlations and then combining the results but it seems like a waste to compute correlation between column combinations that I

I have two 1D arrays and I want to see their inter-relationships. What procedure should I use in numpy? I am using numpy.corrcoef(arrayA, arrayB) and numpy.correlate(arrayA, arrayB) and both are giving some results that I am not able to comprehend or understand. Can somebody please shed light on how to understand and interpret those numerical results (preferably using an example)? Thanks. ebarr numpy.correlate simply returns the cross-correlation of two vectors. if you need to understand cross-correlation, then start with http://en.wikipedia.org/wiki/Cross-correlation . A good example might be

I have two sets of temperature date, which have readings at regular (but different) time intervals. I'm trying to get the correlation between these two sets of data. I've been playing with Pandas to try to do this. I've created two timeseries, and am using TimeSeriesA.corr(TimeSeriesB) . However, if the times in the 2 timeSeries do not match up exactly (they're generally off by seconds), I get Null as an answer. I could get a decent answer if I could: a) Interpolate/fill missing times in each TimeSeries (I know this is possible in Pandas, I just don't know how to do it) b) strip the seconds

Assume we have two numeric vectors x and y . The Pearson correlation coefficient between x and y is given by cor(x, y) How can I automatically consider only a subset of x and y in the calculation (say 90%) as to maximize the correlation coefficient? If you really want to do this (remove the largest (absolute) residuals), then we can employ the linear model to estimate the least squares solution and associated residuals and then select the middle n% of the data. Here is an example: Firstly, generate some dummy data: require(MASS) ## for mvrnorm() set.seed(1) dat <- mvrnorm(1000, mu = c(4,5),

I have various time series, that I want to correlate - or rather, cross-correlate - with each other, to find out at which time lag the correlation factor is the greatest. I found various questions and answers/links discussing how to do it with numpy, but those would mean that I have to turn my dataframes into numpy arrays. And since my time series often cover different periods, I am afraid that I will run into chaos. Edit The issue I am having with all the numpy/scipy methods, is that they seem to lack awareness of the timeseries nature of my data. When I correlate a time series that starts in