correlation

Efficient columnwise correlation coefficient calculation

由 人走茶凉 提交于 2019-12-07 04:14:08
问题 Original question I am correlating row P of size n to every column of matrix O of size n×m. I crafted the following code: import numpy as np def ColumnWiseCorrcoef(O, P): n = P.size DO = O - (np.sum(O, 0) / np.double(n)) DP = P - (np.sum(P) / np.double(n)) return np.dot(DP, DO) / np.sqrt(np.sum(DO ** 2, 0) * np.sum(DP ** 2)) It is more efficient than the naive approach: def ColumnWiseCorrcoefNaive(O, P): return np.corrcoef(P,O.T)[0,1:O[0].size+1] Here are the timings I get with numpy-1.7.1

Create a Correlation Matrix From a Correlation Vector in R

由 时光怂恿深爱的人放手 提交于 2019-12-07 00:02:22
问题 I want to create a correlation matrix given the correlation vector, which is the upper (or lower) triangular matrix of the correlation matrix. The goal is to transform this vector to this correlation matrix with 1s on the diagonal. Do you know if there is a method creating a matrix given the triangular above the diagonal and to set the diagonal to 1? 回答1: I don't know if there is an automatic way to do this, but expanding on my comment: myvec <- c(-.55, -.48, .66, .47, -.38, -.46) mempty <-

How to get correlation coefficients in matlab

由 China☆狼群 提交于 2019-12-06 15:40:15
In every paper i read about encryption they like to show the correlation coefficients of their encrypted image by showing 3 values: Horizontal correlation coefficient . vertical correlation coefficient. diagonal correlation coefficient . and they show these 3 values for encrypted image and also for plain image(lena). My question is how to do this in matlab ? and if there is no matlab function for it , what are the equation they are using to get those 3 values ? Table 2 Correlation coefficients of two adjacent pixels in two images example: Plain-image Ciphered image Horizontal 0.92401 0.01589

generating correlated numbers in numpy / pandas

由 自古美人都是妖i 提交于 2019-12-06 13:22:13
I’m trying to generate simulated student grades in 4 subjects, where a student record is a single row of data. The code shown here will generate normally distributed random numbers with a mean of 60 and a standard deviation of 15. df = pd.DataFrame(15 * np.random.randn(5, 4) + 60, columns=['Math', 'Science', 'History', 'Art']) What I can’t figure out is how to make it so that a student’s Science mark is highly correlated to their Math mark, and that their History and Art marks are less so, but still somewhat correlated to the Math mark. I’m neither a statistician or an expert programmer, so a

Making sense of CCA (Matlab implementation) 2

由 随声附和 提交于 2019-12-06 12:49:33
问题 I am using CCA for my work and want to understand something. This is my MATLAB code. I have only taken 100 samples to better understand the concepts of CCA. clc;clear all;close all; load carbig; data = [Displacement Horsepower Weight Acceleration MPG]; data(isnan(data))=0; X = data(1:100,1:3); Y = data(1:100,4:5); [wx,wy,~,U,V] = CCA(X,Y); clear Acceleration Cylinders Displacement Horsepower MPG Mfg Model Model_Year Origin Weight when org subplot(1,2,1),plot(U(:,1),V(:,1),'.'); subplot(1,2,2)

Remove strongly correlated columns from DataFrame

由 耗尽温柔 提交于 2019-12-06 12:20:37
I have a DataFrame like this dict_ = {'Date':['2018-01-01','2018-01-02','2018-01-03','2018-01-04','2018-01-05'],'Col1':[1,2,3,4,5],'Col2':[1.1,1.2,1.3,1.4,1.5],'Col3':[0.33,0.98,1.54,0.01,0.99]} df = pd.DataFrame(dict_, columns=dict_.keys()) I then calculate the pearson correlation between the columns and filter out columns that are correlated above my threshold of 0.95 def trimm_correlated(df_in, threshold): df_corr = df_in.corr(method='pearson', min_periods=1) df_not_correlated = ~(df_corr.mask(np.eye(len(df_corr), dtype=bool)).abs() > threshold).any() un_corr_idx = df_not_correlated.loc[df

translation/rotation through phase correlation in python

由 送分小仙女□ 提交于 2019-12-06 11:50:30
I have two pictures, one that was the original and another one that I have modified so that it's translated up and left a bit and then rotated 90 degrees (so the shape of the picture is transposed as well). Now I'd like to determine how many pixels (or any distance unit) the modified picture is translated from the original, as well as the degrees of rotation relative to the original. Phase correlation is supposed to solve this problem by first converting the coordinates to logpolar coordinates, then doing a number of things so that in the end you get a correlation matrix. From that matrix I'm

Error in bootstrapping: Error in if (const(t, min(1e-08, mean(t, na.rm = TRUE)/1e 06)))

由 别来无恙 提交于 2019-12-06 10:45:32
问题 I get the following error message when calculating bootstrap confidence intervals with the "boot" package: Error in if (const(t, min(1e-08, mean(t, na.rm = TRUE)/1e+06))) { : Missing Value, where TRUE/FALSE is necessary This error occurs only if I want to calculate bootstrap confidence intervals for spearman correlation coefficients across countries of a specific variable and is thus not easily reproduced. For many variables the code runs nicely, but for this specific one I obtain the above

Correlation coefficient on gnuplot

由 安稳与你 提交于 2019-12-06 07:31:12
问题 I want to plot data using fit function : function f(x) = a+b*x**2 . After ploting i have this result: correlation matrix of the fit parameters: m n m 1.000 n -0.935 1.000 My question is : how can i found a correlation coefficient on gnuplot ? 回答1: If you're looking for a way to calculate the correlation coefficient as defined on this page, you are out of luck using gnuplot as explained in this Google Groups thread. There are lots of other tools for calculating correlation coefficients, e.g.

Correlation among 2 images

由 蹲街弑〆低调 提交于 2019-12-06 06:58:13
问题 I am trying to find the following correlation among two images f1 and f2 where the size of the image is PXP. I have written a for loop program for the same but I think an inbuilt function would be faster for the same. Which function in matlab can help me compute this ? Also if the size of both the images are M X N can someone tell me how this formula will change or if the function will be able to handle it. EDIT: Is there any faster function than xcorr2 that can help me seeing that it takes
订阅 correlation