correlation

When taking fft(signal, nfft) of a signal, how does nfft change the outcome and why? Can I have a fixed value for nfft, say 2^18 , or do I need to go 2^nextpow2(2*length(signal)-1) ? I am computing the power spectral density(PSD) of two signals by taking the FFT of the autocorrelation, and I want to compare the the results. Since the signals are of different lengths, I am worried if I don't fix nfft, it would make the comparison really hard! There is no inherent reason to use a power-of-two (it just might make the processing more efficient in some circumstances). However, to make the FFTs of

I have a long numerical time series data of approximately 200,000 rows (lets call it Z ). In a loop, I subset x (about 30) consecutive rows from Z at a time and treat them as the query point q . I want to locate within Z the y (~300) most correlated time series segments of length x (most correlated with q ). What is an efficient way to accomplish this? The code below finds the 300 segments you are looking for and runs in 8 seconds on my none too powerful Windows laptop, so it should be fast enough for your purposes. First, it constructs a 30-by-199971 matrix ( Zmat ), whose columns contain all

I have a correlation matrix X of five elements(C1,C2,C3,C4,C5) C1 C2 C3 C4 C5 C1 * 1 0 1 0 C2 1 * 0 0 1 C3 0 0 * 1 1 C4 1 0 1 * 0 C5 0 1 1 0 * I want to use MatLab to move as many as non-zero cells close to diagonal, while keep the diagonal cells are "*". For example, you may notice that the columns and rows is shifting in the following matrix, while the diagonal cells are "*". C1 C4 C2 C5 C3 C1 * 1 1 0 0 C4 1 * 0 0 1 C2 1 0 * 1 0 C5 0 0 1 * 1 C3 0 1 0 1 * Because I want to do clustering, so I want as many as non-zero cells get close to diagonal after shifting. It's an NP-hard problem. Anyone

Is there a way of elegantly calculating the correlations between values if those values are stored by group in a single column of a data.table (other than converting the data.table to a matrix)? library(data.table) set.seed(1) # reproducibility dt <- data.table(id=1:4, group=rep(letters[1:2], c(4,4)), value=rnorm(8)) setkey(dt, group) # id group value # 1: 1 a -0.6264538 # 2: 2 a 0.1836433 # 3: 3 a -0.8356286 # 4: 4 a 1.5952808 # 5: 1 b 0.3295078 # 6: 2 b -0.8204684 # 7: 3 b 0.4874291 # 8: 4 b 0.7383247 Something that works, but requires the group names as input: cor(dt["a"]$value, dt["b"]

I have been desperately looking for a way to compute a polychoric correlation matrix, with significance in R. If that is very hard then polychoric correlation between two variables with significance would be sufficient. What I have tried so far: library(polychor) poly <- polychor(var1,var2) poly <- polychor(DatM) #where DatM is a DF converted to matrix library(polycor) hetcor(Dat2) #I am however uncertain hetcor is something I would want if I am looking for polychoric correlation. library(psych) polychoric(Dat$for2a,smooth=TRUE,global=TRUE,polycor=FALSE, ML = FALSE, std.err=TRUE) None of these

I am trying to compute a correlation matrix of several values. These values include some 'nan' values. I'm using numpy.corrcoef. For element(i,j) of the output correlation matrix I'd like to have the correlation calculated using all values that exist for both variable i and variable j. This is what I have now: In[20]: df_counties = pd.read_sql("SELECT Median_Age, Rpercent_2008, overall_LS, population_density FROM countyVotingSM2", db_eng) In[21]: np.corrcoef(df_counties, rowvar = False) Out[21]: array([[ 1. , nan, nan, -0.10998411], [ nan, nan, nan, nan], [ nan, nan, nan, nan], [-0.10998411,

I have a matrix data with m rows and n columns. I used to compute the correlation coefficients between all pairs of rows using np.corrcoef : import numpy as np data = np.array([[0, 1, -1], [0, -1, 1]]) np.corrcoef(data) Now I would also like to have a look at the p-values of these coefficients. np.corrcoef doesn't provide these; scipy.stats.pearsonr does. However, scipy.stats.pearsonr does not accept a matrix on input. Is there a quick way how to compute both the coefficient and the p-value for all pairs of rows (arriving e.g. at two m by m matrices, one with correlation coefficients, the