regression

Ridge-regression model: glmnet

馋奶兔 提交于 2019-12-06 15:49:40
问题 Fitting a linear-regression model using least squares on my training dataset works fine. library(Matrix) library(tm) library(glmnet) library(e1071) library(SparseM) library(ggplot2) trainingData <- read.csv("train.csv", stringsAsFactors=FALSE,sep=",", header = FALSE) testingData <- read.csv("test.csv",sep=",", stringsAsFactors=FALSE, header = FALSE) lm.fit = lm(as.factor(V42)~ ., data = trainingData) linearMPrediction = predict(lm.fit,newdata = testingData, se.fit = TRUE) mean(

Statsmodels.formula.api OLS does not show statistical values of intercept

我的未来我决定 提交于 2019-12-06 15:44:21
I am running the following source code: import statsmodels.formula.api as sm # Add one column of ones for the intercept term X = np.append(arr= np.ones((50, 1)).astype(int), values=X, axis=1) regressor_OLS = sm.OLS(endog=y, exog=X).fit() print(regressor_OLS.summary()) where X is an 50x5 (before adding the intercept term) numpy array which looks like this: [[0 1 165349.20 136897.80 471784.10] [0 0 162597.70 151377.59 443898.53]...] and y is a a 50x1 numpy array with float values for the dependent variable. The first two columns are for a dummy variable with three different values. The rest of

Percentiles from VGAM

痞子三分冷 提交于 2019-12-06 15:11:14
I am using following example from help pages of package VGAM library(VGAM) fit4 <- vgam(BMI ~ s(age, df = c(4, 2)), lms.bcn(zero = 1), data = bmi.nz, trace = TRUE) qtplot(fit4, percentiles = c(5,50,90,99), main = "Quantiles", las = 1, xlim = c(15, 90), ylab = "BMI", lwd = 2, lcol = 4) I am getting a proper graph with it: How can I avoid plotting points from the graph? Also I need to print out values for these percentiles at each of ages 20,30,40...80 (separately as a table). How can this be done? Can I use ggplot() format command rather than qtplot() command here? Thanks for your help. How

“Rolling” Regression in R

二次信任 提交于 2019-12-06 15:07:50
问题 Say I want to run regressions per group whereby I want to use the last 5 year data as input for that regression. Then, for each next year, I would like to "shift" the input for that regression by one year (i.e., 4 observations). From those regressions I want to extract both the R2 and the fitted values/residuals, which I then need in subsequent regressions that follow similar notions. I have some code working using loops, but it is not really elegant nor efficient for large datasets. I assume

Data Transformation in R for Panel Regression

我的梦境 提交于 2019-12-06 14:48:48
I really need your help regarding a problem which may seem easy to solve for you. Currently I work on a project which involves some panel-regressions. I have several large csv-files (up to 12 million entries per sheet) which are formatted as in the picture attached, whereas the columns (V1, V2) are individuals and the rows (1, 2, 3) are time identifiers. In order to use the plm() -function I need all these files to convert to the following data structure: ID Time X1 X2 1 1 x1 x2 1 2 x1 x2 1 ... ... ... 2 1 x1 x2 2 2 ... ... I really struggle with this transformation and I'm really frustrated

Out of memory when using `outer` in solving my big normal equation for least squares estimation

∥☆過路亽.° 提交于 2019-12-06 14:48:37
Consider the following example in R: x1 <- rnorm(100000) x2 <- rnorm(100000) g <- cbind(x1, x2, x1^2, x2^2) gg <- t(g) %*% g gginv <- solve(gg) bigmatrix <- outer(x1, x2, "<=") Gw <- t(g) %*% bigmatrix beta <- gginv %*% Gw w1 <- bigmatrix - g %*% beta If I try to run such a thing in my computer, it will throw a memory error (because the bigmatrix is too big). Do you know how can I achieve the same, without running into this problem? This is a least squares problem with 100,000 responses. Your bigmatrix is the response (matrix), beta is the coefficient (matrix), while w1 is the residual (matrix

Python or SQL Logistic Regression

若如初见. 提交于 2019-12-06 14:48:03
问题 Given time-series data, I want to find the best fitting logarithmic curve. What are good libraries for doing this in either Python or SQL? Edit: Specifically, what I'm looking for is a library that can fit data resembling a sigmoid function, with upper and lower horizontal asymptotes. 回答1: If your data were categorical, then you could use a logistic regression to fit the probabilities of belonging to a class (classification). However, I understand you are trying to fit the data to a sigmoid

AIC with weighted nonlinear regression (nls)

独自空忆成欢 提交于 2019-12-06 13:48:10
I encounter some discrepancies when comparing the deviance of a weighted and unweigthed model with the AIC values. A general example (from ‘nls’): DNase1 <- subset(DNase, Run == 1) fm1DNase1 <- nls(density ~ SSlogis(log(conc), Asym, xmid, scal), DNase1) This is the unweighted fit, in the code of ‘nls’ one can see that ‘nls’ generates a vector wts <- rep(1, n) . Now for a weighted fit: fm2DNase1 <- nls(density ~ SSlogis(log(conc), Asym, xmid, scal), DNase1, weights = rep(1:8, each = 2)) in which I assign increasing weights for each of the 8 concentrations with 2 replicates. Now with deviance I

Automation testing tool for Regression testing of desktop application [closed]

僤鯓⒐⒋嵵緔 提交于 2019-12-06 12:40:05
问题 Closed. This question is off-topic. It is not currently accepting answers. Want to improve this question? Update the question so it's on-topic for Stack Overflow. Closed 3 years ago . I am working on a desktop application which uses Infragistic grids. We need to automate the regression tests for same. QTP alone does not support this, we need to buy new plug in for same which my company is not very much interested in. Do we have any open source tool for automating regression testing of desktop

R Shiny: Rendering summary.ivreg output

一笑奈何 提交于 2019-12-06 11:06:03
问题 I'm trying to render an instrumental variable regression summary in R Shiny Here is the code: iv=ivreg(lwage~educ+exper|nearc4+exper) summary(iv) When I use renderTable I get the following error: no applicable method for 'xtable' applied to an object of class "summary.ivreg" Any suggestions how to go around this issue? This is my website, if you want to see what I'm doing exactly: https://ieconometrics.shinyapps.io/test/ 回答1: renderTable expect an object for which a xtable methods exist, you