logistic-regression

I am using the logistic regression function from sklearn, and was wondering what each of the solver is actually doing behind the scenes to solve the optimization problem. Can someone briefly describe what "newton-cg", "sag", "lbfgs" and "liblinear" are doing? If not, any related links or reading materials are much appreciated too. Thanks a lot in advance. Well, I hope I'm not too late to the party! Let me first try to establish some intuition before digging in loads of information ( warning : this is not brief comparison) Introduction A hypothesis h(x) , takes an input and gives us the

I am trying to use a deep neural network architecture to classify against a binary label value - -1 and +1. Here is my code to do it in tensorflow . import tensorflow as tf import numpy as np from preprocess import create_feature_sets_and_labels train_x,train_y,test_x,test_y = create_feature_sets_and_labels() x = tf.placeholder('float', [None, 5]) y = tf.placeholder('float') n_nodes_hl1 = 500 n_nodes_hl2 = 500 n_nodes_hl3 = 500 n_classes = 1 batch_size = 100 def neural_network_model(data): hidden_1_layer = {'weights':tf.Variable(tf.random_normal([5, n_nodes_hl1])), 'biases':tf.Variable(tf

I have made some experiments with logistic regression in R, python statmodels and sklearn. While the results given by R and statmodels agree, there is some discrepency with what is returned by sklearn. I would like to understand why these results are different. I understand that it is probably not the same optimization algorithms that are used under the wood. Specifically, I use the standard Default dataset (used in the ISL book ). The following Python code reads the data into a dataframe Default . import pandas as pd # data is available here Default = pd.read_csv('https://d1pqsl2386xqi9

I try to run following code. Btw, I am new to both python and sklearn. import pandas as pd import numpy as np from sklearn.linear_model import LogisticRegression # data import and preparation trainData = pd.read_csv('train.csv') train = trainData.values testData = pd.read_csv('test.csv') test = testData.values X = np.c_[train[:, 0], train[:, 2], train[:, 6:7], train[:, 9]] X = np.nan_to_num(X) y = train[:, 1] Xtest = np.c_[test[:, 0:1], test[:, 5:6], test[:, 8]] Xtest = np.nan_to_num(Xtest) # model lr = LogisticRegression() lr.fit(X, y) where y is a np.ndarray of 0's and 1's I receive the

I have started using Vowpal Wabbit for logistic regression, however I am unable to reproduce the results it gives. Perhaps there is some undocumented "magic" it does, but has anyone been able to replicate / verify / check the calculations for logistic regression? For example, with the simple data below, we aim to model the way age predicts label . It is obvious there is a strong relationship as when age increases the probability of observing 1 increases. As a simple unit test, I used the 12 rows of data below: age label 20 0 25 0 30 0 35 0 40 0 50 0 60 1 65 0 70 1 75 1 77 1 80 1 Now,

I have a binary prediction model trained by logistic regression algorithm. I want know which features(predictors) are more important for the decision of positive or negative class. I know there is coef_ parameter comes from the scikit-learn package, but I don't know whether it is enough to for the importance. Another thing is how I can evaluate the coef_ values in terms of the importance for negative and positive classes. I also read about standardized regression coefficients and I don't know what it is. Lets say there are features like size of tumor, weight of tumor, and etc to make a