Kernlab kraziness: inconsistent results for identical problems

问题

I've found some puzzling behavior in the kernlab package: estimating SVMs which are mathematically identical produces different results in software.

This code snippet just takes the iris data and makes it a binary classification problem for the sake of simplicity. As you can see, I'm using linear kernels in both SVMs.

library(kernlab)
library(e1071)

data(iris)
x <- as.matrix(iris[, 1:4])
y <- as.factor(ifelse(iris[, 5] == 'versicolor', 1, -1))
C <- 5.278031643091578

svm1 <- ksvm(x = x, y = y, scaled = FALSE, kernel = 'vanilladot', C = C)

K <- kernelMatrix(vanilladot(), x)
svm2 <- ksvm(x = K, y = y, C = C, kernel = 'matrix')

svm3 <- svm(x = x, y = y, scale = FALSE, kernel = 'linear', cost = C)

However, the summary information of svm1 and svm2 are dramatically different: kernlab reports completely different support vector counts, training error rates, and objective function values between the two models.

> svm1
Support Vector Machine object of class "ksvm" 

SV type: C-svc  (classification) 
 parameter : cost C = 5.27803164309158 

Linear (vanilla) kernel function. 

Number of Support Vectors : 89 

Objective Function Value : -445.7911 
Training error : 0.26 
> svm2
Support Vector Machine object of class "ksvm" 

SV type: C-svc  (classification) 
 parameter : cost C = 5.27803164309158 

[1] " Kernel matrix used as input."

Number of Support Vectors : 59 

Objective Function Value : -292.692 
Training error : 0.333333

For the sake of comparison, I also computed the same model using e1071, which provides an R interface for the libsvm package.

svm3

Call:
svm.default(x = x, y = y, scale = FALSE, kernel = "linear", cost = C)


Parameters:
   SVM-Type:  C-classification 
 SVM-Kernel:  linear 
       cost:  5.278032 
      gamma:  0.25 

Number of Support Vectors:  89

It reports 89 support vectors, the same as svm1.

My question is whether there are any known bugs in the kernlab package which can account for this unusual behavior.

(Kernlab for R is an SVM solver that allows one to use one of several pre-packaged kernel functions, or a user-supplied kernel matrix. The output is an estimate of a support vector machine for the user-supplied hyperparameters.)

回答1:

Reviewing some of the code, it appears that this is the offending line:

https://github.com/cran/kernlab/blob/efd7d91521b439a993efb49cf8e71b57fae5fc5a/src/svm.cpp#L4205

That is, in the case of a user-supplied kernel matrix, the ksvm is just looking at two dimensions, rather than whatever the dimensionality of the input is. This seems strange, and is probably a hold-over from some testing or whatever. Tests of the linear kernel with data of just two dimensions produces the same result: replace 1:4 with 1:2 in the above and the output and predictions all agree.

来源：https://stackoverflow.com/questions/33813972/kernlab-kraziness-inconsistent-results-for-identical-problems

标签

machine-learning

svm

kernlab