Multi-Class SVM( one versus all)

Question

I know that LIBSVM only allows one-vs-one classification when it comes to multi-class SVM. However, I would like to tweak it a bit to perform one-against-all classification.

Answer 1

From the code I can see you are trying to first turn the labels into "some class" vs "not this class", and then invoke LibSVM to do training and testing. Some questions and suggestions:

1. Why are you using the original TrainingLabel for training? In my opinion, should it be model = svmtrain(newClass, TrainVec, '-c 1 -g 0.00154');?
2. With modified training mechanism, you also need to tweak the prediction part, such as using sum-pooling to determine the final label. Using -b switch in LibSVM to enable probability output will also improve the accuracy.

Answer 2

%# Fisher Iris dataset
load fisheriris
[~,~,labels] = unique(species);   %# labels: 1/2/3
data = zscore(meas);              %# scale features
numInst = size(data,1);
numLabels = max(labels);

%# split training/testing
idx = randperm(numInst);
numTrain = 100; numTest = numInst - numTrain;
trainData = data(idx(1:numTrain),:);  testData = data(idx(numTrain+1:end),:);
trainLabel = labels(idx(1:numTrain)); testLabel = labels(idx(numTrain+1:end));
%# train one-against-all models
model = cell(numLabels,1);
for k=1:numLabels
    model{k} = svmtrain(double(trainLabel==k), trainData, '-c 1 -g 0.2 -b 1');
end

%# get probability estimates of test instances using each model
prob = zeros(numTest,numLabels);
for k=1:numLabels
    [~,~,p] = svmpredict(double(testLabel==k), testData, model{k}, '-b 1');
    prob(:,k) = p(:,model{k}.Label==1);    %# probability of class==k
end

%# predict the class with the highest probability
[~,pred] = max(prob,[],2);
acc = sum(pred == testLabel) ./ numel(testLabel)    %# accuracy
C = confusionmat(testLabel, pred)                   %# confusion matrix

Answer 3

Instead of probability estimates, you can also use the decision values as follows

[~,~,d] = svmpredict(double(testLabel==k), testData, model{k});
prob(:,k) = d * (2 * model{i}.Label(1) - 1);

to achieve the same purpose.