2

This is more of a conceptual question.

I have learned about Neural Nets, and I have some clue as to how Support Vector Machines work. I read somewhere however that given the appropriate kernel (is that right?), the SVM is identical to the Neural Net. Could someone who understands this please enlighten me as to how that's possible?

1 Answer 1

1

Context

two-layer neuronal networks and svms do linear classification = splitting good vectors into bad vectors by putting a line or flat layer between them which has one dimension less than the vector space. (line in 2d space; dot onto line).

When you have three layer neuronal networks they can classify the XOR (in a,b out a xor b). So you can classify xor by chaining two linear classifiers. From my point of view you can do this because you can train chained networks.

Hunch: Because you usually do not train chained SVMs, YOU must provide the first classifiers that do the same job as the first neuronal networks.

Example: In case of xor where you have the input vectors (a, b) for a three layer network, you would not pass the (a, b) kernel to the svm because the space is not linearly separable. you would pass (a, b, not a, not b) which means thet you expand the kernel. This way the SVM is as 'identical' to the neuronal network.

xor classification

2
  • For clarification, you're saying that in SVMs, you must provide the expected label by way of passing (a,b,not a, not b), but I don't get what exactly you mean by "not a, not b" and why that's important. For XOR, wouldn't you pass in input:(x1,x2) and expected output?
    – Josh F
    Dec 28, 2013 at 17:15
  • you would pass (x1, x2) which I named (a, b) but as the image shows you can not separate the output blue and black with a single line. But when you have the 4-dimensional space (x1, x2, 1 - x1, 1 - x2) you can use a 3d hyper-plane to separate the values (quite well). That is my argument. I am not an expert though but maybe you know how to go on investigate.
    – User
    Dec 28, 2013 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.