Support Vector Machine (SVM)

This is a totally different method of supervised learning. Its not some addon to logistic regression.

It is mainly formulated for Binary Classification

Given training data

$$\{({\mathbf{x}}_1,y_1),({\mathbf{x}}_2,y_2),\dots ,({\mathbf{x}}_n,y_n)\}$$

where

$${\mathbf{x}}_i\in {\mathbb{R}}^n{y}_i\in \{-1,+1\}$$

We are trying to solve the following optimization problem

$$\underset{\mathbf{w},b}{\min }\frac{1}{2}||\mathbf{w}|{|}^2\text{subject to}{y}_i({\mathbf{w}}^T{\mathbf{x}}_i+b)\ge 1$$

To work with multiple classes, do a one vs rest approach

machineLearning