Question-2

Which of the following classifiers produce a linear decision boundary in the feature space?

• Perceptron
• Logistic regression
• Kernel SVM with Gaussian kernel
• A neural network with two hidden layers that has ReLU activations in the hidden layers

Answer

• Perceptron
• Logistic regression
• Kernel SVM with Gaussian kernel
• A neural network with two hidden layers that has ReLU activations in the hidden layers

Solution

The boundary for a perceptron and logistic regression is linear and is of the form \(\mathbf{w}^{T} \mathbf{x} + b = 0\). Kernel SVM with a Gaussian kernel will have a non-linear boundary. Same is the case with a neural network with non-linear activation functions in the hidden layers.