Abstract

Convolutional neural networks (CNNs) are similar to “ordinary” neural networks in the sense that they are made up of hidden layers consisting of neurons with “learnable” parameters. These neurons receive inputs, performs a dot product, and then follows it with a non-linearity. The whole network expresses the mapping between raw image pixels and their class scores. Conventionally, the Softmax function is the classifier used at the last layer of this network. However, there have been studies (Alalshekmubarak and Smith 2013; Agarap 2017; Tang 2013) conducted to challenge this norm. The cited studies introduce the usage of linear support vector machine (SVM) in an artificial neural network architecture. This project is yet another take on the subject, and is inspired by (Tang 2013). Empirical data has shown that the CNN-SVM model was able to achieve a test accuracy of $\approx$ 99.04% using the MNIST dataset(LeCun et al. 2010). On the other hand, the CNN-Softmax was able to achieve a test accuracy of $\approx$ 99.23% using the same dataset. Both models were also tested on the recently-published Fashion-MNIST dataset(Xiao et al. 2017), which is suppose to be a more difficult image class