An Architecture Combining Convolutional Neural Network (CNN) and Support Vector Machine (SVM) for Image Classification
Abien Fred M. Agarap
Abstract
Convolutional neural networks (CNNs) are similar to “ordinary” neural networks in the sense that they are made up of hidden layers consisting of neurons with “learnable” parameters. These neurons receive inputs, performs a dot product, and then follows it with a non-linearity. The whole network expresses the mapping between raw image pixels and their class scores. Conventionally, the Softmax function is the classifier used at the last layer of this network. However, there have been studies (Alalshekmubarak and Smith, 2013; Agarap, 2017; Tang, 2013) conducted to challenge this norm. The cited studies introduce the usage of linear support vector machine (SVM) in an artificial neural network architecture. This project is yet another take on the subject, and is inspired by (Tang, 2013). Empirical data has shown that the CNN-SVM model was able to achieve a test accuracy of $\approx$ 99.04% using the MNIST dataset(LeCun et al., 2010). On the other hand, the CNN-Softmax was able to achieve a test accuracy of $\approx$ 99.23% using the same dataset. Both models were also tested on the recently-published Fashion-MNIST dataset(Xiao et al., 2017), which is suppose to be a more difficult image
中文速览
卷积神经网络（CNN）在图像分类任务中通常以Softmax函数作为最后一层的分类器，但已有研究尝试用支持向量机（SVM）替换它以获得更好效果。受Tang（2013）启发，作者搭建了一个两层卷积的简单CNN，分别将最后一层替换为L2-SVM和传统Softmax，在MNIST手写数字数据集和更具挑战性的Fashion-MNIST服装图像数据集上进行对比实验。结果显示，在MNIST上CNN-Softmax以约99.23%的测试准确率略胜CNN-SVM的约99.04%，在Fashion-MNIST上同样是CNN-Softmax（约91.86%）稍优于CNN-SVM（约90.72%），并未重现Tang原文中SVM更优的结论，作者认为这主要源于本研究未做任何数据预处理且模型较为简单。这项工作表明，CNN末端直接替换分类器的效果对模型复杂度和数据处理方式高度敏感，也为后续引入更完善的预处理和更深网络来复现SVM优势提供了基准参考。
原文 arXiv:1712.03541;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1712.03541v2