In many traffic sign recognition system, one of the main tasks is to classify the shapes of traffic sign. In this paper, we have developed a shape-based classification model by using support vector machines. We focused on recognizing seven categories of traffic sign shapes and five categories of speed limit signs. Two kinds of features, binary image and Zernike moments, were used for representing the data to the SVM for training and test. We compared and analyzed the performances of the SVM recognition model using different feature representations and different kernels and SVM types. Our experimental data sets consisted of 350 traffic sign shapes and 250 speed limit signs. Experimental results have shown excellent results, which have achieved 100% accuracy on sign shapes classification and 99% accuracy on speed limit signs classification. The performance of SVM model highly depends on the choice of model parameters. Two search algorithms, grid search and simulated annealing search have been implemented to improve the performances of our classification model. The SVM model were also shown to be more effective than Fuzzy ARTMAP model.