This paper presents a novel traffic sign recognition system which can aid in the development of Intelligent Speed Adaptation. This system is based on extracting the speed limit sign from the traffic scene by Circular Hough Transform (CHT) with the aid of colour and non-colour information of the traffic sign. The digits of the speed limit sign are then extracted and classified using SVM classifier which is trained for this purpose. In general, the system detects the prohibitory traffic sign in the first place, specifies whether the detected sign is a speed limit sign, and then determines the allowed speed in case the detected sign is a speed limit sign. The SVM classifier was trained with 270 images which were collected in different light conditions. To check the robustness of this system, it was tested against 210 images which contain 213 speed limit traffic sign and 288 Non-Speed limit signs. It was found that the accuracy of recognition was 98% which indicates clearly the high robustness targeted by this system.