Wooden railway sleeper inspections in Sweden are currently performed manually by a human operator; such inspections are to large extent based on visual analysis. In this paper a machine vision based approach has been considered to emulate the visual abilities of the human operator to enable automation of the process. Digital images from either ends (left and right) of the sleepers have been acquired. A pattern recognition approach has been adopted to classify the condition of the sleeper into classes (good or bad) and thereby achieve automation. Appropriate image analysis techniques were applied and relevant features such as the number of cracks on a sleeper, average length and width of the crack and the condition of the metal plate were determined. Feature fusion has been proposed in order to integrate the features obtained from each end for the classification task which follows. The effect of using classifiers like multi-layer perceptron and support vector machines has been tested and compared. Results obtained from the experiments show that multi-layer perceptron and support vector machines have achieved encouraging results, with a classification accuracy of 90%; thereby exhibiting a competitive performance when compared to a human operator.