Electricity price forecasting plays a crucial role in liberalized electricity markets. Generally speaking, short term electricity price forecasting is essential for electricity providers to adjust the schedule of production in order to balance consumers’ demands and electricity generation. Short term forecasting results are also utilized by market players to decide the timing of purchasing or selling to gain maximized profit. Among existing forecasting approaches, neural networks are regarded as the state of art method. However, deep neural networks are not studied comprehensively in this field, thus the motivation of this study is to fill this research gap. In this paper, a novel hybrid approach is proposed for short term electricity price forecasting. To be more specific, categorical boosting (Catboost) algorithm is used for feature selection and a bidirectional long short term memory neural network (BDLSTM) serves as the main forecasting engine. To evaluate the effectiveness of the proposed approach, two datasets from the Nord Pool market are employed in the experiment. Moreover, the performance of multi-layer perception (MLP) neural network, support vector regression (SVR) and ensemble tree models are evaluated and compared with the proposed model. Results show that the proposed approach outperforms the rest models in terms of mean absolute percentage error (MAPE).