This paper presents a two-step pseudo likelihood estimation technique for generalized linear mixed models with correlated random effects. The proposed estimation technique does not require reparametarisation of the model. Multivariate Taylor's approximation has been used to approximate the intractable integrals in the likelihood function of the GLMM. Based on the analytical expression for the estimator of the covariance matrix of the random effects, a condition has been presented as to when such a covariance matrix can be estimated through the estimates of the random effects. An application of the model with a binary response variable has been presented using a real data set on credit defaults from two Swedish banks. Due to the use of two-step estimation technique, proposed algorithm outperforms the conventional pseudo likelihood algorithms in terms of computational time.