As the molecular marker density grows, there is a strong need in both genome-wide association studies and genomic selection to fit models with a large number of parameters. Here we present a computationally efficient generalized ridge regression (RR) algorithmfor situations where the number of parameters largely exceeds the number of observations. The computationally demanding parts of the method depend mainly on the number ofobservations and not the number of parameters. The algorithm was implemented in the R package bigRR based on the previously developed package hglm. Using such an approach, a heteroscedastic effects model (HEM) was also developed, implemented and tested. Theefficiency for different data sizes were evaluated via simulation. The method was tested for a bacteria-hypersensitive trait in a publicly available Arabidopsis dataset including 84 inbred lines and 216 130 SNPs. The computation of all the SNP effects required less than10 seconds using a single 2.7 GHz core. The advantage in run-time makes permutationtest feasible for such a whole-genome model, so that a genome-wide significance threshold can be obtained. HEM was found to be more robust than ordinary RR (a.k.a. SNPBLUP) in terms of QTL mapping, because SNP specific shrinkage was applied instead of acommon shrinkage. The proposed algorithm was also assessed for genomic evaluation and was shown to give better predictions than ordinary RR.