The purpose of this study is to investigate various alternatives for addressing class imbalance issues and to analyze the behavior of machine learning models when these alternatives are applied to the dataset collected from four automatic milking systems (AMS) farms in Sweden and the Netherlands. Artificial neural network (ANN) models were trained to detect the presence of clots in milk using data from these AMS farms. Initially, without adjusting for class imbalance, the baseline models predicted clots with a sensitivity (Se) of 22% and specificity (Sp) of 99%. Despite the high Sp, the low Se could be attributed to the severe class imbalance in the data. Class imbalance is common in classification tasks where the response variable has an unequal class distribution. Resampling strategies such as ROSE, SMOTE, ROS, and SMOTE-TOMEK were employed to balance the dataset, and new models were trained on these synthetic datasets. The hybrid resampling method SMOTE-TOMEK achieved a sensitivity of 51% and specificity of 90%. The ROSE-applied model obtained a sensitivity of 38% and specificity of 96%, while the SMOTE-applied model achieved a sensitivity of 33% and specificity of 96% at default decision thresholds. Although ROSE and SMOTE produced equivalent results, ROSE demonstrated slightly better performance. Model comparisons were conducted using ROC/PR-AUC and MCC metrics. The highest ROC-AUC (0.809) and PR-AUC (0.365) were seen in the ROSE-applied models, whilst highest MCC of 0.32 is observed on SMOTE+TOMEK-applied models. Furthermore, threshold moving on baseline models improved the sensitivity score from 22% to 70%, however, the specificity scores reduced to 72% from 99%. More optimal thresholds can be determined using the ROC/PR curves. Overall, it can be said that both resampling methods and threshold moving successfully improved model performance, albeit sub-optimally. However, threshold moving may be more advantageous due to its simpler execution compared to resampling methods. Applying imbalanced learning techniques enhanced the performance of the ANN models, though there was a trade-off between sensitivity and specificity scores.