Umbrella reviews (reviews of systematic reviews) are increasingly used to synthesize findings from systematic reviews. One important challenge when pooling data from several systematic reviews is publication overlap, that is, the same primary publications being included in multiple reviews. Pieper et al. have proposed using the corrected covered area (CCA) index to quantify the degree of overlap between systematic reviews to be pooled in an umbrella review. Recently, this methodology has been integrated in Excel- or R-based tools for easier use. In this short letter, we highlight an important consideration for using the CCA methodology for pairwise overlap assessment, especially when reviews include varying numbers of primary publications, and we urge researchers to fine-tune this method and exercise caution when review exclusion decisions are based on its output. © 2023 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.