In the emergency department (ED) Registered Nurses (RNs) often perform triage, i.e. the sorting and prioritizing of patients. The allocation of acuity ratings is commonly based on a triage scale. To date, three reliable 5-level triage scales exist, of which the Canadian Triage and Acuity Scale (CTAS) is one. In Sweden, few studies on ED triage have been conducted and the organization of triage has been found to vary considerably with no common triage scale. The aim of this study was to investigate the accuracy and concordance of emergency nurses acuity ratings of patient scenarios in the ED setting. Totally, 423 RNs from 48 (62%) Swedish EDs each triaged 18 patient scenarios using the CTAS. Of the 7550 triage ratings, 57.6% were triaged in concordance with the expected outcome and no scenario was triaged into the same triage level by all RNs. Inter-rater agreement for all RNs was kappa = 0.46 (unweighted) and kappa = 0.71 (weighted). The fact that the kappa-values are only moderate to good and the low concordance between the RNs call for further studies, especially from a patient safety perspective.