Comparison of Assessment of Citrus Canker Foliar Symptoms by Experienced and Inexperienced Raters

Citrus canker (Xanthomonas citri subsp. citri) is destructive in many citrus production regions in tropical and subtropical parts of the world. Assessment of canker symptoms is required for diverse reasons, including monitoring epidemics, evaluating the efficacy of control strategies, and disease response in breeding material. The objectives were to compare the ability of experienced and inexperienced raters at assessing citrus canker, to identify factors that affect the quality of the assessment, to determine common sources of error, and to discern how error is related to actual disease magnitude. Two-hundred digital leaf images (0 to 37% area infected) were assessed once by 28 raters, five of whom were experienced plant pathologists (PPs), and 23 who had no experience in disease severity assessment (NPPs). True disease (lesion number [LN], % necrotic area [%N], and % chlorotic+necrotic area [%CN]) was measured using image analysis on a leaf-by-leaf basis, and each parameter was estimated by the 28 raters. LN was neither severely over- nor underestimated, while %N was greatly overestimated, with a lesser tendency to overestimate %CN over the true severity range of these two symptom types. A linear relationship existed between estimate of the disease and true disease for all measures of severity. Data were heteroscedastic and error was not constant with increasing true disease. Agreement between rater estimates and true disease was measured with Lin's concordance correlation coefficient (ρ(c)). LN showed greatest agreement (ρ(c) = 0.88 to 0.99), followed by %CN (ρ(c) = 0.80 to 0.95) and %N (ρ(c) = 0.19 to 0.84). Greater lesion number resulted in overestimation of area infected for both %N and %CN. Overestimation was particularly noticeable at low disease severities. There was a linear relationship between log variance and log true disease for LN (r2 = 0.71), %N (r2 = 0.85), and %CN (r2 = 0.88), and raters tended to estimate disease above 10% to the nearest 5 or 10%. GLM analysis showed differences between PP and NPP groups in assessing disease. For LN, precision of assessment for both groups was similar (r2 > 0.92 and 0.94, respectively), but for estimates of %N and %CN, the PPs were more precise (%N and %CN, r2 = 0.61 and 0.73, respectively) compared to NPPs (%N and %CN, r2 = 0.45 and 0.58, respectively). Absolute error for mean LN was low. The absolute error of %N and %CN showed overestimation to approximately 8% area infected. Above 8%, absolute error increased, but comprised both over- and underestimation. For %N and %CN, relative error was almost exclusively positive and dramatic at severity <8% (up to approximately 600%), but at severity >10% it was relatively small. Error in rater estimates of canker severity is ubiquitous. Understanding these sources of error will aid in the development of both appropriate training and relevant rating aids.