Performance of raters to assess locomotion in dairy cattle

Abstract Locomotion scoring systems are procedures used to evaluate the quality of cows’ locomotion. When scoring locomotion, raters focus their attention on gait and posture traits that are described in the protocol. Using these traits, raters assign a locomotion score to cows according to a pre-determined scale. Locomotion scoring systems are mostly used to classify cows as lame or non-lame. A preselected threshold within the scale determines whether a cow is classified as lame or non-lame. Since lameness is considered an important problem in modern dairy farming evaluation of locomotion scoring systems is utmost important. The objective of this thesis was to evaluate the performance of raters to assess locomotion in dairy cattle in terms of reliability (defined as the ability of a measuring device to differentiate among subjects) and agreement (defined as the degree to which scores or ratings are identical). This thesis also explores possibilities for the practical application of locomotion scoring systems. In a literature review comprising 244 peer-reviewed articles, twenty-five locomotion scoring systems were found. Most locomotion scoring systems varied in the scale used and traits observed. Some of the most used locomotion scoring systems were poorly evaluated and, when evaluated, raters showed an important variation in reliability and agreement estimates. The variation in reliability and agreement estimates was confirmed in different experiments aiming to estimate the performance of raters for scoring locomotion and traits under different practical conditions. For instance, experienced raters obtained better intrarater reliability and agreement when locomotion scoring was performed from video than by live observation. In another experiment, ten experienced raters scored 58 video records for locomotion and for five different gait and posture traits in two sessions. A similar number of cows was allocated in each level of the five-level scale for locomotion scoring. Raters showed a wide variation in intra- and interrater reliability and agreement estimates for scoring locomotion and traits, even under the same practical conditions. When agreement was calculated for specific levels when scoring locomotion and traits, the lowest agreement tended to be in level 3 of a five-level scale. When a multilevel scale was transformed into a two-level scale, agreement increased, however, this increment was likely due to chance. The variation in reliability and agreement is explained by different factors such as the lack of a standard procedure for assessing locomotion or the characteristics of the population sample that is assessed. The factor affecting reliability and agreement most, however, is the rater him/herself. Although the probability for obtaining acceptable reliability and agreement levels increases with training and experience, it is not possible to assure that raters score cows consistently in every scoring session. Given the large variation in reliability and agreement, it can be concluded that raters have a moderate performance to assess consistently locomotion in dairy cows. The variable performance of raters when assessing locomotion limits the practical utility of locomotion scoring systems as part of animal welfare assessment protocols or as golden standard for automatic locomotion scoring systems.