Continuous real-time cow identification by reading ear tags from live-stream video

In precision dairy farming there is a need for continuous and real-time availability of data on cows and systems. Data collection using sensors is becoming more common and it can be difficult to connect sensor measurements to the identification of the individual cow that was measured. Cows can be identified by RFID tags, but ear tags with identification numbers are more widely used. Here we describe a system that makes the ear tag identification of the cow continuously available from a live-stream video so that this information can be added to other data streams that are collected in real-time. An ear tag reading model was implemented by retraining and existing model, and tested for accuracy of reading the digits on cows ear tag images obtained from two dairy farms. The ear tag reading model was then combined with a video set up in a milking robot on a dairy farm, where the identification by the milking robot was considered ground-truth. The system is reporting ear tag numbers obtained from live-stream video in real-time. Retraining a model using a small set of 750 images of ear tags increased the digit level accuracy to 87% in the test set. This compares to 80% accuracy obtained with the starting model trained on images of house numbers only. The ear tag numbers reported by real-time analysis of live-stream video identified the right cow 93% of the time. Precision and sensitivity were lower, with 65% and 41%, respectively, meaning that 41% of all cow visits to the milking robot were detected with the correct cow’s ear tag number. Further improvement in sensitivity needs to be investigated but when ear tag numbers are reported they are correct 93% of the time which is a promising starting point for future system improvements.