Automatic teat detection for rotary milking system based on deep learning algorithms

• Published in: Computers and electronics in agriculture 2021-10, Vol.189, p.106391, Article 106391
• Main Authors: Lu, Zhiheng, Zhao, Manfei, Luo, Jun, Wang, Guanghui, Wang, Decheng
• Format: Article
• Language: English
• Subjects: Algorithms; Datasets; Error detection; Image acquisition; Machine learning; Object recognition; R2Faster R-CNN algorithm; Robotic milking; Rotary milking system; Rotated region; Teat detection; Teats
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2021.106391

•An udder image dataset was established under the rotary milking system environment.•A teat detection model was developed to detect teats’ rotated bounding boxes.•This algorithm can detect overlapped or partially occluded teats.•The automatic milking device can apply teat cup attachment accurately. In dairy farming, the milking process is the most labor-intensive activity. In order to automate the milking process, we acquire cows’ udder images in the working conditions of the rotary milking system to establish the udder image dataset. According to the characteristics of the dataset, rotated bounding boxes are used to label the teats. An arbitrary-oriented object detection framework R2Faster R-CNN is used to develop the teat detection model. The AP of the model is 80.41%, the mean orientation error is 2.29°, and the detection time is 0.19 s per image. In addition, R2Faster R-CNN algorithm can detect overlapped or partially occluded teats, and has high stability under actual rotary milking system environment. Finally, an automatic milking device is built to perform teat cup attachment experiments in the laboratory environment. Among the 20 groups experiments, all teats are detected correctly, the average refresh time is 0.39 s. The success rate of teat cup attachment is 100%, namely the detection error of the sensing system is within 5 mm. The results prove the feasibility and accuracy of the teat sensing system.