YOLO-Based Framework with Different Annotated Datasets for BISINDO Sign Language Recognition

Humans require communication to interact with each other using language. There are exceptional cases when people cannot rely on sound or written language due to physical limitations. The purpose of this research is to assist individuals by utilizing Artificial Intelligence to detect BISINDO hand sign language in real-time using a camera. Using the newly developed YOLOv12, the writers compared it with other models such as Random Forest and YOLOv11. The writers created a custom hand sign dataset of BISINDO, which contained an A to Z alphabet with 200 images for each letter, totaling 5,200 images. The dataset was augmented with rotation, blurring, and horizontal flipping, resulting in an expanded dataset of 12,954 images. Then, the dataset is split into three parts, with a ratio of 80:10:10 percent for training, testing, and validation, respectively. The results show that Random Forest, YOLOv11, and YOLOv12 are effective. The best training result model is YOLOv12 with MediaPipe annotation, which achieves a precision of 99.97% and a recall of 99.8%. However, when the model is tested in real-time, YOLOv11 with RoboFlow annotation performs better than YOLOv12 with MediaPipe annotation, achieving 83.2% in terms of real-time accuracy.