Vision-Based Human Action Recognition Using Skeleton Graph Neural Networks

HAR is an essential computer vision task, which provides the machine with an automatic representation of human actions and human action classification on the basis of visual representations. Older methods of HAR are based on RGB or RGB-D videos, and, in spite of their success in controlled settings, they are prone to occlusions, variations in lighting, and a high level of computational complexity. HAR based on skeletons has been proposed as an alternative, a representation of human poses in terms of joints or keypoints and tracking both spatial and temporal behavior of movement. This abstraction enhances resiliency to visual noise, lowers computational costs, and supports real time usage. Graph Neural Networks (GNNs), and in particular Graph Convolutional Networks (GCNs), have proven to have a phenomenal success in skeleton-based HAR, by considering the human body as a spatial-temporal graph with joints representing nodes and bones or temporal relationships as edges. The paper provides a systematized review of the vision-based human action recognition based on skeleton GNNs through skeleton representation, graph construction and development, early and advanced GNN architecture, and how they are enhanced by adaptive graphs, attention, and hierarchical modeling. Some of the most critical issues, including skeleton noise, time mismatch, generalization across views, over-smoothing in deep GCNs, and computational complexity, are addressed, as well as solutions to these problems. Also, we emphasize surveillance, healthcare, human-computer interaction, and robotics. Lastly, the future research directions are suggested, which are multi-moding, lightweight models of edge devices, self-monitored learning, and cross-dataset generalization. The current review offers a solid background to researchers who want to come up with strong, effective, and understandable skeleton-based systems of HAR.