RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning

Ultrasound (US) examination has been commonly utilized in clinical practice for assessing the rheumatoid arthritis (RA) activity, which is hampered by low intra-observer and inter-observer agreement as well as considerable time and expense to train experienced radiologists. Here, we present Rheumatoid ArthriTIs kNowledge Guided (RATING) deep learning model that scores RA activity and generates interpretable features to assist radiologists' decision-making. RATING model was developed using paired grey-scale US and color Doppler US images, and tested both in a prospective setting and on power Doppler US images collected from an external medical center. RATING model generalized well across settings, predicting the EOSS combined score with an accuracy of 86.1% (95% confidence interval (CI)=82.5%-90.1%) in the prospective setting and 85.0% (95% CI=80.5%-89.1%) on the US images collected from the external medical center. Prospective experiments demonstrated that RATING model improved the combined score accuracy of radiologists from 41.4% to 64.0% on average. Automated AI models for the assessment of RA may facilitate US RA examination and provide support for clinical decision-making.