Channel-Agnostic vs. Channel-Adaptive: Foundation Model Strategies for Spatial Proteomics

Spatial proteomics via multiplex immunofluorescence (mIF) requires vision foundation models capable of handling variable channel composition and leveraging tissue structure information. Using DINOv2 framework, we trained and benchmarked channel-agnostic (standard ViT-s/14) and channel-adaptive (KRONOS-style ViT-s/14 with marker embeddings) architectures, on a large mIF dataset of 14,946 slides (48,993 single-marker images), 45 markers, and diverse tissue and disease types. Including the autofluorescence (AF) channel enabled channel-agnostic models to achieve consistently high linear probing F1-scores (BCL-2: $0.76, \text{HBsAg}: 0.69, \text{HBcAg}: 0.71$), compared to channel-adaptive (BCL-2: $0.61, \text{HBsAg}: 0.52, \text{HBcAg}$: 0.69) and reference models trained on H&E (UNI) or natural images (DINOv2). Channel-adaptive model performance fluctuated between tests (HBsAg (SD): 0.21), while channel-agnostic models remained robust (HBsAg (SD): $0.01)$. AF-containing models also produced more accurate tissue compartmentalization maps. Expanding training data further improved channel-agnostic model performance. These results establish the importance of AF and channel-agnostic ViT architectures for generalizable mIF representation learning.