The potential bias introduced into COVID-19 vaccine effectiveness studies at primary care level due to the availability of SARS-CoV-2 tests in the general population

Abstract Background With SARS-CoV-2 self-tests, persons with acute respiratory infections (ARI) can know their COVID-19 status. This may alter their decision to consult a general practitioner (GP), potentially biasing COVID-19 vaccine effectiveness (VE) studies. We explore bias mechanisms, simulate magnitude, and verify control methods. Methods We used directed acyclic graphs (DAGs) to illustrate the bias mechanisms. Based on the European primary care VEBIS multicentre test-negative design (TND) study, we simulated populations with varying true VE (20%–60%), proportions of persons with ARI self-testing (10%–30%), effect of COVID-19 vaccination on self-testing (1.5–2.5), and effect of self-test result on GP consultation (0.5–2). We performed 5000 runs per scenario, estimating VE among those consulting a GP. We calculated bias as true VE minus mean simulated VE, unadjusted and adjusted for self-testing, using logistic regression. Results DAGs suggested collider stratification bias if vaccination had an effect on self-testing and if self-test results affected GP consultation. Bias was −12% to 18% at 20% true VE, with the most extreme associations and 30% self-testing. With 60% true VE and 10%–20% self-testing, bias was lower. Bias was higher (−18% to 45%) if both positive and negative self-test results affected GP consultation. Adjusting for self-testing removed the bias. Conclusions Self-testing may bias COVID-19 VE TND studies in primary care if self-testing is high, particularly with low VE. We recommend primary care TND VE studies collect self-testing information to eliminate potential bias. Observational studies are needed to understand the relationship between vaccination, self-testing, and GP consultation, in these studies’ source population.