Hong Kong was one of the first places hit with the COVID-19 outbreak in January 2020 and has since responded with several periods of full or partial school suspension, during which teaching and learning often occurred remotely using digital means. During this time, collecting research data and testing students have become more difficult than before. In this article, we report on a large-scale digital literacy (DL) performance assessment in Hong Kong that had to be conducted in a mix of modes--onsite, online supported, and online self-directed--during the pandemic. We discuss the administrative and methodological challenges of assessing DL, and explore preliminary data on the associations between student performance and the assessment mode adopted. We further discuss potential implications, including for high-stakes testing, and make recommendations for assessing DL remotely.