The evolving digital world requires scientifically literate citizens who are able to critically evaluate Internet sources of varying credibility. Instruction on evidence evaluation in postsecondary education often focuses on peer-review as a singular indicator of credibility. With increased access to web-based scientific information, students must also learn to think critically in real-time about the dimensions of credibility. This study describes the integration of "sInvestigator," a computational evidence-based scientific reasoning tool, with a class of 32 students in an undergraduate honors course focused on socio-scientific issues. A cross-disciplinary team of researchers with expertise in science education, scientific literacy, and evidence evaluation developed and implemented an online questionnaire to measure students' development of digital scientific literacy. After using "sInvestigator" to evaluate sources of scientific evidence based on publisher reputation, author competence, and author objectivity, students were better able to assess the credibility of online information. Results of this study also confirm the potential to authentically assess students' use of author and publisher information to evaluate digital scientific sources. The need for further research on the operationalization and measurement of digital scientific literacy is discussed.