In today's world, there is too much information pollution and people circulate it without questioning, and the claims on controversial issues often contain fallacies and conspiracy theories. Considering this point of view, it is necessary to create citizens who critically evaluate information. In order to achieve this goal, science educators need to address students' evaluation of fallacies on controversial issues. The aim of the present study, thus, is to explore eighth graders' evaluation of fallacies about vaccination. We used case study in the study in which 29 eighth grade students participated. We adapted a rubric that was developed by Lombardi et al. (Int J Sci Educ 38(8):1393-1414, 2016. https://bibliotheek.ehb.be:2102/10.1080/09500693.2016.1193912) to assess students' evaluation levels between claims and evidence and used it to analyze students' evaluations of each fallacy in groups and individually. The findings of this study indicate that students were mostly unable to evaluate claims and evidence critically. We advocate that attempts should be made so that students cope with misinformation and disinformation, ensuring that they establish a consistent relationship between claim and the evidence, and the social and cultural factors that affect their evaluation of false claims should be identified. The implications of this study suggest to make deliberate attempts in order to enable middle school students to evaluate claims and evidence critically in various science topics, particularly in health issues, due to the COVID-19 pandemic. Implications of the present research include suggestion of a method including discussions of fallacies about controversial issues and utilizing additional data sources such as interviews to reveal student ideas in-depth and analyze students' decision-making skills.