The Internet has become a pivotal source of information among university students. However, studies routinely show that many students lack digital information literacy skills (i.e. skills needed to find and evaluate information online). In this paper, we report results from an experimental study testing the effect of a workshop for third-year students of a German university. The workshop was designed to teach relevant information literacy skills in a computer lab. Afterwards, students were given academic search tasks and their search behavior was recorded with a tracking device. We find that, compared with the control group, workshop participants significantly increased their use of academic databases and cited more articles from scholarly journals. On the other hand, we find no effect on the relevance of the content students found online. Teaching digital information literacy is essential and feasible, but it is no panacea for increasing the academic quality of students' work.