Plagiarism in academic writing is known to be an issue of concern for educators, administrators, and students alike. Using self-reporting studies and plagiarism detection software, previous research has established that plagiarism in university-level academic writing is relatively common amongst the work of L1 and particularly L2 writers and that paraphrasing and the various definitions of what constitutes plagiarism are notable challenges. However, previous research has been limited in the plagiarism detection systems used to evaluate student writing and has shed little light on their accuracy. This study seeks to contribute to scholarship on plagiarism by both measuring potential plagiarism in the writing of students at a university in Japan and assessing the accuracy of two plagiarism detection systems, Google Classroom and Grammarly. It was found that student writing was slightly less likely to feature content which may be deemed plagiarized, and the total amount of such content as a percentage of the total was notably low. The accuracy of the two systems differed greatly, with the Google Classroom version being more reliable in terms of generating fewer false positives and having more potential to be used as a self-editing or learning tool for students.