This study investigated to what extent the Peabody Picture Vocabulary Test (PPVT-4) is a reliable tool for measuring vocabulary knowledge of English as a second language (L2), and to what extent L1 characteristics affect test outcomes. The PPVT-4 was administered to Dutch pupils in six different age groups (4-15 years old) who were or were not following an English educational programme at school. Our first finding was that the PPVT-4 was not a reliable measure for pupils who were correct on maximally 24 items, but it was reliable for pupils who performed better. Second, both primary-school and secondary-school pupils performed better on items for which the phonological similarity between the English word and its Dutch translation was higher. Third, young unexperienced L2 learners' scores were predicted by Dutch lexical frequency, while older more experienced pupils' scores were predicted by English frequency. These findings indicate that the PPVT may be inappropriate for use with L2 learners with limited L2 proficiency. Furthermore, comparisons of PPVT scores across learners with different L1s are confounded by effects of L1 frequency and L1-L2 similarity. The PPVT-4 is however a suitable measure to compare more proficient L2 learners who have the same L1.