What does it mean to learn a word? How can we tell when a sequence of letters or sounds becomes a word in the mind of the learner? While many second language (L2) vocabulary teaching and learning studies continue to use traditional vocabulary tests to measure learning (such as multiple choice, translation, gap-fill), these measures tend to come short when researchers want to address theoretical questions about the nature of L2 word knowledge. In the present paper, I argue for conceptualising word learning as lexicalisation, which necessitates the use of alternative approaches to measuring learning. I then propose approximate and conceptual replications of two theoretically motivated L2 word learning studies, Elgort (2011) and Qiao and Forster (2017), that used the Prime Lexicality Effect as a measure of lexicalisation of deliberately learned L2 words.