In this study, an instrument for measuring proper name (PN) familiarity was developed for a psycholinguistic experiment investigating the effect of PNs on Japanese university students' English reading fluency. Familiarity has previously been operationalized in disparate ways, producing contradictory results. Furthermore, authors of previous studies did not conduct validation analyses on their familiarity instruments. To address this issue, a four-point Likert-type scale instrument was constructed to assess Japanese university students' familiarity with a set of 100, two-syllable PNs. The responses of 216 participants from 2 Japanese universities were subjected to Rasch analysis with the rating scale model to determine whether the resulting data fit the expectations of the model. The results suggested that a dichotomous response instrument was more appropriate than the scale-based instruments utilized in previous studies.