The work at hand is part of a wider study the aim of which was to determine what kind of factors influence failures in pragmatic items of an online adaptive placement test taken by a group of 34 Spanish students in their first year at university. A preceding analysis (Carrió & Martín, 2016) showed the type of personal factors, such as lack of vocabulary, that caused the exam takers failures in the pragmatic part of the test according to their own perception. In this paper, we go deeper and analyze those specific items holding higher percentage of failures so that we can reach some conclusions about its content validity and also about the degree of relationship between these and the pragmatic categories dealt with in the most relevant research carried out in the field of pragmalinguistic testing and assessment.