Artificial intelligence (AI) literacy is at the top of the agenda for education today in developing learners' AI knowledge, skills, attitudes and values in the 21st century. However, there are few validated research instruments for educators to examine how secondary students develop and perceive their learning outcomes. After reviewing the literature on AI literacy questionnaires, we categorized the identified competencies in four dimensions: (1) affective learning (intrinsic motivation and self-efficacy/confidence), (2) behavioural learning (behavioural commitment and collaboration), (3) cognitive learning (know and understand; apply, evaluate and create) and (4) ethical learning. Then, a 32-item self-reported questionnaire on AI literacy (AILQ) was developed and validated to measure students' literacy development in the four dimensions. The design and validation of AILQ were examined through theoretical review, expert judgement, interview, pilot study and first- and second-order confirmatory factor analysis. This article reports the findings of a pilot study using a preliminary version of the AILQ among 363 secondary school students in Hong Kong to analyse the psychometric properties of the instrument. Results indicated a four-factor structure of the AILQ and revealed good reliability and validity. The AILQ is recommended as a reliable measurement scale for assessing how secondary students foster their AI literacy and inform better instructional design based on the proposed affective, behavioural, cognitive and ethical (ABCE) learning framework.