To assess the integration of information and communication technologies (ICTs) in teaching and learning, a new construct for ICT competence was developed by the Programme for International Student Assessment (PISA) in 2022. The framework for ICT assessment involves four major dimensions; access to ICT, use of ICT, students' ICT competencies, and subject-related ICT use. In this paper, we analysed this ICT assessment in terms of validity construction of test scores with data from PISA 2018 and investigated ICT competence in relation to the variables of mathematics, science, and reading literacy. The analyses were based on public data from PISA 2018 for Turkey (N = 6890). After removal of univariate and multivariate outliers from the Turkish sample, factor analysis, correlation analysis, and structural equation modelling were performed on 5438 data items. In the study, we confirmed the four-dimensional structure of ICT competence for the Turkish sample. In this study, the construct validity of the PISA 2018 application ICT test was verified using the Turkish sample. The ESCS variable was included as a control variable in the analyses due to its significant impact on the relevant variables. The analysis showed that the mathematical literacy scores of the PISA 2018 Turkish students were positively low affected by the availability of ICT, negatively low affected by the entertainment of ICT, and positively low affected by the interest and perceived competence in ICT, use of ICT for educational purposes at school and outside school, and the use of ICT related to the subject. Considering the ICT variables that affected the PISA 2018 students' science literacy scores; the entertainment of ICT and the interest and perceived competence in ICT significantly low affected the students' use of ICT on the subject. On the other hand, the availability of ICT negative significantly low affect their science literacy scores. While the entertainment of ICT negatively low affected the science literacy scores, interest and perceived competence in ICT and use of ICT for educational purposes at school and outside school positively low affected the students' use of ICT on the subject. Finally, when we look at the ICT variables that low affected the students' reading literacy scores; and use of ICT for educational purposes at school and outside school and the interest and perceived competence in ICT significantly positive low affected the use of ICT on the subject. On the other hand, the availability of ICT negative significantly low affect reading literacy scores. While the entertainment of ICT negatively low affected reading literacy scores; interest and perceived competence in ICT positively low affected the use of ICT on the subject.