The aim of this study is to construct a STEM competencies assessment framework and provide validity evidence by empirically testing its structure. Common interdisciplinary assessment frameworks for STEM seem to be scarce in the literature. Many studies use students' mathematics or science scores obtained from large-scale assessments or exams to assess STEM achievement. In the current framework, the multidimensional and integrated structure of the STEM competencies was addressed by taking into account science, technology, engineering, and mathematics domains. Science, technology, engineering, and mathematics-related problems that required mathematical calculations were used as a medium to develop an assessment tool for this framework. A test was developed based on this assessment framework and items were calibrated using the item response theory. We tested the structure of the framework through confirmatory factor analysis by collecting data from 8th grade students. The empirical findings supported the structure of the STEM assessment framework.