ERIC Number: ED523107
Record Type: Non-Journal
Publication Date: 2009
Pages: 419
Abstractor: ERIC
ISBN: ISBN-978-9-2640-4808-9
ISSN: N/A
EISSN: N/A
PISA 2006 Technical Report
OECD Publishing (NJ1)
The Organisation for Economic Cooperation and Development's (OECD's) Programme for International Student Assessment (PISA) surveys, which take place every three years, have been designed to collect information about 15-year-old students in participating countries. PISA examines how well students are prepared to meet the challenges of the future, rather than how well they master particular curricula. PISA is methodologically highly complex, requiring intensive collaboration among many stakeholders. The successful implementation of PISA depends on the use, and sometimes further development, of state of the art methodologies and technologies. The "PISA 2006 Technical Report" describes those methodologies, along with other features that have enabled PISA to provide high quality data to support policy formation and review. The descriptions are provided at a level that will enable review and, potentially, replication of the implemented procedures and technical solutions to problems. This report contains a description of the theoretical underpinning of the complex techniques used to create the PISA 2006 database, which includes information on nearly 400,000 students from 57 countries. The database includes not only information on student performance in the three main areas of assessment--reading, mathematics and science--but also their responses to the student questionnaire that they completed as part of the assessment. Data from the school principals of participating schools are also included. There are five sections in this report. Section One, Instrument design (Chapters 1-4), describes the design and development of both the questionnaires and achievement tests. Section Two, Operations (Chapters 5-7), gives details of the operational procedures for the sampling and population definitions, test administration procedures, quality monitoring and assurance procedures for test administration and national centre operations, and instrument translation. Section Three, Data processing (Chapters 8-10), covers the methods used in data cleaning and preparation, including the methods for weighting and variance estimation, scaling methods, methods for examining inter-rater variation and the data cleaning steps. Section Four, Quality indicators and outcomes (Chapters 11-14), covers the results of the scaling and weighting, reports response rates and related sampling outcomes and gives the outcomes of the interrater reliability studies. The last chapter in this section summarises the outcomes of the PISA 2006 data adjudication; that is, the overall analysis of data quality for each country. Section Five, Scale construction and data products (Chapters 15-18), describes the construction of the PISA 2006 levels of proficiency and the construction and validation of questionnaire-related indices. The final chapter briefly describes the contents of the PISA 2006 database. Appended are: (1) PISA 2006 main study item pool characteristics; (2) Contrast coding used in conditioning; (3) Design effect tables; (4) Changes to core questionnaire items from 2003 to 2006; (5) Mapping of ISCED to years; (6) National household possession items; (7) Exploratory and confirmatory factor analyses for the embedded items; and (8) PISA consortium, staff and consultants. Individual chapters contain notes. (Contains 1 box, 51 figures, and 200 tables.)
Descriptors: Policy Formation, Scaling, Academic Achievement, Interrater Reliability, Achievement Tests, Questionnaires, Research Reports, Foreign Countries, Surveys, Adolescents, Research Methodology, Data Collection, Educational Policy, Reading Achievement, Mathematics Achievement, Science Achievement, Test Construction, Sampling, Quality Assurance, Data Analysis, Test Validity, Test Items, Factor Analysis, Cross Cultural Studies, Correlation
OECD Publishing. 2, rue Andre Pascal, F-75775 Paris Cedex 16, France. Tel: +33-145-24-8200; Fax: +33-145-24-8500; Web site: http://www.oecd.org
Publication Type: Numerical/Quantitative Data; Reports - Descriptive; Tests/Questionnaires
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Organisation for Economic Cooperation and Development
Identifiers - Assessments and Surveys: Program for International Student Assessment
Grant or Contract Numbers: N/A