Reliability analysis of PISA 2018 reading literacy student questionnaire based on item response theory (IRT): Turkey sample
Reliability analysis of PISA 2018 reading literacy
Abstract
PISA (Program for International Student Assessment) tests have enabled the education systems of countries to see not only the success of their students in gaining the ability to solve the daily problems they may encounter in their lives but also the place in the world rankings as a result of an objective evaluation compared with the achievement results of other countries. Therefore, it is very important for the reliability and prestige of the program that the PISA exams and related student survey results are as unbiassed and error-free as possible. In this paper, validity and reliability study of the PISA 2018 Reading Literacy Student Survey (RLSS) Turkey sample was conducted including the analysis of survey items within the framework of Item Response Theory (IRT). The item fit indexes, item parameters according to the GPCM (Generalized Partial Credit Model), standard error values and item characteristics’ reliability of the survey were determined via IRT, respectively. The data set included the answers of 6111 15-year-old Turkish students participated in PISA 2018. In the data analysis, through using the local independence assumption, Q3 statistics; IRT calibrations were tested with the help of the “Mirt v.1.30” program within the scope of “R v.4.0.5”. In the study, each set of questions in the PISA student survey was examined independently from each other and each question set was considered as a separate attitude scale. The results showed that, although some of the items in PISA 2018 (RLSS) gave low level information, all the question sets in the test provided an acceptable model fit according to the GPCM. Upon examining the item characteristic curves, it was understood that the survey items showed valid and reliable results for testing different ability levels.
Downloads
Published
Issue
Section
License
Copyright (c) 2021 International Journal of Curriculum and Instruction
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an article implies that the work described has not been published previously (except in the form of an abstract or as part of a published lecture or academic thesis), that it is not under consideration for publication elsewhere, that its publication is approved by all authors and tacitly or explicitly by the responsible authorities where the work was carried out, and that, if accepted, will not be published elsewhere in the same form, in English or in any other language, without the written consent of the Publisher. The Editors reserve the right to edit or otherwise alter all contributions, but authors will receive proofs for approval before publication.
Copyrights for articles published in International Journal of Curriculum and Instruction are retained by the authors, with first publication rights granted to the journal. The journal/publisher is not responsible for subsequent uses of the work. It is the author's responsibility to bring an infringement action if so desired by the author.
