The State of Qatar has adopted an ambitious educational reform program that is designed to give Qatari students the necessary skills and knowledge that are required for pursuing advanced studies in Qatar and abroad and provide the state with qualified human resources for the local job market.
While most educational reforms naturally focus on curricular and administrative improvements, it is imperative that they also include a highly innovative evaluation component to ensure that decision makers have access to high-quality and objective information.
In the Qatar educational reform effort, such information is collected and disseminated by the Evaluation Institute through the annual Qatar Comprehensive Educational Assessment (QCEA) in the subject matters of Arabic, English, mathematics and science. The QCEA is based on the Qatar Curriculum Standards, which set a very high goal and compare with the best and most demanding educational standards worldwide.
The QCEA, however, is only one part in a comprehensive assessment system. Another role is played by classroom based tests developed by teachers. The third component consists of international assessments that allow for the comparison of the performance of Qatar students with those in other countries. The evaluation Institute oversees the implementation of all three major international assessments in Qatar.
-
Program For International Student Assessment (PISA)
-
Trends in International mathematics and science Study (TIMSS)
-
Progress in International Reading Literacy Study (PIRLS)
What is PIRLS?
The PIRLS 2011 is the third cycle of IEA’s Progress Reading Literacy Study (PIRLS) which comes after its predecessors PIRLS 2001 and PIRLS 2006. PIRLS 2011 is an international comparative study which focuses on the achievement and reading experiences of children in 53 countries in grades equivalent to fourth grade in the United States of America and collects data to provide information on trends in reading literacy achievement and policy and practices related to literacy of fourth-grade students in those countries, in their native language, while providing baseline data for new countries.
With PIRLS 2011 those countries that participated in PIRLS 2001 and 2006 can obtain data about changes in children’s reading achievement. They can also collect valuable information about changes in reading instruction and how those changes relate to students’ performance in reading. Countries new to the study can collect baseline data about their students’ levels of reading achievement, as well as home, school, and classroom influences on that achievement.
The study includes a written test of reading comprehension and a series of questionnaires focusing on the factors associated with the development of reading Literacy.
Combining newly developed reading assessment passages and questions for 2011 with a selection of secure assessment passages and questions from 2001 and 2006, PIRLS 2011 offers a state-of-the-art assessment of reading comprehension that allows measurement of change since 2001, and includes a full complement of questionnaires to investigate the experiences young children have at home and school in learning to read.\
This grade was chosen for PIRLS as it is an important transition point in children’s development as readers. Typically, at this point, students have learned how to read and are now reading to learn. Thus, shortcomings in understanding written texts at this stage may have a negative impact on performance in most other subject areas.
What questions can PIRLS answer
PIRLS is carefully constructed reading assessment, consisting of a test of reading literacy and questionnaires to collect information about fourth-grade children's reading literacy performance.
PIRLS will help Educators and policymakers by answering questions such as:
-
How well do our students read?
-
How students in one country compare with students in another country?
-
Do fourth-grade students value and enjoy reading?
-
Internationally, how do the reading habits and attitudes of students vary?
-
Do our students come from homes that foster literacy development?
-
How is early reading instruction organized in our country’s schools?
-
How do our teachers’ instructional practices compare with those of teachers in other countries?
-
Is reading achievement improving?
Why Participate?
In summary, PIRLS provides trends and international comparisons on:
-
Fourth graders’ reading achievement
-
Students’ competencies in relation to goals and standards for reading
education
-
The impact of the home environment and how parents can foster
reading literacy
-
The organization, time, and materials for learning to read in schools
-
Curriculum and classroom approaches to reading instruction.
Objectives of PIRLS
-
The objective of the study is to provide countries with international comparative data as well as with national trend data on reading literacy of 4th grade students.
-
The design of the PIRLS study focuses mainly on analyzing school systems, it is not designed to report individual student scores.
-
Besides the reading tests, questionnaire data from school principals, teachers, students and parents are collected and analyzed. These background information is intended to identify factors or combination of factors that are associated with high levels of reading literacy.
Target population
The international population for PIRLS includes students in the fourth grade of schooling. In Qatar, the PIRLS administration is census based. That means all eligible students in MoE, private Arabic and Independent schools in grade 4 participated in the PIRLS 2006 main administration.
The components of PIRLS
Definition of Reading Literacy
“For PIRLS, reading literacy is defined as the ability to understand and use those written language forms required by society and/or valued by the individual. Young readers can construct meaning from a variety of texts. They read to learn, to participate in communities of readers in school and everyday life, and for enjoyment.”
PIRLS is based on a comprehensive framework that calls for students to be able to demonstrate their understanding of a wide variety of texts classified under two major purposes for reading:
-
To acquire and use information.
-
For literary experience.
The Assessment
Like the Qatar Curriculum standards, PIRLS places a strong emphasis on critical analysis, not only a repetition of learned or read facts.
The PIRLS tests consist of texts that include one realistic story or tale and one informational text - both appropriate for the grade level and collected from several countries. The students have to read each of the two texts and answer some questions (both multiple choice and open ended questions) that test the depth of understanding of the students who will be also asked to engage in a full repertoire of reading skills and strategies, including:
-
Focus on Retrieve Explicitly Stated Information
-
Make straight forward inferences
-
Interpret and integrate ideas and information
-
Examining and evaluating text features (content, language and textual elements).
There are altogether five literary texts and five informational texts which are combined in different variation to produce a total of 13 different booklets containing each one literary text and one informational text. The students have about 80 minutes of testing time to read and answer question for the two texts.
Reading passages are printed in some student’s assessment booklets, while other students were given the “PIRLS Reader”, a short anthology of a variety of reading texts, in addition to the assessment booklet. Using different booklets allows PIRLS to report results from more assessment items than can fit in one booklet, without making the assessment longer.
To provide good coverage of each skill domain, the test items developed asked for about seven hours for each student by clustering items in blocks and randomly rotating the blocks of items throughout the ten student test booklets. As a consequence, no student receive all items but each item was answered by a representative sample of students.
Questionnaire
Background questionnaires were administered to collect information about student’s home and school experiences in learning to read. A student questionnaire addressed student’s attitudes towards reading and their reading habits. In addition, questionnaires were given to students’ teachers and school principals to gather information about students’ school experiences in developing reading literacy. In Qatar, a parent questionnaire is also administered.
In all, PIRLS takes 1½ to 2 hours of each student’s time, including the assessment and Background Questionnaire.
Encyclopedia
To provide a rich context for interpreting PIRLS results, each participating country contributes a chapter to the PIRLS Encyclopedia, describing the context for reading instruction in that country. The PIRLS 2001 and 2006 Encyclopedia provides general information on the cultural, societal, and economic situation in each country, as well as a more focused perspective on the structure and organization of the education system as it pertains to the promotion of reading literacy.
How was PIRLS constructed and administered?
A group of distinguished reading scholars, the Reading Development Group (RDG), was formed to construct the PIRLS framework and endorse the final reading assessment. Each country followed internationally prescribed procedures to ensure valid translations and representative samples students. Quality Control Monitor (QCM) then appointed in each country to monitor the testing sessions at the schools to ensure that the high standards of the PIRLS data collection process were met.
Reading literacy achievement was measured by using a selection of 5
Literary passages drawn from children’s storybooks and 5 informational texts. Submitted and reviewed by the PIRLS countries, the literary passages included realistic stories and traditional tales. The informational texts included chronological articles, a biographical article, and an informational leaflet.
Importance of PIRLS for Qatar
-
The analysis of student performance in PIRLS will inform Qatar about the present reading abilities of Qatari students in grade 4 as well as analyses of sub-groups (e.g. boys vs. girls, differences by school type, etc.).
-
PIRLS will provide comparative results of the position of Qatari students in comparison to those of 53 countries worldwide.
-
Since PIRLS is conducted every 5 years, Qatar’s participation in PIRLS 2006 represents the baseline from which the country can track its progress in 4th grade reading performance over time. The Qatari educational reform is expected to lift up students performance in the next 5 years. Thus, the comparison of 2006 and 2011 cycles of PIRLS will present Qatar with an external validation of the impact of its educational reform efforts on reading comprehension in grade 4 students.
-
Together with the questionnaire data it will also be possible to identify factors that are related to literacy acquisition, like instructional practices, school resources, encouragement for reading by the family, etc.
-
By pointing out strength and weaknesses in reading literacy of Qatari students, decision makers will have additional information about where to focus resources for improving early reading literacy acquisition.
PIRLS will complement results from the QCEA in the subject matter Arabic. PIRLS, due to its very specific focus on reading and understanding longer text passages, can offer deeper insights in reading literacy performance of Qatari primary students than annual tests in the subject matter Arabic may.
PIRLS is organized by the International Association for the Evaluation of Educational Achievement (IEA), which is an independent, international cooperative of national research institutions and governmental research agencies. More information about PIRLS is available online at : http://www.pirls.com/ where you can download a copy of the PIRLS international Report and the PIRLS Encyclopedia (2001 & 2006).
When PIRLS will be administered again?
PIRLS will be administered in the participating countries in 2016. The next PIRLS assessment is scheduled for 2021.
