A+ A-

Literacy in SA surveyed in all 11 official languages


Researchers from the Centre for Evaluation & Assessment (CEA) at the University of Pretoria have completed the South African portion of the Progress in International Reading Literacy Study (PIRLS) to assess the state of numeracy and literacy levels in primary schools. The South African team had the unique logistical challenge of designing survey instruments and translating them into all 11 of South Africa’s official languages, resulting in more than 200 different survey instruments.

Education in South Africa is facing severe challenges, but the scale of the problem is not clear without detailed, rigorous research. For the team at the CEA, assessing the education system effectively meant designing assessments and questionnaires in all 11 official languages, reaching schools across the country, and mining vast quantities of data for useful answers. The small dedicated team were more than up for the challenge.

Prof Sarah Howie, National Research Coordinator for PIRLS South Africa, understands the education system. Its well-being is very close to her heart as she and her team want to prevent “another lost generation” of South African learners.

The PIRLS SA study measured more than 1000 variables in the home, classroom and school environments, using the assessment instruments designed by the team. This process is extremely complex when one factors in the different languages spoken at home and in the classroom, the language spoken by the teacher, parents or guardians, and the fact that all these variables are interlinked.

For the the 2016 survey specifically, Howie worked with a team that included Gabriel Mokoena, Mishack Tshele, Nelladee McLeod Palane, Karen Roux and Celeste Combrinck; each of whom played an important role in designing and implementing the survey, and processing the resulting data.

Mokoena provided logistical support that included the packaging of testing material, such as achievement booklets and questionnaires and also worked as a fieldwork coordinator. One of his tasks was to contact the randomly selected schools to arrange times to test the learners.

During the project he had to overcome challenges like community protests, as well as the difficulties associated with accessing schools in remote rural areas. These schools were difficult to contact as they often had no landlines or fax machines at the school, or the correct contact numbers were not available.

Mokoena explains that data collection happened mainly around November, which is exam time. “We often had to convince schools to cooperate, especially with Annual National Assessments being conducted at the same time,” he says.

Mishack Tshele, a data manager at the CEA, was responsible for the huge amounts of data produced as a result of the PIRLS project. He was the link between the CEA and the International Association for the Evaluation of Educational Achievement (IEA) data processing centre in Hamburg for everything involving national and international data. Tshele was also responsible for making sure that the translated surveys were still laid out correctly and accurate. He says he personally ensured the correct layout of more than 200 survey instruments.

Celeste Combrinck, Project Coordinator for the PIRLS study and Acting Director of the CEA, relished the research challenges of getting the PIRLS study design and implementation just right.  

Ms Combrink works in the area of psychometrics and, as part of the PIRLS team, used statistics to measure how well an instrument was functioning in the real world. She explains that she used principles of measurement similar to those used in the natural sciences.

“The major challenge in the social sciences is quantitative measurement, since the things we measure are usually qualitative in nature. Even aptitude or ability, like reading literacy, is qualitative,” she says. “Psychometrics is like trying to build a ruler, by using questions in a test.

"Measurement elevates a discipline to a science.”

While other countries participating in the PIRLS project may also encounter logistical hurdles, South Africa has the unique challenge of having many more languages to work with while still quantifying the data to make sure that, in Combrinck’s words, “all the points on the ruler are equal”.

Howie says that the researchers have risen admirably to the challenge of PIRLS 2016.

“The team at the CEA are fantastic. They are a small team - we were 13 when we were doing the fieldwork and every person had a job,” she says. “Each time we do this, I think it's impossible. Each time the team was different and each time the team rose to the occasion. We’ve been lucky to have such smart and driven people at the CEA working on the PIRLS project.”

The final PIRLS report was recently released both internationally and in South Africa.

CEA researchers will now embark on a new journey focussing on producing knowledge and recommendations from all of the data they have carefully produced and published.


The PIRLS 2016 team at the Centre for Evaluation & Assessment completed the South African portion of the international PIRLS study.

Privacy Policy

The University is firmly committed to protecting the privacy of users of the website. No personal information about users of this website will be disclosed to a third party without the prior consent thereto by the user. (Personal information shall at all times be dealt with in accordance with the provisions of the Protection of Personal Information Act, 2013 (Act 4 of 2013).)

The University reserves the right to automatically collect information on users' usage of the website (for example, via cookies) in order to improve users' browsing and interaction with the University and for non-personal statistical purposes.

Changes to this privacy policy

The University reserves the right to change, amend, or update this privacy policy periodically.

Modifications to the website

The University reserves the right to modify, change, amend or discontinue the website (or any part thereof) temporarily or permanently, without prior notice.

Links

The University may provide links to other websites or resources. This does not imply the University's endorsement of such sites. The University does not have any control over these websites and will, therefore, not be liable for any damages whatsoever arising from the utilisation of these websites by users.

The University does not prohibit third-party sites to link to publicly visible content on this website. However, it is expressly prohibited for any third party to frame any page on this website in any way whatsoever without the prior written approval of the University.

University of Pretoria proprietary rights

The copyright and other intellectual property rights (which include the University’s brand and logo), which are owned by or licensed to the University, existing in and attaching to this website, are the property of the University. These include but are not limited to text, content, design, layout, graphics, organisation, digital conversion and other information related to the website.

Users are granted a non-exclusive, non-transferable, revocable licence to:

  • access and use this website strictly in accordance with these terms;
  • use this website solely for personal, non-commercial purposes; and
  • download or print out or distribute content from the website, or any part thereof, solely for personal, non-commercial purposes, provided that all copyright and other intellectual property notices therein are unchanged.

Any reproduction of the content of this website, or a portion thereof, must include the following copyright notice: ©University of Pretoria. Users who wish to use the content from this website for commercial purposes may only do so with prior written permission from the University.

Disclaimer

This website is for information purposes only. No representations or warranties are given by the University of Pretoria (hereafter referred to as the University) regarding the accuracy of the information this website contains, any material this website provides for or any part of this website. Any reliance by the user on any information this website contains, any material this website provides for or any part of this website, is at the user’s own risk and the University shall not be liable in any way whatsoever in respect of the user or any other person, directly or indirectly, arising from the utilisation of the information this website contains, any material this website provides for or any part of this website.

The user hereby agrees that in the event of any dispute arising from the utilisation of this website in any manner, form or substance whatsoever, the relevant South African law will apply and the appropriate courts of South Africa will have jurisdiction.

Terms & Conditions

By accessing this website, the user hereby agrees to the following:

The use of this website is at the user’s sole risk. This website is provided on an "as is" and "as available" basis. The University gives no warranty that (i) the information posted on this website will meet the user’s requirements; (ii) the information posted on this website will be uninterrupted, timely, secure, virus free or error free; and (iii) the information posted on this website will be accurate or reliable.

Any material downloaded from or otherwise obtained through this website is utilised at the user’s own risk, and the user will, therefore, be liable for any and all damages of any nature whatsoever arising from such utilisation of the website.

Limitation of liability

The user expressly understands and agrees that the University shall not be liable for any damages (subject to the provisions of Chapter 2 of the Consumer Protection Act, 2008 (Act 68 of 2008) (even if the University has been advised of the possibility of such damages) resulting from: (i) the use or the inability to use the website; (ii) the cost of procurement of substitute goods and services resulting from any data, information or services obtained or messages received or transactions entered into through the website; (iii) unauthorised access to or alteration of the user’s transmissions or data; (iv) statements or conduct of any third party on the website; or (v) any other matter relating to the website.