(Updated on 08/12/16 to correct legend in first graphic — thanks to @aripwow)
Three years ago, I published my most-commented blog post, entitled “Indonesian kids don’t know how stupid they are”. It included data from the 2012 PISA tests, which compare how good 15 year-old schoolkids are at maths, science and reading across dozens of countries. Though Indonesians came near the bottom of the pile in all three areas of learning, they topped the charts for happiness at school.
Since then, the curriculum has been revamped in Indonesia, and the years of compulsory schooling increased. Now the results of the 2015 PISA tests are out. Indonesia’s average score has improved a little bit in maths and science, though students have moved backwards in reading. In science, the proportion scoring at level 1 or lower (in other words, 15 year-olds who can NOT even “use basic scientific knowledge to identify a valid conclusion from a simple data set” — example at the end of the post) has fallen by a princely 6% since 2006, but it still stands at over half of all students. In maths, two thirds of Indonesian 15-year-olds IN SCHOOL can still not “extract relevant information from a single source… and make literal interpretations of the results”, virtually unchanged since 2006. The proportion who can’t read and comprehend written Indonesian with any proficiency drifted up very slightly, from 53% in 2009 to 55% in 2015. Let me repeat that. In 2015, over half of Indonesian 15 year-olds surveyed were not able to “recognise the main idea in a text, understand relationships, or construe meaning within a limited part of the text when the information is not prominent.” And yes, ALL of the PISA tests are translated into Bahasa Indonesia. (If you want to know more about the data, the sampling, the exact types of questions asked, they are all comprehensively documented at http://www.oecd.org/pisa/. You can download the raw data and do your own analyses if you disbelieve me or the OECD analysts.)
My previous blog post was greeted with howls of protest from some who felt it just wasn’t fair to compare Indonesia with other, richer countries with very different economies. For their convenience, I’ve made some graphs that include only the participating countries that are also our neighbours in ASEAN. This one compares the proportion that can’t manage a Level 2 (of six) score in different domains. In other words, they don’t have enough reading ability, maths or science skills to function effectively in a modern economy.
Fewer than one Indonesian students in every four is effectively competent in all three areas. Some 85% of students in Singapore and over three quarters in Vietnam managed to score at least Level 2 across all areas, but we all know some of us are bad at maths, or lack the concentration to read carefully, so maybe 3/3 competence is too much to hope for. Still, the fact remains that more than two out of every five young Indonesians fails to meet minimum standards in all three domains simultaneously. (Girls clock up just 36% comprehensive incompetence, making them slightly more functional than boys, who manage three strikes in 46% of cases.)
This year’s report focuses on science, so let’s look at that in greater detail:
So: 56% of 15 year-olds in Indonesia score at Level 1 or below, thus performing two thirds worse than their neighbours in Malaysia. And to those who are itching to write and tell me about all the science Olympiads won by Indonesians, that’s admirable, but let’s also remember that they must be among the measly 2% that manage to answer questions at level 4, 5 and 6. Vietnam, whose per capita income is just over half of Indonesia’s according to the IMF, can muster 15 times that proportion of high performers.
How to explain this dismal performance, besides the absent teachers, underused resources etc.? Perhaps it is related to an excessive fondness for orthodoxy. Science progresses by questioning, by experimenting, by organising results and by analysing them critically. Sadly, there’s no Indonesian data available for the sub-indicators that measure these skills. When Indonesian students themselves were asked how they are taught science, quite high proportions of them say they are regularly encouraged to question, argue, experiment. (This contrasts with Thailand, the second-lowest achiever in science of the ASEAN countries that participated in PISA. There, far fewer students report being given opportunities for critical thinking or expression.)
But something in this equation doesn’t add up. Because despite all that supposed encouragement to think critically, Indonesian students were the least likely of all 72 nations/economies in the survey to think of science as a critical and iterative process. Just six in ten said that that ideas in science sometimes change, and an even fewer, 58%, said that ideas in science books and text books could change. It’s tempting to draw a parallel with the sort of mindset that sees blasphemy in the mere questioning of the interpretation of something written in a text that to some may be sacred. The photo above, taken in a school in West Kalimantan, illustrates the seriousness with which school texts are taken. While this girl’s teacher sat smoking in the staff room, she copied out her text book, diligently including all the incorrect answers for the multiple choice questions.
Some believe inter-country comparisons such as those provided by PISA are either meaningless or unhelpful. I don’t really see any value in comparing Indonesia with Finland or Peru. But comparing the competence of Indonesia’s soon-to-be-labourforce with that of its near neighbours makes more sense. Because since the inception of the ASEAN Economic Community last year, employers are at least in theory free to hire people from any ASEAN member. If you were staffing up a lab, would you rather pick from a pool where 56% of potential hires are scientifically illiterate (Indonesia), or from Vietnam, where the figure is 6%? That’s probably a Level 1 maths question, but it’s an important one for Indonesia to think about.
Sample Level 2 question, as promised. This comes after a whole introductory screed about fish farming, and another question about the design of fish farms.
Researchers have noticed that the water that is being returned to the ocean contains a large quantity of nutrients. Adding which of the following to the farm will reduce this problem?
More marsh grass
You can try other PISA questions if you like.