Towards an index of education outputs

In addition to the Data Bank, an important goal of the Learning Curve project has been to create a comparative index of educational performance – the Global Index of Cognitive Skills and Educational Attainment. The results are meant not only to be interesting in themselves, but to help identify likely sources of good practice.

First, a caveat

The exercise has not been simple. One hurdle was determining how to measure performance. While it would have been desirable to include broader labour market and social outcomes on which education arguably has an impact, this proved impossible. Even were it demonstrably clear that education played a definite role in these areas, it is impossible to determine a way – consistent across time and geography – to isolate and measure the impact of that effect.

While more direct measures of educational results abound, robust, internationally comparative ones are rare. PISA, TIMSS and PIRLS testing has had such an impact in part because of the void it helped to fill. The Index therefore, through necessity, takes a view of educational performance based on where reasonably good data exist. The first such area, drawing on the results of the aforementioned tests, is the inculcation of cognitive skills. The second is a broader measure of educational attainment, which relies on literacy levels and graduation rates.

This focus does not eliminate data issues. Education systems are local: international comparability will never be perfect. Canada’s tertiary graduation rate, for example, is modest in the calculations for this Index because they draw on university results. If one includes graduates from Canada’s community colleges, though – tertiary type-B institutions to use the international classification – the graduation rate becomes one of the highest in the OECD. A lack of data on the results for type-B colleges, though, makes it impossible to do so generally. Moreover, metrics selected for the Index suffer from data lacunae. Singapore’s low educational attainment score in the Index – 33rd out of 40 – arises largely from a complete lack of available data on graduation rates.[28] Finally, combining results from different tests in a meaningful way required rebalancing of the existing data.

Ultimately, these data are inevitably proxies for broader results, and far from perfect ones. As Dr Finn points out of graduation rates, "they are complicated. You can raise your graduation rate by lowering academic expectations.” On the other hand, such rates, like literacy levels, do indicate in a rough way the breadth of education in a country. Similarly, Professor Hanushek notes that “countries that do well on PISA do well on tests of deeper knowledge.”

The methodology appendix describes in more detail the Index’s construction and relevant data issues. The broader message of this lengthy disclaimer is that the Index is very much a first step. We hope that, as understanding of the outcomes of education grows, the Index will become more complex and nuanced as well as be populated with more robust and varied data. For now, however, it is better to light a candle than curse the statistical darkness.

What the leaders have – and don't have – in common

Given the attention paid to the results of international education tests, the leading countries in the cognitive skills category of the Index come as no surprise. The top five – Finland, Singapore, Hong Kong, South Korea and Japan – all score more than one standard deviation above the norm in this part of the Index. The educational attainment category, based on literacy and graduation rates, tells a slightly different story. Here South Korea leads, followed by the UK, Finland, Poland and Ireland, with Japan, Hong Kong and Singapore further down the table. Because of their strength in both measures, then, Finland and South Korea are the clear overall leaders of the Index.

Chart 9: Global Index of Cognitive Skills and Educational Attainment – overall results

Chart 9: Global Index of Cognitive Skills and Educational Attainment – overall results

Note: The Index scores are represented as z-scores. The process of normalising all values in the Index into z-scores enables a direct comparison of country performance across all the indicators. A z-score indicates how many standard deviations an observation is above or below the mean of the countries in the Index.

Source: Economist Intelligence Unit.

These results mirror the conventional wisdom: already in 2007, the BBC referred to the two countries as “among the superpowers of education.”[29] But what do these have in common that might help to identify the keys to educational success? On the face of it, there is remarkably little.

In many ways, it is hard to find two education systems more different. South Korea’s schools are frequently described as test-driven, with a rigid curriculum and an emphasis on rote learning. Most striking is the amount of time spent in study. Once the formal school day is over, the majority of students go to private crammer schools, or hagwons. According to OECD data, of 15-year-old students for whom data was available in 2009, 68% engaged in private study of the Korean language, 77% in mathematics, 57% in science and 67% in other subjects. In later years, students typically do far more privately. The government has become so worried about the extent of these studies that it has banned hagwons from being open after 10pm, but still needs to send out patrols to shut down those which mask illegal, after-hour teaching by posing as self-study libraries.

On the other hand Finland, in the words of Professor Schwartz, “is a wonderful case study. Kids start school later; school hours are shorter than most others; they don’t assign homework; their teachers are in front of kids less. By one estimate, Italians go to school three years longer.” The PISA data shows that very few Finns take out-of-school lessons either, and those who do typically do worse on standardised tests, suggesting that this is largely remedial help. Finally, the system has a reputation for being focussed on helping children understand and apply knowledge, not merely repeat it.

The existing data also paint a picture of two distinct approaches. In some cases, the systems are widely different: average teacher salaries in South Korea are over twice the national average, while those in Finland are almost exactly average; pupil-teacher ratios, on the other hand, are much higher in South Korea. Where the two systems are similar, they are usually near the average for all countries in the Index. The only difference is school choice, where both are highly restrictive. That said, the vast amount of after-school private education in South Korea brings into question the relevance of that metric.

The two systems, though, do share some important aspects when examined closely. “When you look at both, you find nothing in common at first,” says Professor Schleicher, “but then find they are very similar in outlook.” One element of this is the importance assigned to teaching and the efforts put into teacher recruitment and training. As discussed above, the practices of the two countries differ markedly, but the status which teaching achieves and the resultant high quality of instruction are similar. Professor Schleicher adds that both systems also have a high level of ambition for students and a strong sense of accountability, but again these are “articulated differently. In South Korea, accountability is exam driven; in Finland, it is peer accountability, but the impact is very similar.”

Finally, there are cultural parallels. The two societies are highly supportive of both the school system itself and of education in general. Of course, other countries are also highly supportive of education, but what may set Finland and South Korea apart is that in both, ideas about education have also been shaped by a significant underlying moral purpose.

Although discussions of Korean attitudes to education frequently reference Confucian ideals, under a quarter of South Koreans were even literate by the end of the Korean War. In the decades that followed, education was not just about self-improvement: it was a way to build the country, especially as the Japanese colonial power had restricted the access of ethnic Koreans to schooling. The immediate cause of this drive has disappeared, but it has helped inculcate a lasting ethic of education which only strengthened the more widespread attitude in Asia that learning is a moral duty to the family and society as well as a necessary means of individual advancement.

In Finland, the ethos is different but no less powerful. As Mr Mackay explains, that country has made “a commitment as a nation to invest in learning as a way of lifting its commitment to equity. They wish to lift the learning of all people: it is about a moral purpose that comes from both a deeper cultural level and a commitment at a political-social level.” In other words, education is seen as an act of social justice.

Both of these moral purposes can cause difficulties in different ways. The high expectations and pressure mean that studies regularly find South Korean teenagers to be the least happy in the OECD. In Finland, the egalitarian system seems less effective at helping highly talented students to perform to the best of their ability than at making sure average results are high. Nevertheless, the power of these attitudes in shaping cultural norms and political decisions in ways that help education attainment overall are undeniable. Mr Angula, after many years as a teacher, Minister of Education, and Prime Minister, believes that “the key ingredient [in creating a successful education system] is for everybody to be committed and to understand that they are doing a public good.”

 


[28]Singapore is one of 14 countries in the Index for which internationally comparable graduation data are lacking. (The countries were nonetheless included in the Index because they met all the other data inclusion criteria.) They were thus assigned the mean z-score of the entire country sample for the given graduation rate indicators. This represents an opportunity for further and improved data collection that will be reflected in later versions of the Learning Curve.

[29] “Finland stays top of global class”, 4 December 2007, http://news.bbc.co.uk/1/hi/7126562.stm

 

[previous page]   [next page]

Download the 2014 report

Download Index data workbook

The Learning Curve articles

Subscribe to The Learning Curve

Receive the latest news and updates

Subscribe