Tests Only Measure Other Tests!

The PISA test results were released this week and once again the US is slipping in math test scores when compared with test results from 60 or so other countries. Of the 35 industrialized countries included the US ranks 31st.

When I was a fourth grade teacher many, many years ago in England we changed from one reading test to another one year without changing anything else; the reading program and the way I taught children to read remained exactly the same. The most incredible thing happened. All my students' reading abilities improved by more than a year. In other words, my fourth graders looked like they were suddenly reading as well as fifth graders. What a remarkably effective teacher I must have been that year. Not really since I did nothing differently. It was because the reading test was clearly easier or standardized differently from the previous one. This was my first experience of tests measuring other tests.  

In fact this very same phenomenon is mentioned at the end of the Hechinger report of the PISA results in these words:

"However, one test released last week —  the 2015 Trends in International Mathematics and Science Study (TIMSS) — showed a surprising gain for the U.S. in 8th grade math.  Historically, PISA and TIMSS tests have shown contradictory results for Eastern European countries and Russia, as they perform much better on TIMSS than on the PISA test. Scholars will need to explain the divergence for the U.S. this past year".

The PISA and TIMMS tests are clearly different and measure different things in maths. It is so easy to point fingers when you see bare test results such as these reported in the media but there are so many other things to consider. 

Take the maths, for example. Maths is not the same the world over. There are vast differences in the math itself before you even start to teach it. In most Asian countries, for example, the numbering system is so much easier for young children to learn than it is in the US. The teen numbers follow  the pattern ten-one, ten-two, ten-three and so on rather than the difficult eleven, twelve, thirteen, etc system we use. Look at which countries are at the top of the PISA table! In some countries such as Singapore children are given vast amounts of homework to do each night and not all children have the opportunity to attend public schools. There are differences in the sizes and cultural diversity of the countries being compared which can all affect score averages on a simple test. 

Like pretty much everything in our lives we can probably do better in maths education but we need to use caution when comparing test scores obtained from students in vastly differing cultures.