League Tables: Will the truth really hurt?

There is a long history in NSW related to the publication of data on student performance that can facilitate the publication of league tables. So concerned have educationalists been about this matter, legislation was introduced to ensure that the results of the Basic Skills Tests were protected from such manipulation. In addition, who could approve some years ago of the Daily Telegraph’s labelling of students at Mount Druitt High School as a bunch of failures because no student had a TER above 50. Since that time, governments and the community have become even more suspicious of how the media might interpret and use this data, but the pressure from parents and the community to provide more comparative data remains intense.

In recent months, the requirement of the Commonwealth Government for states to release more information on student performance has again raised the possibility of league tables. Those concerned about league tables have had their fears realised in Queensland and Tasmania where web site information about school performance has been used by newspapers to construct and publish league tables. It would seem that what is considered by many as an undesirable possibility is now becoming a reality.

In NSW, the Government has had to introduce legislation to amend the previous prohibition on the release of this data so that it can comply with the Commonwealth’s stated intention which will also be a requirement of funding. The NSW Opposition has had the support in the Upper House of the Greens and the Shooter’s Party to propose alternative legislation that maintains the prohibition on leagues tables and prescribes punishments for any publication. Given Liberal-National support in the past for league tables, how long this position can be maintained is not known. It is likely that they will give in, leaving only the minor parties to challenge the publication of these tables. In all of these matters the teachers’ unions are likely to play an important role and how much they will eventually oppose and stand up on this educational issue, as opposed to an industrial issue, will be interesting to observe.

Just what is being proposed? It appears that data will be provided on both schools that are geographically grouped and those socioeconomically grouped. That is, parents will have information about how local schools compare regardless of the education system they belong to and also data about how each school ranks with other similar schools mainly grouped on socioeconomic data. Parents will be most excited about geographic data and may consider the implications of the other data. Highest ranking schools geographically will no doubt attract the increased requests for enrolment tapered only by the fee structures.

The basis of the ranking of schools will initially be the Year 3, 5, 7 and 9 literacy and numeracy tests. At a later time tests in science and history/civics might be used. Using a statistical process, School Certificate and HSC could also be used to provide similar information for NSW schools. This information already exists or could be retrieved but the legislation prevents any publication. The proposed change in legislation will make the use of this data irresistible.

The usefulness of Basic Skills and other tests has always been their diagnostic value and how schools could use the information to improve teaching and assist individual students to improve. That teacher suspicion about these tests has negated these benefits has been a disappointment and affected the learning of students. A recent New Zealand study of how teachers followed up testing programs in that country concluded National testing will not improve literacy results unless teachers are taught how to interpret the data properly and help students make improvements.

One of the strongest proponents of national testing has been the Australian Council for Educational Research (ACER) which in recent years has positioned itself to construct these tests. In 2000 while working for this organisation Ken Rowe in the Journal of Educational Enquiry Vol 1 No 1 stressed caution in generating and publishing potentially invalid and misleading information ... consisting of schools’ raw "ability adjusted" or "value added" average achievement scores, with the risk of generating both individual and institutional harm. His arguments about this "harm" are well presented, but have been unheeded by his own organisation in the rush to secure government funds for such tasks.

The education literature and research is full of such precautions, but the quest continues and is driven by community pressure. In the Sydney Morning Herald 20 August 2009, Ken Boston, former Director-General of NSW Department of Education and Training and until recently CEO Qualifications and Curriculum Authority in England spoke about the British experience of league tables.

The high stakes attached to league tables in England had "seriously damaged the breadth and quality" of the primary school curriculum, making it "narrower and poorer". The role of national tests had changed from providing a diagnostic tool for improvement to a determinant of a teacher's future employment. As a result, a recent survey had shown 70 per cent of primary schools were spending three hours a week on prepping students for literacy and numeracy tests, which had narrowed the focus on other subjects.

About 68 per cent of primary schools had employed additional staff to help prepare students for national tests.

The distortion of teaching, curriculum, resources and testing to satisfy the desire for dubious league table data is a warning to all.

Christian perspective

What do Christians think about this issue? Can we bring a Christian perspective to these deliberations? The answer is yes, and Christians should be applying their understanding of God and Biblical principles to this issue so that their voice can be heard amongst all the other voices with an opinion on this matter.

Some of the values for consideration from a Christian perspective are:

Truth - is fundamentally linked to the character of God. The search for truth has traditionally been the major focus of Christian education. It would seem that parents and students have some rights to the truth about their performance compared to other students but this could be provided by the use of standards. Already the HSC provides standard- referenced marks related to six bands of performance. This information provides information about not only state wide standards but performance from year to year as the standards represented by the bands do not change.

From this information comparisons with other students are possible and by publishing the number of students within bands or percentages of students within bands at different schools league tables is possible. Newspapers continue to do this based on the publication of students and their schools in the top band and students who achieve Band 6 in all subjects. The public is getting use to this data and knows that it does not really provide much more than evidence of where the brightest students attend school given the dominance of the government selective schools. Much more data is needed to make an assessment of the quality of schools. Providing limit "truth" is not very helpful.

For years the NSW Department of Education and Training has been calculating school performance using the Basic Skills tests, School Certificate and HSC data to develop "value added" measures for each school. This data has not been publically available but has been progressively released to schools about their own performance. In recent years, like schools have been grouped and comparisons made and this data too has been released to individual schools to help them understand how they are performing against other schools. This data has not been made public. More of the "truth" has been released to schools but the community remains mostly in the dark but continues to demand more from governments about the performance of schools. The question is: What sort of data would give them the information they want? Is the release of data on literacy and numeracy any measure of school performance or just a reflection of the intelligence and abilities of the students within that school? Without value added data this position could well be argued.

Others have wanted to point out that the correlation between postcode and performance is very strong. How should that be interpreted? Are bright students unevenly distributed across socioeconomic groups? What other factors affect student performance including teacher expectations?

The truth is elusive when it comes to school performance. The sort of data to be released will be important for making any decisions about the performance of schools and where parents might like to send their children. More might be better than restricting the data to literacy and numeracy.

Accountability is a focus of Christians who realise their accountability to God given all that he has done for them in Christ. They want to serve him out of gratitude and are accountable to him for their actions on this earth.

In a secular society, we are all accountable to someone and public schools are accountable to the community through government and private schools to their own authorities and all schools are accountable to their school communities, namely parents.

The issue of accountability has two different and conflicting sides. Parents want the information about student performance and schools should provide this in meaningful ways. But the other side is about what this data indicates about schools and teachers. There can be brilliant teachers in poor performing schools and vice versa. School data hides what educationalists have known for years that the greatest variation in performance occurs not between schools but between classrooms. High performing schools are more likely to have a greater number of high performing teachers. There is a real fear that performance data will be used to hold teachers accountable and this has industrial implications and is one reason why the teachers’ unions are so interested in this matter. The bottom line is that, Christians would want to be accountable and would see their work as part of their worship. It is just so important for their witness that they are good teachers. As Christians they would not want to be shy about accountability, but would want to ensure that any measures used are fair and reflect a bigger picture than narrow measures of academic achievement. (More about this later.)

Love is also a fundamental characteristic of God. How truth and love work together is an everlasting pursuit. The truth can be brutal and hurtful, and that is unloving. Truth that is delivered with love can be empowering. Both need to be held together in juxtaposition. There was nothing loving about the way the Daily Telegraph reported the performance of the Mount Druitt High School students HSC results. The truth about what these students had achieved was also obscured. The fact that not all had even applied for a TER and what they had already lined up as post school options was never revealed. Reducing truth to a single composite exam score hides too much and is simply not a reliable indicator of what students know and can do. To use it to bludgeon students, schools or education systems is unloving.

So Christians would want to support a broader accountability that recognises student achievement in a range of areas including academic, which allows comparison and provides some measure against standards. Holistic reporting is a more loving approach and therefore more Christian.

Problems to be addressed:

1. The need for comparative information

   Across the nation, schools have always reported on student performance. The reports have contained information about individual growth and sometimes included information about how the child was performing against standards and their own cohort, usually within the school. Some schools have done this better than others. Many parents of primary aged students have been concerned that the information they received rarely gave any comparisons within the school or beyond the school. Unsure about how their child was performing they would come to a reality check when high school reports provided more comparative information within the school and then state-wide information at Year 10. It was then too late to address learning deficits.

   Recent changes to reporting, involving standards represented by five grades, have improved the reporting against standards across all schools but parents want more to make better judgements about which school might be best for their child. There is little consolation when a child performs highly in a poor performing school if that level of performance ranks them lowly in another school. Parents want more information and governments want to be seen to be reacting by responding to this demand.

   In terms of accountability to parents governments and educationalist are going to have to provide more information to parents about school performance, but what should be provided?

2. The data available and its reliability

   In any discussion about providing additional information to parents some fundamental purposes of education need to be revisited so that this matter can be held in perspective. Education has always been more than academic performance. Educating the whole child is not simply rhetoric. Children also develop physically, emotionally and socially while at school and schools foster these developments. Some children may improve academically as a result of changing schools while other could flounder amongst a new set of social relationships that cause them anxiety. What is good for one child it not necessarily a solution for another. While most parents know this, it is difficult to let these matters over ride the desire for improved academic performance, but they remain key factors that develop a child into a responsible adult able to handle relationships in work and in the community. Decisions to chase academic performance need to be placed in a broader perspective.

   Given this broader perspective: What data might be provided? Currently only the national literacy and numeracy tests at Years 3, 5, 7 and 9 are provided. This data will allow for the different types of ranking being proposed. It will be easy to organise league tables. But is the data an adequate or even accurate measure of student performance? After all, it is only part of two subjects English and mathematics. Is the information provided here enough to base a decision about which school is best or best for my child? How closely do these test results relate to IQ? Could all the money on testing be saved by using IQ to rank schools?

   Literacy and numeracy tests are narrow measures of student ability and might be considered inappropriate to base decisions about whole school performance that are made from associated league tables. What about science, the humanities and the arts - are these not worthy of inclusion in measures of student performance?

   Another aspect of this issue is how the national focus on literacy and numeracy will affect the curriculum. Will English and mathematics become the dominant subjects for most of Years K-9 so that test performance can be maximised? In NSW, some primary schools in the past have prided themselves on being literacy and numeracy schools to the detriment of other subjects in the curriculum.

   The bottom line is: Is it fair to make judgements about schools based on the narrow data of literacy and numeracy?

3. More and not a single number

   Parents need more information and in recent years this data has been provided in NSW by the reporting framework of A-E standards. Schools have been increasingly aware of what these standards are through the work samples of the Board of Studies assessment website. Parents can be increasingly sure about how their children are performing against state wide standards but they still lack information about how other schools are performing.

   Can this information be best provided by literacy and numeracy tests or are wider measures of performance needed? At this point, it should recognised that composite marks that lead to league tables can disguise anomalies. Perhaps separate leagues tables for literacy and numeracy would be more useful.

   In this regard, more national testing that examines a wider selection of the curriculum is preferred and the reporting should be by separate league tables for each measure. In this way parents can make better decisions and the stigma of a single number league table is removed. More, not less, could be the best way to provide better more comprehensive information to parents and to depower the single digit league tables showing them up for the limited information they provide.

   Other league tables would also help. Why not rank schools on the implementation of student welfare initiative including measures of absenteeism, homework completion and incidents of bullying? Why not survey students about their teachers and publish league tables about student perceptions of teaching? Why not establish league tables about the amount of involvement schools have in their communities or how students express civic awareness? The list could be limitless and all would provide useful information for parents.

   So why do we want to place so much emphasis on one composite score of literacy and numeracy to rank schools? Without other broader academic and social measures these proposed league tables are not going to provide much that is useful and will do considerable harm to teaching, the curriculum, the use of school resources and existing testing programs. However, additional broader measures and separate reporting of all the components might provide more of the truth.

   The current proposals tell part of the truth and will distort the true picture by not focusing on the broader academic and social goals of schooling. As a consequence, parents will be misled about whole school performance, resources misallocated, the curriculum narrowed and students and schools placed under pressure that is unlikely to yield higher outcomes except in the narrow areas measured.

John Gore