The Grattan Institute has released its first report, an analysis of student progress measures by Ben Jensen. It argues that ‘value-added’ measures – that is, how much students improve between NAPLAN tests – are a more useful way of assessing a school’s performance than simply looking at its absolute results.
The report meets Grattan’s claims to be ‘objective, evidence-driven and non-aligned’. It is well-researched, uses data, and presents ideas that could easily be adopted by either major political party. While the media played up its differences with the about-to-be-launched My School website, it’s hard to imagine that Julia Gillard would have any fundamental objection to the ideas presented. And that was pretty much how she handled it today:
From what I’ve seen of the reports of the Grattan Institute work, they are saying that this is a good start but they are wanting to see more. Of course we are going to keep building on this website year by year as we get more results from national testing, more results on Year 12 retention, more results on vocational education and training pathways and attendance at school.
[Update 28/1: Barry McGaw, chair of the Curriculum, Assessment and Reporting Authority, confirms that value-added measures are on the agenda when the data becomes available.]
Grattan has done well in targeting different markets for the paper. For those who might be put off by the report’s including things like ‘similar to residuals in a standard regression model, these scores sum to zero which means that a school with a value added score of zero …’ there is a YouTube summary.
I have to put my editor’s hat on to make my only real criticism of the first Grattan paper. It contains too many examples of the stage-3 error of footnote numbers inside the punctuation for this to be just a proofreading lapse. The main cause of this error is the influence of the author-date system of referencing, which does go inside the punctuation. While the author-date system is widely used, in my view it should be banished from all work intended for a general audience. It disrupts the argument’s flow with clunkily-expressed information of little interest to most readers.
Andrew Norton 
I haven’t read the report itself, only the op-ed by Ben Jensen in today’s Oz. The op-ed is a bit limp: on the one hand, it applauds the My Schools website for offering “the greatest increase in transparency in decades”, allowing parents and families “to compare the average score of students in their school with other, similar schools”. This suggests that Jensen believes the website provides valuable information. But straight after, Jensen argues that because the average NAPLAN score data don’t take account of the socioeconomic backgrounds of students, the indicators may be biased and hence ought to be replaced by ‘value-added measures’. This suggests that Jensen believes the raw NAPLAN data are misleading. So is My Schools in its present form a good thing or not? It’s not clear, but assuming the latter, Jensen fails to consider the very real possibility that parents may actually be interested in the absolute average scores at a school irrespective of students’ socioeconomic backgrounds or rates of improvement. This is partly because ‘value added’ measures may themselves suffer from distortions. For example, assuming diminishing educational returns to school quality, a given % improvement in scores between years 3 and 5 may be easier to secure at a low-achieving school than at a high-achieving school. So is it fair to compare a 10% improvement at school A (from 40% to 50%) with a 5% improvement at school B (from 80% to 85%)? Also, what good are rates of improvement if a parent wants to send his child to a school that will offer the child a compatible and challenging cohort? No parent in their right mind would send a child who scored 90% on the test to a school where the average score was 60%, whatever the rate of improvement between years 3 to 5. And then there is the scope for schools to game value-added measures: a canny school might deliberately underprepare year 3 students for the tests to generate a disproportionate rate of improvement between years 3 and 5. Jensen would have had more credibility if he argued for absolute scores to be supplemented by value-added measures rather than replaced by them. An inadvertant slip? I think not. His op-ed gives the impression of someone trying to appease both sides of the debate rather than stand up for a valuable (albeit imperfect) reform.
LikeLike
I agree with Rajat. I’m not sure if it’s possible without running into some privacy issues, but the best thing would be to release enough data so that people can replicate the Fed Govt’s own value-added scores and to create their own “value-added” measures. That would of course include the raw average scores.
What I’d also like to see is if school A falls into the bottom 25% (in regards to raw average scores) in one year, then the next year the students of school A would be free to apply to a government school of their choice outside their allocated zone, say school B, and be treated as if they lived within school B’s zone.
LikeLike
I agree with Rajat also — especially because the ‘value added’ component will have different effects on different individuals based on factors that you can apriori identify and are already known in the educational literature (e.g., your ability compared to the average ability of the class — Rajat’s challenging cohort).
.
Alternatively, I do think the reason this is being done is simply so the government can cynically say it is doing something and is hence generating some noise about having an “education revolution” when the overall effect on the system this change is going to have is tiny at best.
LikeLike
Brendan – I don’t see how your proposal could work. What happens to the children left in the poor-scoring school? There are plenty of parents who don’t care, or who will opt for the convenience of the closer school.
How does moving a child to a school in a higher socio-economic area (because the socio-economics mirrors the scores) give that child a good night’s sleep, breakfast, someone who checks and helps with homework etc?
Given there will always be kids in the bottom 25% how would the higher scoring schools cope with the continuous inflow of students from those schools – physically many of them don’t have enough room as it is.
LikeLike
If we are going to have league tables why not do it properly.
Each school would be benchmarked in their own league not all in. If a school was in the last 25% in their league they would be relegated to a lower league whilst the top 25% were bumped up to stiffer competition.
The lowest in any league would have the pick of the top individual student performers for the last year and the top performing schools would have to take the low performing students from the previous year from other schools.
We could have similar rules for ranking teachers and shift them around too.
LikeLike
Francis, surely you are joking. Why would you create incentives for students to do poorly? Shifting teachers around can be achieved much more simply than by using league tables – through giving principals autonomy and accountability, including the right to hire and fire.
LikeLike
“What happens to the children left in the poor-scoring school?”
Are they any better off in the status quo? Wouldn’t they still be there anyway? If parents want to send their children to the closest school that’s their choice and one has to respect their freedom to choose. And if parents want to send them somewhere else, that’s their choice as well. I do not accept that artificial barriers like school zones should be set up so that people who want to send their child to the closest school can feel better about themselves.
“How does moving a child to a school in a higher socio-economic area (because the socio-economics mirrors the scores) give that child a good night’s sleep, breakfast, someone who checks and helps with homework etc?”
I never implied that. The students would be given the option to move, not the obligation.
“how would the higher scoring schools cope with the continuous inflow of students from those schools – physically many of them don’t have enough room as it is.”
If that were the case, it would be the same way they would cope if those children had physically moved into the school zone – by expanding.
LikeLike
Brendan – the point is that your proposal doesn’t improve the status quo for the kids left in the poor scoring school, whereas I think it’s the government’s responsibility to improve that school. I’m surprised that you are standing up for parental neglect as a matter of ‘freedom to choose’ – the child isn’t able to choose yet they suffer the consequences.
You don’t address the non-school issues that affect school performance – isn’t it time to stop blaming schools for being solely responsible for results?
What happens to schools that are forced to grow and grow – up to 3,000 pupils?, 5,000 pupils? – under your plan? Wouldn’t you be placing so much pressure on the high-scoring schools that you would reduce their ability to remain good schools?
LikeLike
I just read Crikey and a contibutor makes the point that many of the lowest scoring schools are in isolated or remote communities – so no opportunity to simply send the kid to a better nearby school.
Like FXH, I suspect, moving the teachers might be a better option. Unlike Rajat, I would prefer to do that with volunteers (for more money) rather than hiring and firing.
LikeLike
Russell: that is what boarding schools are for and is precisely why they have historically been used. Funding more remote area scholarships might be useful.
I think it significant that the only Education Minister who seems keen on giving parents public information about school performance is the only Education Minister not responsible for running any schools. Until the patent conflict of interest of the biggest provider being also the regulator is addressed, school performance will continue to be more problematic than it should be.
LikeLike
Michael – I’m getting a Pied Piper image here of all these country kids being piped into these new ‘boarding schools’ – it’s very generous of you because it sounds like it would cost a fortune. Closing the low-scoring schools in those country towns perhaps wouldn’t do much for the local economies ?
LikeLike
I’m not sure what is so bad about this.
LikeLike