Much has been said about accountability measures and systems in education over the past year or so. This is a good thing as the debate is much needed due to the flaws in the system as it currently stands. Accountability systems are crucial for ensuring that the quality of our education system improves and accountability measures are a significant lever in educational reform.
I am not offering an all-encompassing solution to the issues, but offer a suggestion to one key issue, that of clearly demonstrating the effectiveness of a school or schools for all children within a given cohort. This is set in the context of some of the blogs that I have read that have helped to shape my thinking. My thoughts will appear obvious to many, and it may well be that similar ideas have already been shared by others; apologies if this is the case.
Others have written extensively about the challenges of accountability measures:
Chris Husbands (@Director_IOE) responded to press reports that an average points score measure would replace the existing 5 A*-C GCSEs including English and maths measure. Of particular value here was the discussion of the difficulty of using the C/D borderline within accountability measures because of its ‘cliff edge’ nature. The result is that the achievements of a significant proportion of students within a cohort go unreported.
Tom Sherrington (@headguruteacher) wrote as comprehensively as ever about the wider issues surrounding accountability – performance measures, inspection, evaluation. He suggested sound ideas for resolving the challenges based on the triangulation of information: about the quality of teaching and learning; contextualised attainment and progress measures; school evaluation and its impact.
Ros McMullen (@RosMcM) contributed an account of the interplay between accountability, school improvement and system improvement during four identifiable stages of school improvement. There was a hint, at a system level, of the dangers of a zero sum game, in which the success of one school is to the detriment of others.
Yesterday I was struck by a blog written by Loic Menzies (@LKMco) about two problems associated with accountability measures – using average progress measures and thresholds. Average progress measures can result in the excellent progress of some students masking the poor progress of others, and a ‘percentage making expected progress’ measure is of value here. But the problem of using attainment measures within an accountability system, and the associated dangers of cliff edge threshold measures, remains. As with progress measures, some students may benefit to the detriment of others.
Of course threshold measures do have a place in accountability systems, as argued by Simon Burgess at The CMPO, who suggested recently that a threshold can be useful in setting a minimum standard of achievement to which all should aspire.
But perhaps it is possible to combine the concepts of progress and attainment in a way that provides insight into a school’s performance for parents, educationalists and those in political power?
I tend towards the use graphs to display information when comparisons are needed and propose a plot of a cohort’s actual attainment in attainment bands (eg 5 C+ grades, 5 B+ grades and so on), compared to the cohort’s expected performance in comparison to national levels according to prior attainment. This generates a clear view of achievement across a range of attainment levels, removing a single cliff edge but demonstrating a school’s effectiveness at each equally important attainment level.
The fictitious Red School is used to illustrate the point.
Attainment. The attainment measure chosen to illustrate the concept is 5 GCSE grades at a particular level as that is currently the dominant measure in performance tables. This could be changed to any grading system, using any number of qualifications, with or without any particular combination of subjects (for example the inclusion of English and maths). It could also be applied to an average point score system, for example ‘> 100 points’, ‘> 200 points’ and so on.
% of cohort achieving within each grade band represents the percentage of a cohort achieving the particular requirements of each grade band. In this case 60% of the cohort attained at least 5 grade A*s, 98% attained at least 5 grade Gs.
Expected for cohort is an indicator of what the cohort should have attained based on their prior attainment compared to the attainment of students with the same prior attainment nationally.
The Red School is successful. All students achieve at a higher level than expected based upon their prior attainment. Students in this cohort have demonstrated particularly high achievement, as a greater proportion of students have achieved higher grades than expected based on prior attainment.
The Purple School has a mixed profile. Those students who were expected to achieve at a higher level – at least 5 grade Bs, As or A*s – have done so; they have made good progress. Students who, based on prior attainment, would be expected to achieve up to 5 C grades have fared less well; their progress has not been as good. This provides quick access to a good deal of information about a school for parents, school leaders and those ensuring the quality of provision.
The differing nature of cohorts can be identified by the expected attainment indicators. The Blue School below has a cohort of very high prior attainment – all are expected to achieve 5 grade Cs and 95% are expected to achieve 5 A*s. Achievement of this cohort is high and exceeds expectations in comparison to national levels.
With this method it is hard to differentiate between schools with very, very high levels of achievement. One solution would be to present additional information and plot the proportion of students achieving more than 8 or 10 of each grade, as seen below. It is now possible to see a marked difference between achievement of 5 and 8 grades in a particular band.
Problems with the attainment and achievement of students at a school would be easily identified, such as seen at the Orange School below. There is a significant problem with the outcomes of students expected to achieve at least 5 grade Cs.
I think there are some clear benefits to this model as a small part of a wider accountability system. There are also problems, as nothing is ever without fault.
- There is no cliff edge because of the inclusion of outcomes at a range of attainment levels.
- Progress is included due to the comparison of actual and expected progress.
- This methodology could be applied to any number of performance indicators to demonstrate the wider successes of a school.
- Student groupings beyond prior attainment could be explored to make use of the wealth of information available at a national level.
- The progress of individual students could still be masked within the cohort.
- The issue of determining expected attainment based on prior attainment is unresolved. It could be based upon the average progress made by students nationally according to prior attainment.
- This method does not seek to address how start or end attainment should be measured, but merely indicates relative progress from point one to another.
- Some will argue that the expected attainment measure will be seen by some as a limit to aspiration and once achieved, the need to strive beyond disappears. This is is where the wider accountability systems come into play to prevent stagnation.
Transparent and understandable accountability measures are required to support and challenge schools. This is one idea. Neither list above is exhaustive and I’m interested in hearing the thoughts of others about this proposal at this early stage.