Making good use of data in schools

The use of ‘Flights Paths’ is an approach used by many schools to to represent pupil progress at key stage 3. While this approach appears to provide a solution to representing pupil progress without using Levels, it also has some inherent weaknesses which have attracted criticism. For example, there is a detailed article by David Didau – The Madness of Flight Paths – which explains why he feels that the flaws in this approach cancel out any usefulness that it may offer to schools. On this page I aim to summarise the thoughts of our team on this subject and to point to the thinking which has helped us to develop some options when using a data system to help to manage assessment at key stage 3. We hope these thoughts will be helpful where schools are reviewing their approaches to curriculum and assessment at key stage 3.

What is a Flight Path?
One approach to assessment at KS3 is the use of predictive Flight Paths. This is where a pupil’s attainment by the end of KS4 is estimated from the average of their English and Maths scores at the end of year 6.
This allows a line to be drawn on a graph based on a national average progression line. When the pupil’s current attainment is to be estimated the school will report, for each subject, whether a pupil is on, above or below their expected progress, usually as +1, 0 or -1 grades.
Below is the sort of example of a Flight Path (using old style letter grades). This takes the form of a set of lines of expectation, or ‘tramlines’, which provide predicted routes from KS2 through to an ‘expected’ GCSE grade.

There are several potential issues with this approach, including:

• not knowing whether the KS2 scores accurately reflect pupil ability
• assumptions made about future attainment in other subjects than En and Ma
• ignores the fact that there is a range of outcomes from any KS2 scores (below from FFT)

What are the practical problems with using Flight Paths?

A Flight Path tends to set in stone what the school thinks the pupil will attain all the way through to year 11. Yet it is based on another school’s assessment of a completely different curriculum. We should really have an expectation that all pupils could benefit from well-informed adaptive teaching and aim for all pupils to attain at the higher GCSE grades.

Assessments are formed by taking an estimate of current performance and comparing it to a prediction of future performance based on their KS2 average in English and maths. The Flight Path can appear to provide apparently clear results, for example Maths +1, Science -1, English 0 etc. i.e. above, below and at an expected level of progress.

It is worth discussing :

a. If a performance in maths and English is a good indicator of attainment in other subjects,
b. Whether the KS2 results of pupils drawn from different primary schools will be consistent,
b. Whether this information helps a pupil, for example, to make improvements,
c. Whether these judgments will orientate pupils towards future subject choices, or put them off,
d. Whether this information tells parents pupils and the school the right things about how pupils are doing.

Using the Flight Path approach and reporting in this way might indicate an over-simplistic approach to planning the curriculum and assessment of a subject. If a parent was to challenge a statement and ask how the teacher knows the pupil is ‘below expectation’, will the range of information to support this judgment be available. In particular, although we might justify a current assessment, it will be difficult to justify what it is being compared with, that is, a prediction based on a different curriculum, in a different subject, in a different school, made a year or two ago.

In a nutshell, what is wrong with Flight Paths is that: The best predictor of future attainment is current attainment rather than historical attainment. It would be better to report current attainment as a ‘working towards’ GCSE grade, or even better than that, to report what the pupil had learnt in each subject, and where the strengths and weaknesses lie.

At the start of Y7, the best information about pupil attainment will be their KS2 results – but from that point onwards, the best information about future attainment will be their current attainment, i.e. the information accumulated from formative assessments and standardised tests. Some schools already use baseline tests in year 7 to try to provide a more standardised starting point of comparison than KS2 scores.

But most of all, assessment using Flight Paths is a lost opportunity to report what parents would really like to know. In most cases this is: “What has my child learnt so far? How well have they learnt it? How can they improve? What grade are they likely to get at GCSE?” A system which can report this information is going to be much more useful and informative than simply reporting a single symbol or number.

How can we improve upon using Flight Paths?
In comparison with the ‘Tramline from KS2’ approach, the Curriculum-Driven Assessment (CDA) approach integral to 4Matrix supports the identification of a manageable number of Learning Objectives which are neither atomistic, nor over-summative (as in using a single number, often unwisely split into sub-units).
The software provides an easy means to indicate whether pupils are mastering these Learning Objectives and the system will use this information to print reports on what pupils are learning, rather than simply produce a single symbol or number. This approach has already been mentioned in other articles on this site, and I will say more about it in a forthcoming article.

Rather than use a Tramlines approach to produce a ‘Flight Path’ the CDA system will construct a Progress Graph for each pupil in each subject – i.e. a retrospective visual record of the actual progress that a pupil is making. There cannot be any objection to this as it is simply an example of data-logging, a very common monitoring technique.
We can use a Progress Graph predictively, and this will be based on evidence a pupil’s actual progress. Additional information can be gathered from formative judgements about how well pupils are mastering what they are learning. Standardised tests can also be used to indicate how pupils’ capability compares to that indicative of particular grades at this point in the course.

Progress Graphs rather than Flight Paths
A Progress Graph can display a forward projection of likely future attainment by the end of key stage 4 graphically to show a convergence of several estimates, including the development of mastery, standardised assessments, estimates from KS2, and Attainment 8 estimates.

A Progress Graph is used in the 4Matrix software to show how cumulated data can provide a visual record of retrospective progress through key stages 3 and 4, whilst indicating through extrapolation a likely range of grades achievable by the end of year 11. As we move through KS3 and KS4 the evidence to support the prediction mounts up and has the effect on refining the accuracy of the predicted end-grade.

So when we consider using predictions from a distant baseline as in the use of Flight Paths, how much better it is to use a data-logging approach to recording progress – with the opportunity to make an extrapolation.
The benefits of this approach include:

a. A worthwhile Programme of Study for all subjects, even if the pupil does not continue the subject to GCSE level.
b. The ability to celebrate the end of the KS3 Programme of Study with a certificate showing what was learnt.
c. A good handover to GCSE work by continuation the same approach using Assessment Objectives rather than Learning Objectives.

---

Mike Bostock is an educationalist specialising in the use of performance data tools for school improvement. Mike has worked as a science teacher, DfE IT in Schools Programme (MEP), LA school improvement adviser, inspector and with software development teams. Mike’s company, New Media Learning, produces the 4Matrix school performance system which is used in around 1,200 secondary schools.

---