Unintended consequences

Making changes in any walk of life often leads to unintended consequences.  This seems to be especially true in education, particularly for the party in power. Recent governments have had one main lever with which to change school behaviour: accountability measures a.k.a. League Tables

For example, take the laudable ambition that as many young people as possible achieve a grade C at GCSE English and Maths to support them in their future careers.  What that has meant in reality is schools have focused their attention preferentially on English and Maths for the D/C borderline students.  No point focusing on those working at a grade or two below D (they might not reach that C), and no need to push those at C or above (they've already reached the required level).

Or how about coursework controlled assessment?  Real life rarely takes place under exam conditions, so what a good idea to ask students to produce work in more realistic situations, and allow them to draft their work (because that's what they'll have to do if they produce reports at work).  Sadly, what happens is that students draft, redraft, re-redraft and so on until they have produced a piece of work that is the best possible grade it might be.  Which leads to controlled assessment which doesn't differentiate between students at all, and which skews the mark balance of the overall qualification.

So many apparently good ideas in education have these unintended consequences because nowadays, not doing well in those League Tables for even one year can lead to a summary dismissal of Headteacher, governing body and academisation in a short space of time.  Even though Ofqual quite helpfully point out (pdf) that even if everything is kept the same there will be natural variation in the results of a school from year to year.

Which brings me to Progress 8, Triple Science, and A-level Physics.

Progress 8 is, I think, an excellent attempt to encourage schools to focus on supporting all young people.  At its heart progress 8 will rewards schools for every grade improvement that their students make; from F to E or from A to A* (of the appropriate numerical grades).  However, the potentially unintended consequence is that there is little benefit to a school, of allowing students to take more than 8 GCSEs, no matter how many GCSEs students take, only the best 8 will count for accountability purposes.  Actually, that's not quite true, a student must sit English Literature and English Language if the best English grade is to be double weighted. So what do I think are the likely unintended consequences of progress 8 for Science*?

1. The number of students taking three separate sciences from 2017 will decrease.  When a student has only got a choice of 6 subjects, using three of them for separate sciences could be seen to be too narrowing - especially as A-levels teachers can't assume that students have all taken the separate sciences, so must use double science as their starting point**.
2. The number of students taking Physics at A-level will decrease as the number of students taking GCSE Physics also decreases.  This will be exacerbated by more schools allowing students to take only 3 A-levels, and given the gender imbalance in the number of students taking AS into A2 physics, likely to lead to a decrease in the number of girls taking A-level Physics. 

I hope I'm wrong, but I suspect that I'm not going to be.

 
*I also think that there will be a large rise in the number of U's in English Literature (or possibly English Language given the greater focus on SPAG in the new GCSEs).  Students don't have to pass both qualifications for the double weighting, just enter them so there could be the temptation for schools to enter students, but not teach the qualification.

** An interesting piece of analysis from the National Pupil Database would be to see how many students studying A-level sciences did separate sciences and how many did Core and Additional science.

UPDATE: Thanks to Frances Wilson (@fflwilson) who directed me to research carried out by Cambridge Assessment in 2013 (pdf pg 15) which showed that around 46% of students taking triple science at GCSE went on to study A-level sciences compared with 13% of students taking core+additional.

Helen Rogerson @HRogerson pointed out that the year 13 who have just received their A-levels were in year 9 when the Ebac was announced. Some schools altered their curriculum offer following this.  This could explain the slight reduction in the number of entrants for A-level Chem, Phys and Bio seen in the results this year.  Yet more unintended consequences.

Comparing A-level Physics 2015 Update

UPDATE Feb 2015:

Ned Prideaux has helpfully updated the spreadsheet using the accredited specs.
He has also included boundary data (from the 2014 series) from Ronan McDonald which may be of interest as well.

The document is online as a google doc and is best downloaded into Excel for viewing.


The exam boards have published their draft specifications for the A-level sciences - albeit with huge caveats about them not yet being accredited by Ofqual.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

@Bio_Joe has helpfully put together a comparison for the A-level biology specs, so I've used his basic structure and done the same for the Physics.

AQA, Edexcel and WJEC Eduqas have produced one specification each - although Edexcel still have the spec repeated so that you can teach by concept or context (though the exam is the same however you teach it).  AQA haven't included a context driven approach in their spec, but will be providing a scheme of work to show how you could do this using the published specification.  OCR has produced two specs, including Advancing Physics, whch are examined using different exams.

Co-teaching.
Most of the boards have arranged topics so that the AS and the first few A-level topics are the same.  Interestingly, WJEC Eduqas hasn't, and the topics are in slightly different orders for each year.  I don't think that will be a bit issue, but it will need a bit of thinking about when planning the teaching.

Multiple choice.
Both AQA and Pearson Edexcel have included some multiple choice questions in the assessment model.

Practical work.
All the boards have, as required, specified practical work that must be carried out for the practical endorsement. There is some overlap in the practicals (for example g by freefall and Young (or Young's) modulus appear in all the specs.  AQA and OCR have stopped at the minimum of 12 practical.  This includes 'Research skills' for OCR and allows students to explore a physics topic they are interested in via books and 'tinternet.Edexcel have included 16 practicals as their minimum. 

WJEC Eduqas have specified far more practicals than the other boards (including for example investigation of radioactive decay via a dice analogy or determination of h using LEDs) which would provide a really nice skeleton of a practical teaching scheme.  To be honest, even if you don't choose this spec, it's worth having a look at the practical work they suggest to help you plan as there are some good straightforward ideas in there.

It's quite a big document so I've resorted to a google spreadsheet.  Please feel free to use, with acknowledgement.

What practical assessment?

It is a truth universally acknowledged that a high stakes accountability system will tend to prevent accurate teacher assessment.  Or at least, that is what Ofqual and the exam boards would like us believe.

Having talked to many teachers about controlled assessment (and having been involved in supervising coursework) I think that actually Ofqual and the exams boards are probably correct.  Current controlled assessment measures very little of a student's ability to skillfully carry out practical work, and is overly focussed on one (or at most two) practical experiences.  The time taken to ensure that the whole class completes the appropriate work, with the appropriate supervision eats into the time that could be used to teach content, or do other practical work.  (And involves numerous catch-up sessions after school and during holidays if my child's experience of year 11 science coursework is anything to go by). 

So Ofqual have, at one fell swoop, removed teacher assessed practical work from A-levels from 2015.  Instead, students will be expected to develop 12 practical skill sets through carrying out a minimum of 12 named practicals.  The skills are the same for all exam boards, but they are free to choose their own practicals.  The skills are given in appendix 5c of the subject criteria, but until the draft specifications are published (towards the end of June) we won't know what the practicals are.

Helen Rogerson published a blog outlining her view on the loss of practical assessment and I think that she makes a very good point. 

But...

What if the removal of examined practical assessment means that teachers will stop doing practical work?  If we don't assess practical work, then even though teachers think that it is educationally desirable, they might stop doing it.  Senior leaders might decide that if practical work isn't assessed then why spend large part amounts of money on science equipment and consumables?

This appears to be the view of the Wellcome trust, various university academics and SCORE.  

Again, I can see why they are worried.  It may be that practical science becomes limited to the 12 named practicals in the specs - much like the teaching of English literature appears to be limited to the books named in the new GCSE specs.  Teachers will plan for, and teach, only the practicals that they have to provide evidence of the students carrying out.  On the other hand, anecdotal evidence from teachers who have switched from GCSE sciences to iGCSE sciences have said that the removal of controlled assessment work has led to an increase in the amount of practical work that they do.

Before we can start to discuss the effect of the changes to practical work it would be helpful to know what practical work A-level students currently do.  Michael Gove and Ofqual have referred to reports of Universities complaining that A-levels don’t prepare students well enough for university.  It may be that, in some schools, practical work has already reduced to only that required to complete the controlled assessment part of the course.

In schools like that, it may be that having to do 12 practicals over the space of 2 years might actually be an improvement.

Without data though, we just don’t know.

To that end, I am intending to carry out some preliminary research into what practical work in science (and particularly physics) is currently planned and carried out at A-level in a number of local educational establishments.  I’m hoping that I will be able to revisit the schools as the changes to A-levels are made and see how they are responding to the new structure.

Watch this space!

The future of assessment?

Levels are ‘over-complicated, vague and unambitious’. So the government have scrapped them. 

Unfortunately, up until now, they didn’t really provide any information about what was going to replace them.  And in the absence of that information, many schools have decided to stick with levels for the time being.

But now we have the information.  Or rather, now we have brief descriptions of the models from nine schools that have been chosen to receive money from the assessment innovation fund.  Each of the schools will get up to £10,000 to turn their assessment models into packages which other schools can use.

Those schools chosen are 2 primary, 1 primary/secondary, 4 secondary and 2 special schools. 

When I first read through the descriptions of the assessment packages, I must confess to (a) being a little disappointed and (b) wishing that I had applied with the model that a colleague and I are developing for use in primary.  Many of the packages sound very much like a combination of formative assessment (including self-assessment booklets) and end-of-topic tests.  A couple focus on ‘skills’ and abilities (primary and special school) while others focus on curriculum content or topics (secondary). 

All very reasonable ways of identifying the progress a pupil has made, and reporting it to others but nothing really innovative.

The DfE have also produced a set of core assessment principles.  This states that:

Schools will be expected to demonstrate (with evidence) their assessment of pupils’ progress, to keep parents informed, to enable governors to make judgements about the school’s effectiveness, and to inform Ofsted inspections.

 

So, what can we determine about acceptable assessment models from the short descriptions given in the DfE press release?  

The models include:

·         skills passports

·         ladders of curriculum content to climb

·         formative assessment which allows students to improve and develop their understanding

·         students assigned to levels of mastery / understanding

·         end of topic tests to provide numerical data

Looking at the ideas described, I think that we can probably summarise the models as:

Primary and Special schools: 

Can the kids do stuff? Can they do more stuff than last half-term?

Secondary Schools

Do the kids know stuff? Do the kids know more stuff than they did last half-term? 

So what should you do when replacing levels?  The concept of backward planning (as ably demonstrated by York Science for KS3) seems to be a good starting point.  

• First of all, decide what you want pupils to do or know
• Then, decide what evidence would let you know this
• Finally, plan your teaching experiences (lesson plans, schemes of learning etc).

In your teaching experiences include opportunities for formative assessment and summative assessment.  These assessment opportunities should make use of the evidence of learning that you have previously identified.  Think about what an 'average' student should be able to do/know, or a student who isn't quite there, or one who is well ahead.

The models picked from the Assessment Innovation Fund have aspects of these principles included, but they aren't overly complicated.  The models look different, and will be used in different ways*. However, they all allow assessment (formative and summative) tracking and reporting of progress.   

And there you have it . . .assessment without levels.

 *Which will perhaps be comforting when it comes to explaining your own assessment model to Ofsted.  Though it might make things much harder for Ofsted when it comes to looking at data.

....................................................

More details about each school assessment model are:

Hillyfield Primary Academy, London (primary)	Skills passport in foundation subjects throughout KS1 and KS2.  On demonstrating mastery of a skill, children stamp the skill in the passport. Longer term, school will develop an app that can be used in class by teachers
Hiltingbury Junior school, Hampshire (primary)	‘ladder’ approach to maths, reading and writing. School agrees expectations for each year group, dividing each subject into key skill areas. Children use a ‘ladder booklet’. Steps are divided into areas of exploring, achieving, exceeding
West Exe technology College, Exeter (primary and secondary)	‘ladder’ approach with objectives driven by curriculum content. Each objective is a short, discrete, qualitative and concrete description of what a student is expected to know and be able to do within a specific subject area and topic. Formative and summative (end of topic) assessments are used. Moderated by experienced professionals to ensure consistency.
Westminster Academy, London (Secondary)	Percentage score system to reflect a student’s mastery.  Curriculum divided into discrete topics (about 15 per year) each assessed by in-class quite, homework and end-of-term exam.  Overall score derived from performance for each topic, with an average produced to cover all topics.
Trinity Academy, Halifax (Secondary)	Curriculum broken down into units (categorised as foundation, elementary, intermediate or advanced).  Students complete a test to ascertain their degree of mastery.  Level of success places them at a point within a category: no progress, expected progress, exceptional progress.
Durrington High School, West Sussex (Secondary)	School used KS2 date and other assessments in year 7 (CATs, reading tests, internal tests) to groups students into 4 thresholds based on their prior assessments: excellence, secure, developing and foundation. Formative feedback provided based on day-to-day work. Summative assessments (half termly/termly) are used to  further assess how well students are doing.
Sirius Academy, Hull (Secondary)	Focussed on D&T only. Not clear what is actually involved in the assessment system.
Swiss Cottage School, London (Special School)	Progression planners. Themed around priority areas for individuals with severe learning difficulties and consistent with principles and ethos of the new NC.
Frank Wise school, Banbury (Special School)	Series of assessments that screen the level of development of basic cognitive skills.