The Negotiated Assessment

The negotiated assessmentI have always been fascinated by the methods and principles of assessment and how it applies to classroom practice. I want to know whether students in my class are improving, whether my instruction is helping (or not helping) students to improve and what I can learn from my colleagues in the next classroom. Being part of various teaching teams across a number of schools I have been struck by an interesting phenomenon that seems to occur reasonably often and, despite seeming innocuous, can actually be


This phenomenon is the negotiation of the items or focus of an assessment after a sequence of teaching and typically goes a little something like this:


We, as a teaching team, might start with a curriculum that we agree to, but often is light on the details of what students are supposed to know and be able to do as a result of our teaching. It might comprise something like “Teaching operations with fractions” with some chapters and exercises from the text as a sample. This is our intended curriculum – the domain of knowledge and skills that we are trying to teach students. Then we head to our classrooms and teach what we understand the curriculum to be – the enacted curriculum. The overlap of the classroom domains in the classroom might look like this: 



It then comes to test time. Someone writes a test, based on their understanding of the curriculum and it gets passed around the group but, because of the lack of overlap in the enacted curriculum in each classroom, the test needs work. We feed in our comments:


“I didn’t teach addition of fractions involving different denominators. Can we get rid of that question?”


“My kids wouldn’t be able to work out the fractional amount of a whole from a worded question.”


“I spent a fair bit of time on ordering mixed numbers – let’s add a question about that.”


With that sort of a conversation what we are doing is negotiating the domain of knowledge and skills the students need to demonstrate on the assessment based on the overlap of the enacted curricula in our classrooms. We are effectively renegotiating the assessment domain so that it looks like this:




This may be a good thing for assessing the ability of our students to demonstrate the skills and knowledge that were commonly taught across the classrooms, but it is a terrible device for assessing the ability of the students in the intended, full domain of the curriculum. It looks like we are consistent in our classrooms (because the scores on the test say so) but they are just an artifice of the negotiation. We have only tested a fraction of what was actually taught in each classroom and it is not a good representation of the domain that we initially agreed to.


In my mind a curriculum should work along these lines:


• As a group of teachers (teaching, say, maths) make some decisions about what the aims and outcomes of our work will be over a specified amount of time. These decisions are informed by national curricula, school curricula and the expertise in the team. In the best cases we are quite specific about what we would expect students to know and be able to do at the end of the sequence of learning.


• Once we have agreed on the goals and outcomes then teach to the goals and outcomes of the curriculum. Although this may look a little different in each classroom the general focus and intent of the learning is defined according to the curriculum we have agreed to before we started teaching. We are clearer about what we are trying to achieve.


• An assessment is constructed which assesses the domain of the intended curriculum.  


If we were to represent the domains of the intended curriculum and the enacted curriculum in each classroom in the case above it might be like this:



I believe there are two ways in which we can approach the negotiated domain problem. The first is to describe our intended curriculum in more detail. The looser the curriculum the more difficult it is for teachers to be consistent with what they are teaching and so the enacted curriculum will be more disparate.


Another way is to change the assessment process. The assessment could be developed before the unit is taught using the intended curriculum as the basis for the assessed domain.
You could also have someone who is not teaching in that team write the assessment. It is quite an interesting test of your intended to curriculum to have someone else, knowledgeable in the subject matter, to write the assessment without input from the teachers in the team. If the intended curriculum is loose it will mean that the assessment that is produced by the external person is less likely to represent the taught curriculum.  However, with a decent curriculum it can be a revealing process.

The First Year – What I have learned about implementing a phonics program (Part 2)

Part 1 of this blog described some the mechanisms of the phonics program, both in classrooms and in meetings, that I think have contributed to the learning of our students. The purpose of this section is to describe some of the things I have learnt about what happens when the team gets together to discuss what has been observed in classrooms over the week.

Harness what you have. The phonics team runs on the principle that the way that we will improve as a teaching team is to identify and adopt the most effective strategies from the members of the team. As a team we are always looking to improve our instruction and are on the lookout for better ways of teaching the phonics program. However, the first place we need to start, and the place where we will get the best traction, is to learn from the other members of the group. The tricky bit is not sharing practice, because that happens everywhere. It is sharing effective practice. To do this you need some of the conditions mentioned in the previous post. Some common way of identifying where students are making more progress in class compared to another (positive deviance), some sort of standard to compare that to and then a willingness to discuss and share. 

This approach is also useful because everyone has something to share. It is rare that one individual in the team will have all the answers – more often different people in the team will be able to contribute some effective practice. This helps build the concept of the team and the self-efficacy of its members, rather than an expert lecturing from the stage.
Grain size is important. When discussing practice, the closer the discussion gets to the specifics of what actually happens in the classroom the better. It is particularly easy for a team, when discussing the differences in the learning of students, and the related instruction, to use broad terms to describe what they are doing. “We did /k/” or “we discussed the GPC a couple of times” is really common. It is like when someone starts talking about a generalised term such as “guided reading”. Every person in the group has their own idea of what “guided reading” looks like and the assumption is that all of the other members of the group have the same idea. The broader the discussion the more room there is for interpretation and miscommunication. 

However, when you start to really break down what actually happened in the classroom differences quickly become apparent. Asking people in the group to show you, rather than describe, what they did (even if they have to play act the events) is really useful. It is when members of the group see this level of detail that the differences emerge – “I didn’t do that” or “ok, I just got the kids to start without that intro”. A good guide is that if you think the grain size is right, go a little smaller!
Independent eyes are useful. The school at which I work heavily resources the Prep literacy program. There is an assistant in each classroom for the literacy component of the day who rotates from classroom to classroom. One of the unintended consequences of having an extra person in the room is that they have the ability to see the instruction across a variety of classrooms. These sets of independent eyes tend to help the group identify the differences in the instruction between classroom when the teachers themselves are stuck. Many times, when two teachers were furiously agreeing that their instruction was the same, one of the assistants would point out that there was actually a big difference in what was happening in the classroom: “Jim, you spend less time working through the GPCs from the previous lesson. You are doing similar things but the amount of time you spend on them is quite different. Judy, your class wrote those letters on paper with lines rather than the mini-whiteboards”. These differences that are identified may or may not be causes of the variation in the learning of the students; however, being able to discuss these differences is an important part of the process.

It’s not you, it is your instruction on this concept. Sometimes it is difficult to separate the practice from the person. I am not sure if this comes with the profession but this conflation of instruction and self does seem to occur more with teachers than in other professions. For the discussions to go well it is important the team know and understand that it is the teaching strategy that is being discussed. This is easily changed. Nothing reinforces the concept that it is the instruction that is important than someone trying something new and finding out it works. And they didn’t have to sacrifice a skerrick of themselves.

Interlude – Asterix, Puns and General Knowledge

Over the most recent school holidays I had the fortune to spend time with my eldest nephew. He is a bright boy with a voracious appetite for reading. He has always read well – he figured out the alphabetic principle early and has adopted generalisations and exceptions well enough for him to access a great variety of teenage texts despite being nine years of age. 

As all relatives with only a very partial responsibility for a young child feels compelled to do, I arranged to provide him with a life-enhancing gift. I introduced him to one of my teenage loves: Asterix comics, with all the zany and pun-riddled text it contained. I have vivid memories of giggling away at the various gags at the Romans’ expense, the punny names (my sides still hurt from the first day I learnt the meaning of cacophony) and the random acts of gluttony and violence. I handed my nephew a copy of one of the Asterix books, sat on the couch next to him and waited for him to bring the laugh.
“I tell you, this book is fantastic. Funny, awesome characters. You’ll love it!”
My nephew was dazzled by the illustrations and tucked in to the character bios at the inside the front cover. He was devouring the book. I was still surprised when, after a minute of two, he lifted his head and asked about what one of the characters was saying, a pun of some kind. A minute later he asked the about another joke. His chuckles were reserved for some fight scenes and a boat that kept sinking. Soon he stopped asking. Something was very wrong with the world
He finished the book, carefully closed the last page and sat in contemplation for a second.
“I thought you said that would be funny.”

“It is funny – look, that guy’s name is Cacofonix, and he has a terrible singing voice! And hey, there are the British people and they even stopped the battle to have tea and biscuits. Hilarious!”
He was not convinced.
So, I picked up the book and started to leaf my way through it. When I got to the first couple of puns I tried to explain them to him. Nothing brings home the realities of how much contextual knowledge and understanding of the world is required for Asterisk than trying to explain a pun to a nine year-old. In fact, if you remove the puns and in-jokes from an Asterisk book you are pretty much left with a basic plot with three elements providing the laughs:

  • Characters slurring and acting stupidly while drunk.
  • Obelisk eating massive amounts of boar.
  • Gauls inflicting random act of violence on Roman soldiers.

What it brought home to me is that a good vocabulary and an understanding of the world has a strong influence on your understanding of a text. My nephew can read most words and knows at least one meaning for them. However, being just nine, he is still learning about history and geography, multiple meaning of words and how language can be used to entertain. All things that will come in time but without which a fantastic read is rendered to quite a patchy bit of comic relief. We need to teach students new vocabulary and homophonic meanings, important aspects of history and world culture. 
If for nothing else then the puns. Given I am a father and my entire catalogue of jokes are entirely pun-based, I think, for puns’ sake, a knowledge curriculum is vital.

The First Year – Some things I have learned about implementing a phonics program (Part 1)

A little over a year ago I had the fortune of starting to work with a group of Prep teachers teaching 5 year olds. When I began this blog it was my intention to share some of the things I was learning along the way and perhaps gain some advice. Somewhere along the way I got a little lost but I thought I would take the time to reflect on what I have learnt so far. 
As a secondary teacher with some familiarity with the alphabetic code and a little expertise in helping teams use data to inform instruction I thought I could help the group with the phonics program they were running. I didn’t exactly know what I was getting into – I worked hard to get some understanding of what a systematic phonics program would look like and asked as many questions of experts as I could.

The team was excellent – although the teachers were very young in their careers, they had the desire to improve their instruction and a lack of defensiveness when we discussed what was working and what was not. We had the benefit of some experienced staff who worked alongside us so the mix seemed perfect.

I have split the post into two parts: Part 1 deals with the mechanisms of the program, and the work of the teachers within it, while Part 2 deals the aspects of what makes our team discussions about the progress of kids more effective.

This is rocket science. I know I have shared this before but it really is not a simple thing to help students decode and encode the English language. Even the start of the process was difficult! We had to consciously learn what movements our mouths make with particular sounds, watch for particular movements in students and help them see those movements in mirrors. To then apply letter formation, pencil grip and posture to the learning of the alphabetic code – it required a significant level of thinking, expertise, planning and patience. Our team needed to spend a long time reading, learning and practising before we go to the classroom. Teaching Prep students to decode is not an easy task.

Sticking to the plan is harder than it looks. The Prep decoding program at my school is incredibly well resourced. Staff are keen and hard-working, meeting time is devoted to discussing and developing instruction, teaching assistants are present in each of the classrooms and we have a plan for learning. Despite all of this, fidelity to our plan was a difficult thing “to achieve on a consistent basis. On reflection there seems to be a number of factors that contribute to going a little “off reservation”. In order for the team to stick to the plan we need to:

  •  Develop a shared understanding, particularly early, of what some of the terms used so frequently actually mean and look like. It is so easy to paint our own mental image of what an activity looks like over the description or name someone else is using to describe their practice.
  • Understand that routine is important and careful thought about when and why we do something different is required. The routines serve the kids well and so we need to maintain them even when we might feel like we are going a bit batty!
  • Develop our understanding around what each activity we use with the kids is meant to achieve and what in the activity helps us achieve that. I wrote about activity substitution last year – it is so easy to start substituting one activity for another because the surface features are the same yet the deep structure may actually be very different. This has a remarkably large impact on student learning.

Teach a variety of children but in more homogenous groups. In our model teachers in the Prep group teach a particular phase of the alphabetic code. For ease of use we adopted the Letters and Sounds Phase model. The students are grouped together at the Phase most appropriate for them. The benefits of this arrangement are threefold: teachers gain a better understanding of the teaching of particular concepts as they repeat the teaching several times with different students; each Prep teacher will, at some point, teach each of the students in the year level, which allows the team to take responsibility for each student’s progress; and students have instruction which is quite narrowly focused on what they need to learn at a particular time.

Tracking progress over time is important.
A key plank in our work is the use and discussion of data that relates to our instruction. It is important to meet each week to examine the progress students are making and the difficulties teachers are facing to see if the team can produce a solution. Each term the team tests the students to get a measure of what they know and are able to do in terms of their decoding, including using the UK Phonics Screening Check. This comparative information allows examination of our instruction. 

  • Why is it that, in one class, students pick up particular GPCs faster than in others? 
  • What is different in terms of the instruction in that class compared to the others and how can we share that instruction? 

Also, it gives us a sense of whether our kids are making the progress we want them too – compared to last year’s cohorts, the UK sample, the assessment last term. Particularly with inexperienced staff, the use of a formal assessment to help begin to link instruction to student learning and then discuss differences in instruction to help improve is a key component of a team that learns together.

Part 2 will expand a little more on what we learnt from how to use the information we had to improve our instruction and how the members of the team worked together.

Why ghoti gives me the tiotce

I am usually a reasonably measured person but I have just about reached my limit. Four times in a little over a week I have encountered somebody using ghoti as an argument against using an aspect of systematic phonics instruction. For those unfamiliar with the term often misattributed to George Bernard Shaw, the expression ghoti (pronounced like ‘fish’) is supposed to be a clever wordplay using a number of the grapheme-phoneme correspondences found in the English language. In this case ‘gh’ as /f/ from enough, ‘o’ as /i/ from women and ‘ti’ as /sh/ from notion.

I even read in a recent paper the use of phtheighchound (not entirely tongue-in-cheek) to be pronounced as taken. Take it as a challenge to work out the what words the GPCs in that nonword are taken from.

What gives me the tiotce is that each of these examples involves clear ‘illegal’ applications of the GPCs: ‘gh’ is never /f/ at the beginning of the word and ‘ti’ is never /sh/ at the end. Yet this doesn’t stop the seemingly incessant delivery of this ‘word’ as a key component of an argument against use of a systematic phonics program or using decoding as the primary reading strategy. It is infuriating because the ignorance of such ‘illegalities’ would never be accepted in other areas like mathematics. I have never seen someone argue that, although 1 + 1 = 2, you could rearrange the components of the expression to give 1 1 + = 2 and so that rearrangement somehow invalidates the principles that underpin the equation.

We know that English has a deep orthography and it is not a transparent alphabet but that is not reason enough to ignore that regularities exist and that systematic instruction in phonics is a key component in teaching kids to read.

So, I will not be teaching my kids to ghoti but I will be teaching them to read.

Balkanism – decoding separate from reading comprehension?

A few months ago I wrote a post describing my first encounter with phonics instruction and some of the early lessons I learned as I tried to understand more about its principles and approaches.  In this post I am trying to work my way through a couple of observations that are troubling me a bit; I don’t quite know what they mean.   

In Australia, most schools would indicate that they include phonics in their early years reading programs. This inclusion can range from homeopathic doses through to a program using the understanding of the alphabetic code as its guiding principle. Despite the range of incorporation, phonics as an add-on or back-up seems to be more of the rule than the exception – the AITSL website includes a demonstration video showing a school which uses phonics as a secondary strategy.


I understand that decoding doesn’t fully equate to reading and that there also needs to be focus on comprehension. Given the work of work of McGuinness and others indicating that most of the variance in reading comprehension can be explained by decoding and oral comprehension skills, decoding is an integral part of the reading process.  Therefore the development of an understanding of the alphabetical principle should underpin the work we do with early readers.


On Saturday I was lucky enough to attend a symposium presented by Learning Difficulties Australia, in which Pr Julian Elliott presented some of his research and thoughts on dyslexia, its diagnosis and the implications of diagnosis in a range of settings. There was disagreement and debate around a number of the opinions presented by the panellists which helped to engage the audience. Even those who were put off by the frank views of the panellists on Reading Recovery and Arrowsmith (and there were a few!) would still have walked away learning more about reading and the factors affecting decoding. As an added bonus I had the opportunity to speak to a number of teachers from various schools across the state about the ways in which they approached early years reading in their school. 


In two separate conversations an odd approach was described to me: the schools in which the teachers worked had separate times in the day for the “Phonics Program” and the “Reading Program”. What was gobsmacking for me was that the principles of the two never overlapped. The approach described by the teachers goes a little like this:


In the morning kids come in and work on the grapheme-phoneme correspondence of the day, practise encoding using the graphemes they have been working on and read decodable texts. Then, later in the day, they head off to the “Reading Program” where they use “authentic” texts and multi-cueing strategies. In the “reading” session students aren’t expected to use any of the decoding strategies they learnt in the morning session to makes sense of the text they are reading. They wouldn’t point to a grapheme in a book they are listening to and enquire about the associated phoneme. They wouldn’t try to decode a sentence that contained words that the students were able to decode.


What has me a little confused is that the teachers talking to me were enthusiastic advocates of phonics approaches. They not only demonstrated that they have more than a passing familiarity with various phonics teaching strategies but they celebrate the progress kids have been making in decoding non-words. They were not averse to phonics nor terrible at their application within the prescribed session. Yet they seemed comfortable with the idea that the elements of the alphabetic principle they taught in one session were not applied at all in a session referred to as “Reading”.


Meeting representatives from schools running this way at the symposium may just have been fortuitous but it matched some similar observations I have made when visiting other schools. It makes me think that there is a cluster of schools that have programs that run like this.


So now I am left musing. Why is it, when you have an obvious support for the development of the alphabetic principle, that “decoding” and “reading comprehension” gets Balkanised like this? Pushing aside the methods used to teach reading comprehension for a moment, surely it makes sense to embed the use of the alphabetic principle whenever using text. What I am wondering about is: What makes this balkanisation occur? 

I am interested in your thoughts.

Why we use the Phonics Screening Check

Students at the school at which I work learn to decode systematically and explicitly. We believe that, given the balance of evidence, a good grounding in phonics, taught systematically, will provide them with the best opportunity to improve their reading comprehension. A key part of our teaching strategy is using assessment evidence to pinpoint what a student can decode and what they still need to work on. 

As an Australian school we don’t have access to an Australian national or state-wide assessment for decoding skills or early reading comprehension. In the absence of such an assessment we have decided to use the UK Phonics Screening Check to help inform our instruction. We use the Phonics Screening Check because:

It provides a standard

One of the most common questions students, staff and parents have is whether a child is “doing ok” – are they at the standard for their age? The Phonics Screening Check gives us a standard that we can measure between year levels and across years. We know that students are at standard for their age when they can pass the Phonics Screening Check. It gives a definitive anchor for our work and helps guide what we do. Instead of having an individual feel for what an appropriate level of decoding might be, we have an agreed standard. This aids conversation: we all know exactly what it means to say a student is above or below that standard and we know what instruction and learning is required to get them there. We are able to detect much earlier when a student is in danger of not making the required level and can intervene earlier and with more of a sense of what is required.  

Another feature of the earlier Screening Checks that is useful is the published item difficulties for each of the words and non-words in the 2012 and 2013 pilots. This gives a good indication of what words or non-words were more difficult than others for the UK students. We can then compare that to how difficult our students found those items and investigate when differences arise. What items are we comparatively strong at? Are there aspects of our instruction around the use of that grapheme that we need to record and make sure we are all include in our practice? 

There may be other words/non-words our students unexpectedly find difficult to decode. Why can’t our students decode the word? What part of the word is proving to be the stumbling block? What do we currently do to teach the decoding of that grapheme and why is not working? What parts of our instruction need to be revised in order for students to improve?

It builds a bridge between classrooms

In our school the Prep (5 year old) classes are fluid – the groups are altered every six weeks and teachers change between classes. This results in a shared responsibility for the progress of all students in Prep. Fantastic conversations are had between teachers as they realise that kids who have been in one class are much better at something than students who have been in another class. It might be as simple as noticing children from class A always construct sentences with a capital letter at the start and a full stop at the end, something that doesn’t happen in class B. What is happening in this class that allows students to do this consistently and how can I teach my kids to do the same?

Sometimes, though, the differences in student learning between classes are not so obvious and it takes a specific assessment to reveal them. On the UK Phonics Screening Check there are times when students who are notionally in an earlier phase of their phonics work that have greater success in decoding a non-word than a class that should have done better. Why did that happen? What instruction around that grapheme phoneme correspondence in that class was so effective and how is best implemented in the other classes? How can we learn from each other in order to improve the instruction for all students?

The sharing of demonstrably effective practice results in teaching that is more successful. It doesn’t necessarily mean that all classes are exactly the same but it does allow the gap in effectiveness of instruction to be decreased. This is an equity issue: a student’s progress should not be based on a lottery depending on whether or not they get an effective teacher when classes are allocated. When the instructional quality of the team is growing, both as a whole and as individuals, all students benefit. 

The Phonics Screening Check is an important component of the process of instructional improvement and allows a sense of what an appropriate level of decoding looks like. As such, I would heartily recommend it to all schools teaching phonics.