It has happened to me at least fifty times. The data from one assessment or another has been collected, been placed into my Sheet of a Million Cells and then presented back to the team for their consideration. Looking around the room all I can see are blank stares, a half filled lesson plan for tomorrow and a distinct eagerness to get onto the “real work”. What has gone wrong with my beautiful data? It is so useful, so elegant. This team must be full of dullards not to use it to inform us about our kids and our teaching – right?
If we don’t do anything as a result of weighing the pig then the process has no use – in fact, the process is worse than useless because we have wasted time collecting and examining data that could have been used on more useful things. It took me quite a while to realise that a significant proportion of the problem lay in the data and its presentation. In response I changed some of my practice and it seems to have helped us use the information to improve our work.
So, in true Buzzfeed fashion, here are “My Top 5 Reasons Teams Are Not Using The Data”.
This is a big one – if the information from the assessment is not tied quite explicitly to what is happening in the classroom it makes it very difficult for teachers and administrators to do anything with it. If you can’t link some elements of instruction or learning goals to assessment outcomes then it is impossible for a teacher to have any agency over the outcome. Why would I pay attention to data when I can’t see how it relates to what I am doing in my classroom?
Common errors here include using a grain size that is too high (i.e. percentages on a test, a scaled or index score like on external testing – I wrote about this in my previous blog), having questions or assessment products that don’t test what we have been teaching or test things we won’t teach again for some time.
When you are familiar with data the temptation is to include a little bit of everything. We have all sat in meetings where someone enthusiastically works us through Charts 1 to 43, each showing some nuance in the data that they think is important, and then leaves us with the Sheet of a Million Cells to “check over in your own time”. It can be overwhelming for people, especially if you don’t work with numbers that often.
The amount of data used with the team needs to be just large enough to answer the questions we have about student progress and our instruction and no more. Keep paring back the data until you have just the essence of what is required.
3. There is no narrative to the data.
Data needs to be worked with and prepared properly before the team meeting. Raw data is almost impossible to interpret quickly, so it needs to be converted into a format that is easily understood. Having members of the team do some of the investigating into the data is important but it must be set up so that any key messages are easy to interpret. If working through the data is too onerous then people avoid it. And just because you think a chart is beautiful and clever doesn’t mean it is useful…
A pretty graph, but is it useful?
When you are shifting representations of data constantly and quickly it is extremely difficult to focus on the details of what the presentation is showing. Having two different presentations from the external agencies, the chart on Geography that Bill from IT produces and the hand-drawn table from the History department means that a lot of time is spent trying to figure out what the presentation is trying to show, where the scores come from and how they link to the instruction in the classroom. Little time is then spent discussing what the data tells us and what we can do about it.
There needs to be consistency in the way that data is presented. For data to be used most effectively and quickly during precious meeting time the team members need to be able to orientate themselves to what charts and tables are showing them, what is more important and what messages they can glean from each representation. Spending time as a team deciding on the best way to present data, and then presenting it this way consistently from assessment to assessment and department to department makes that orientation process faster and makes team members more confident in their ability to read and interpret the information.
The last point is that, despite having a consistent presentation of a set of data that has been pared back and clearly shows a narrative, if a team hasn’t got system for using that information then it will be difficult to extract meaning. There are many books and theories about using the data, for DataWise to elements of the PLC approach. Anything that has a series of norms and a process for working through data systematically helps provide security and support to a team working with data.