A.+Overall+Teacher+Judgement's+(Massey)


 * Moderation and OTJs Workshop **

Contributors: Brian Coles, Anna Stephenson, Anne Radford, Lisa Kaye, Chris Braid, Janine Remnant, Fiona Rossiter, Lote Hensman, Robin Holt.

Abstract This workshop will be in two parts; first several gnarly issues around moderation and OTJs to discuss; second, opportunity to choose a Maths or Reading practical OTJ decision session.


 * Gnarly Issues **
 * // For each of the following issues discuss to reach a consensus recommendation and justify your recommendation with reference to better learning as well as NAG compliance. //**

** 1. What do we mean by ‘at’? **
Does ‘at’ mean ‘standard achieved’ or ‘standard the student is working within’?

// Vigorous debate occurred during both days, the Numeracy people in particular, deeply divided.......... // // What is actually communicated to the student when an assessment tool is used, by the student themselves, in consultation, or by their teacher. // // Consider a tool such as a Written Language matrix or IKAN interview // // If several indicators are highlighted and dated what does that mean? // // It clearly conveys that these indicators describe things the student can now do, not just once, but as an established pattern of behaviour across several contexts. The student may then be encouraged to set goals from among the indicators not yet achieved.// //If the student reports their achievement to parents it will be a description and examples of what they **can do**. Plus new learning goals and how they plan to achieve them.// //Thus, 'at' means 'standard achieved'.// //In our AtoL experience schools that adopt a policy of reporting based on 'at' meaning 'working within' always end up with confusion among staff, students and parents and eventually change their policy to 'at' meaning 'achieved', as this is what students, parents and teachers in practice understand.//
 * // Here is an Assessment for Learning perspective. //**
 * // Start with the student, not the report. //**

//The above highlighted matrix raised another issue about 'best fit' for an OTJ. Consensus about this issue was that the decision about 'best fit should be based on what level or standard best suited the future learning of the student, rather than inventing some arbitrary rules or algorithms for determining 'best fit'.//

**2. Do we assess to locate which standard a student has achieved, or do we choose the expected standard and assess ‘below’, at, above etc?**
Read the Gazette comment by Mary Chamberlain (Page 7, Education Gazette 29 March 2010. Attachment B). Terry Crooks, on National Radio, has also preferred locating the standard. But most advice appears to suggest the second option. NAG 2A requires ‘report to students and their parents on their progress and achievement in relation to National Standards.’ so either option would meet that requirement.

**What is an Assessment for Learning view?**
//If the student is doing their own reporting to their parents which option would we choose; what has been achieved, or performance against a standard? The latter choice could result in a student who has made little progress being able to report 'above' or a student who may have made considerable progress reporting they are 'well below' standard.// //These examples hint that what's really desirable is that extra challenge in the NAG requirement, reporting of progress. Preferably by the student.// //If standard achieved is reported, a student can report their progress to the next standard. If 'well below' is chosen, the student may make considerable progress but still be 'well below' at the next reporting time.// //So an Assessment for Learning approach favours reporting standard achieved.//
 * //Start with the student.//**

3. How much moderation is enough?
a. Alison Gilmore (Page 8, Education Gazette, 20 March, 2010) asks the question and answers with 'give parents and the community confidence that the overall teacher judgements are robust' (Attachment C). But Reeves (Attachment D) from his 90 90 90 schools article (Google 90 90 90) describes moderation processes as essential to school transformation.

**An Assessment for learning Perspective**
//If thinking about assessment for reporting only, then the 'robustness' and parents' confidence is all we need to worry about.// //But how will improvement to teaching and learning actually happen? Here, the use of assessment for diagnosis of individual student strengths and weaknesses and assessment used to enquire into the effect of teaching approaches and strategies are crucial to any improvement in teaching and learning. From this perspective, shared standards, more accurate diagnosis and better quality evidence about progress students have made across classes and across the school, could lead to better teaching and learning.// //Reeves, in his review of evidence about 90 90 90 schools in the U.S. (These are schools that have 90% of students from black or ethnic backgrounds, 90% of students from poor families BUT 90% of students achieving state-mandated standards) describes the differences between these schools and schools of similar composition but lower achievement. Common and shared standards, focus on student work, analysis based on progress, authentic or performance assessments rather than tests, taking account of assessment data to change programmes as the year develops, are all highlighted. From this perspective it seems a school leader wanting to shift her school to 90 90 90 status would put huge emphasis on moderation of OTJs as a staff development activity right through the year.// //Another advantage of this approach is that OTJs would be obvious and very robust by the end of the year when the summary report to the BoT is due. No extra meetings would be required, staff decisions, having been moderated all year, would be trustworthy.//

**The next two issues were not discussed during the workshops, but we include them for your own debates and include an Assessment for Learning perspective for each.**
See the OTJ Factsheet (TKI Download) (Attachment E) Page 1 recommends triangulation. Does this mean each teacher brings at least three pieces of evidence about each student and each set is considered by all participants?
 * 4. ** ** Procedures for making OTJs **

Can you recommend a more effective procedure? // If moderation is engaged in thoroughly early in the year, so teachers are actually using common standards while making everyday assessments about student learning, with students, then teachers will be able to make accurate OTJs about most students at any time. // // In some cases there will be students about whom the evidence is uncertain or conflicting, or their progress is causing concern. These cases are the ones to discuss in detail with assembled evidence. This process helps teachers ensure their standards are not ‘drifting’ and allows teachers to discuss and share theories and practical ideas about students whose learning may be of concern. //
 * An Assessment for Learning Perspective. **

Read the Moderation NS Factsheet (Attachment F) Are there other moderation processes you would recommend to schools? You may choose to focus on the leader’s role (page 3) How do you suggest student be involved in school-wide moderation processes? (page 4 and also see OTJ Factsheet page 2) // Experience has taught us that the first moderation meetings are slow. Everyone needs to get to know the standards and indicators being used and agree that their own judgements won’t do if they cannot justify them with reference to these shared criteria. The multiple purposes of the exercise also need to be understood and shared by all; common standards to be used for teaching and learning in all classes, students’ involvement, diagnostic use of the information to set goals, reliable assessments that can be aggregated for teacher enquiry and school-wide enquiry, and dependable assessments for reporting. // // When teachers do understand the multiple purposes, including the work that is saved by using common standards across the school, they often discuss student work with each other informally, including that of students whose progress is causing concern. These informal discussions are a very valuable process in a school learning community. // // Further assurance of common standards can be achieved by the moderation leader assessing a structured, random sample of student work across the school, but it is the shared discussions and efforts to assess high priority goals to common standards that is the most effective professional development activity for teachers. // // Further assurance will very likely NOT be necessary when teachers get their students involved in very similar moderating processes to those teachers use. Beginning with shared critique of student work from elsewhere, then development of assessment criteria and exemplars, the students can move on to making reliable assessments and provide constructive feedback about work done by peers and eventually their own work. //
 * 5. Moderation Procedures **
 * An Assessment for Learning Perspective **