Why does “norming” matter for teacher evaluations?

GUEST COLUMN | by Libby Fischer

One of the most critical components of teacher evaluation is often the most overlooked: norming.

From a compliance-based perspective, when teacher evaluations are tied to high stakes outcomes like pay raises, promotions, etc., it is imperative that all evaluators within an organization, whether an entire district or a single school, evaluate teachers with the same expectations and interpretation of the rubric. If one evaluator has a tendency to be generous and another tends to be strict, then the system is completely unfair.

When teacher evaluations are tied to high stakes outcomes like pay raises, promotions, etc., it is imperative that all evaluators within an organization, whether an entire district or a single school, evaluate teachers with the same expectations and interpretation of the rubric.

Thinking about implications for teacher development, evaluations can be great tools to understand teachers’ strengths and weaknesses in order to personalize professional development to individual needs. However, if some evaluators inflate or deflate scores, data could point to the wrong growth areas and true growth needs can go undetected. Using a technology-based system for evaluation can help school leaders catch these abnormalities through real-time reporting more quickly than a paper-and-pencil system can. In any case, whether using an edtech solution, or a traditional model, the key to scoring consistency is through norming.

How do I ensure my evaluators are normed?

In 2014, TNTP open-sourced its Core Teaching Rubric, making it available to schools across the country. With it, they also released a guide on Norming (found here), insisting, “[t]his new approach only works…when the people assessing teacher performance—observers, coaches and other instructional staff—are aligned in their expectations and prepared to use the rubric in a consistent way.”

The guide identifies four best practices for training evaluators to give normed evaluation scores:

1. Set Goals
2. Design Your Practice
3. Evaluate Progress/Mastery
4. Continue to Practice

Set Goals

Making sure everyone is on the same page about the use of the evaluation rubric is critical. According to TNTP, important questions to ask in goal setting include:

• Will it primarily help education leaders identify opportunities for high-impact teacher development?
• Will it be used as an evaluation tool in your school/district?
• If both, how do we need to prepare evaluators to effectively use the tool for dual purposes?

Design Practice

The importance of practice can be summed up with this quote: “It takes more than just a few hours of group discussion or a close read of the rubric to successfully norm observers.” TNTP offers the following best practices:

• Preparation is key. Review the ratings observers have submitted in advance and strategically plan your conversation. Do you notice a particular performance area in which a large number of observers are misaligned? Look for patterns in their rationale that can surface misunderstandings or incorrect interpretations of the rubric. This dissonance should be the meatiest part of your discussion.
• Anticipate the devil’s advocate. While providing a robust, logical rationale grounded in objective evidence is critical to helping observers understand and replicate the master rating, oftentimes explaining the reasons why the rating is NOT one higher or one lower can be just as powerful. Be prepared to talk about the disconfirming as well as the confirming evidence.
• Facilitate, don’t dictate. Norming should be a discussion, not a presentation. Listen closely to observers’ justification for their ratings, especially when they don’t align. This will help you understand the root causes of misunderstanding.
• Look for the root cause. Misalignment happens for two main reasons: observers are collecting insufficient or biased evidence, and/or observers are misinterpreting the performance descriptors or essential question. Listen closely to the evidence and rationale observers share to determine which issue must be addressed.
• Keep the boat on course. Allow your observers to actively engage in and drive the conversation, but don’t hesitate to reinforce the “right” answer. You have master ratings for a reason, so be prepared to fully explain and justify the master rationale.
• Ensure that “right is right.” Push observers to justify their ratings and rationale with objective evidence from the lesson. Avoid assumptions, projections or other assertions unsupported by student or teacher actions.
• Summarize and generalize. The goal is to ensure that observers walk away from a norming conversation with a clear understanding of how to replicate key judgments in novel situations.

Evaluate Progress/Mastery

Create specific criteria for mastery and stick to it. The norming guide details TNTP’s exact process, but here are some keys:

• Some number of norming evaluation scores should be Exact Matches
• Some number of norming evaluation scores should be within one point of an exact match

Continue to Practice

Norming doesn’t end after initial training! Without continued training, evaluators have a tendency to inflate or deflate observation scores over time. TNTP suggests planning multiple “checkpoints” over the course of the year to re-calibrate.

—

Libby Fischer is CEO of Whetstone Education, an online observation and coaching platform for teachers. Libby is proud to be partnering with the country’s foremost thought leaders on instructional development. She was recognized as a Forbes 30 under 30 in Education in January of 2016.