Rating Scales: A New Take

Observation has long been a keystone of teacher evaluation.  The evaluator, usually the principal, comes into the classroom with clipboard in hand, fills out the district approved tool, makes some notes, and ultimately generates a score that is used for purposes such as determining bonuses or guiding professional development.  Recently many states (including Arizona, Massachusetts, and Colorado) have passed legislation mandating structured observation at least twice during the course of the year.

Given the important role of rating scales, it makes sense to look at how they are typically constructed and used.  Many of these instruments that are in use today have been written by a team of experts, pilot tested, and then ultimately adopted.  They exist as a single tool.

We think there is a better more flexible way.  Rather than constructing a single instrument, we are working to develop a bank of items that can be used to construct customized rating scales, much as we provide the capability to construct customized benchmark assessments that provide targeted information on student mastery of the specific standards that have been a focus of instruction.

Approaching the construction of instructional rating scales in much the same way that we construct assessments for student learning provides a number of advantages.  First and foremost is that just as student assessments may be customized, rating scales may be built to reflect the particular interests of a district.  If a new professional development program has been implemented, then a tool can be constructed that reflects the specific skills that were emphasized all without losing the ability to compare scores across time with an instrument that has a somewhat different focus.  The second advantage of the approach is that scores may be produced that can be used to identify those specific teacher actions in the classroom that lead to better results. 

ATI is releasing rating tools that are built using this approach this fall.  We look forward to hearing responses from districts as they implement the new approach.