On March 9th President Obama made a speech about the vision of his administration for education. The speech included calls for some controversial things. One of the hottest topics was the idea of rewarding more effective teachers with extra pay. Conversations around this topic quite rightly raise questions. What measures are fair to determine which teachers are the most effective? How do we account for the fact that teachers don’t get a randomly selected group of students? What kinds of stats are the most fair to evaluate the results?
All of these questions warrant careful consideration. In that light, a quick look at what we already know about some of the issues involved in answering these questions is in order.
The first issue that should be considered is the objective research data which speaks to whether there is a teacher effect on student achievement that can be quantified. In short, the research shows that an effect for teachers can be demonstrated and that the effect lasts beyond the time the student is in that teacher’s class. Some findings have shown that the impact of having an effective teacher can still be measurable 3-4 years later. This finding would suggest that students who are assigned to a particularly effective teacher for several years in a row will likely be far ahead of a student who hasn’t been assigned to equally effective instructors. Interestingly, the links between teacher variables such as credentialing have been at best weakly associated with achievement. This research is nicely, and thoroughly, summarized in a monograph prepared by the RAND Corporation.
While the research world has pretty consistently shown that a teacher effect on student outcomes can be measured, it gets a whole lot more complicated when you dive into the specifics. One of the first nitty gritty questions that must be considered is how exactly should a teacher effect be quantified? Many researchers have employed Value Added Modeling (VAM) to address this question. In short, VAM asks the question of how much variance in a student measure of achievement can be attributed to the teacher. Estimates of teacher impact obtained by utilizing VAM can be influenced by a number of variables including the impact of the way in which teacher variables and other possible confounds are modeled, the measure of student achievement that is selected, and the impact of missing data. The RAND monograph provides a very useful summary of the impact of some of these issues and their impact on conclusions that might be drawn.
All of these questions might lead one to the conclusion that developing a VAM based approach to measuring teacher effectiveness is too complicated to be able to pull off effectively. Such a conclusion would be misplaced. Many papers on the topic of VAM have consistently shown that the data provided by this approach can be useful in answering the types of questions that would face policy makers charged with delivering on President Obama’s call to design a system that can be used to reward teachers who are effective. While VAM is certainly useful, we would suggest that several things happen if it is to be employed for this type of work. The first is that skilled researchers who are familiar with the types of issues that impact VAM estimates should be involved in the design of the system on which policy makers make their decisions. Second, these same researchers also need to be able to engage in research to further understanding of the impact of various issues on VAM estimates.
Before I sign off for this post I want to raise a different but related issue for consideration. Ill bring it up here as an introduction for a post that will follow shortly
Implicit to the merit pay discussion is the idea that providing such incentives will ultimately elevate the level of instruction and lead to higher student achievement. It is our view that the goal of elevating student achievement should be looked at with all the tools that are at our disposal. Just like any other tool, VAM type analyses have certain strengths and notable weaknesses. One of the most notable shortcomings of VAM is that it can tell us little about what effective instruction looks like. It can’t provide any information about what the more effective teachers do that makes a difference. Other types of approaches that I will talk about in subsequent posts can nicely complement findings from VAM based work by addressing this very issue.
As always we look forward to hearing the thoughts of our readers
No comments:
Post a Comment