Why did student scores go down on the most recent assessment?

As the year progresses, student Development Level (DL) scores generally go up from one assessment to the next. This is a reflection of student growth, and it is a rewarding state of affairs for all involved. However, sometimes student DL scores go down from one benchmark or interim assessment to the next. When that happens, school or district administrators are understandably quite concerned, and they sometimes contact us looking for an explanation. There are many things that can contribute to a decrease in DL scores, and no two situations are exactly alike, but a basic understanding of how DL scores are calculated can help to solve the mystery or, at least, to focus the investigation in a more fruitful direction.

Galileo DL scores are scale scores, and in Galileo K-12 Online the DL scores for interim assessments within a given grade and subject are all put on the same scale, so that the scores on different assessments can be compared meaningfully. The way this is accomplished is by relying on the item parameter estimates for the items that appear on the assessments. Item parameters (described in a previous blog) summarize various aspects of the way an item performs when students encounter it on an assessment, including its relative difficulty. When student DL scores for an interim assessment are calculated, the algorithm takes into account both the number of items the student got correct and the relative difficulty of the items on the assessment.

Consider a situation in which two students take two different assessments in 5th grade math. Both students get 65% correct, but by chance Assessment A, which was taken by Student A, was much more difficult than Assessment B, taken by Student B. If you know that there was a difference in the difficulty of the assessments, it doesn’t seem fair that both students should get the same Development Level score based on the score of 65% correct. The algorithm that calculates DL scores takes the relative difficulty of the assessments into account, and in this scenario Student A would end up with a higher DL score than Student B.

Now apply this situation to a series of interim assessments taken by the same set of students. Suppose, for example, the average student raw score was 65% correct on both the first and second interim or benchmark assessments. If the second assessment was more difficult, student DL scores will go up relative to the first. In fact, it’s possible for student DL scores to go up even if their raw scores drop a bit relative to the first assessment if the second one was much more difficult than the first. And, sometimes, student DL scores will go down on a second assessment. This can happen if it was easier, in terms of item parameters, than the first assessment and student raw scores either stayed the same, went down a bit, or only improved slightly but not enough to account for the relative ease of the second assessment.

That brings us to the most important point. If student DL scores drop rather than increase, it is important to identify which skills or concepts the students are having difficulty with and then to do something about it, via re-teaching, intervention, or whatever the best course of action seems to be given the situation and the available resources. There are at least three Galileo K-12 Online reports that are helpful in this regard: the Intervention Alert, the Development Profile, and the Item Analysis report, especially when run with the ‘detailed analysis’ display. All of these indicate student performance on specific skills and therefore provide a starting place for improving student performance.