ATI Releases Research Brief: Categorical Growth Analysis – A Method Involving the Repeated-Measure T-Test

ATI releases a research brief summarizing current research on Categorical Growth Analysis, a statistical method involving the repeated-measure t-test written by Sarah Callahan, Ph.D., Research Scientist with ATI.

As part of the growing implementation of instructional effectiveness initiatives, districts and schools have expressed a need for information about the growth of students for which each teacher and principal is responsible. To be useful, student growth must be estimated based on valid, reliable student assessments and categorized via robust statistical approaches. Via the Galileo® K-12 Online Instructional Improvement and Instructional Effectiveness System (IIIES), ATI provides districts/schools with a wide variety of valid, reliable assessments including instructional effectiveness (IE) pretests and posttests as well as integrated statistical approaches designed to provide districts/schools with precise information about student growth.

This research brief describes and illustrates the underlying method for one of the statistical approaches developed by ATI to provide information about student growth, Categorical Growth Analysis. Categorical Growth Analysis enables educators to evaluate growth throughout the year. Information about student growth can be accessed along with information about student achievement in Galileo reports and used to guide professional development, student intervention, and other activities. Categorical Growth Analysis supports a fair, constructive approach to educator evaluation that evaluates educators against a defensible standard while providing every educator with the opportunity to succeed. This brief also summarizes the results of a simulation study evaluating the appropriateness of the method underlying Categorical Growth Analysis for a wide range of possible datasets.

Read the full Research Brief.

Galileo K-12 Online Categorical Growth Summary Report Overview

How are your schools preparing to show data on expected student growth per teacher? You can now review expected growth rate data within Galileo K-12 Online by running the new Categorical Growth Summary Report. An educator can utilize this data to monitor their students’ growth between two assessments and an administrator can utilize this data for an educator’s evaluation.

How does Categorical Growth Analysis work?
The Categorical Growth Analysis (CGA) model provides an objective evaluation of the magnitude of student growth. ATI uses the CGA approach to evaluate student growth from the pretest to the posttest that can be applied even to individual classrooms, regardless of district/charter school size. This is how the approach works:

• Cut scores for the pretest and posttest are established. The cut scores on the posttest are raised relative to those on the pretest by the amount of progress that is expected over the course of the year. The posttest cut score is based on the average expected growth across a large number of districts.
• Classify students at Time 1 (the first test/pretest) with regard to whether they demonstrated mastery of the state standards.
• At Time 2 (the second test/posttest), track whether each student's mastery classification stays the same as it was at Time 1, or changes.
• Focus on the students who changed mastery status from Time 1 to Time 2.
• The hope is that more students will have moved from non-mastery to mastery than from mastery to non-mastery.
• Conduct a chi-square analysis to determine if the number of students moving in one direction is significantly greater than the number of students moving in the other direction.

To evaluate the educators’ effectiveness, the chi-square analysis yields the following outcomes:

1. Expected growth is not maintained. This outcome is supported in those instances in which there is a significant decrease in the relative proportion of students achieving standards mastery.
2. Expected growth is maintained. This outcome is supported in those instances in which there is no significant change.
3. Expected growth is exceeded. This outcome is supported in those instances in which there is a significant increase in the relative proportion of students achieving standards mastery.

If you have any questions or would like to learn more, please contact your Field Services Coordinator at 1.877.442.5453 or email GalileoInfo@ati-online.com.