How does ATI calculate my district’s psychometrics benchmark test data?

Statewide test data will soon become available to the public. As you begin to review your data from varying sources (e.g., district results from state-wide testing and from district specific assessments) you may be interested to learn how ATI is able to process and compare data. To generate ATI’s scaled scores (ATI refers to these as the Developmental Level Scores or DL scores) ATI uses an analysis based in Item Response Theory (IRT). IRT takes information about the difficulty of the items into account when generating the estimates of student ability (i.e., the DL scores).

What this means for you is that a change in DL scores is a direct measure of growth. For example, if a student obtains a DL score of 1000 on Benchmark #1 and 1100 on Benchmark #2, that would mean that the student’s ability increased by 100 points or one standard deviation. In contrast, by looking at raw scores or percent correct, you cannot be sure what it means if scores increased.

For example, if a student obtained 70 percent correct on Benchmark #1 and 80 percent on Benchmark #2, this increase might be related to the fact that the items on Benchmark #2 were easier or the fact that the student’s ability increased, or both. So DL scores are generally preferable to raw scores as a way to evaluate student progress; however, sometimes the results of this approach are not so intuitive. A teacher may see that his or her class improved in terms of percent correct from Benchmark #1 to Benchmark #2, but that their DL scores have actually decreased. In this case, the items on Benchmark #2 were probably easier than those on Benchmark #1, so essentially the students didn’t get as much “credit” for getting them right.

Once ATI has the DL scores, ATI sets the cut scores that correspond to the achievement levels (e.g., Exceeds, Meets, Approaches). ATI uses two approaches to set cut scores. The first approach is called equipercentile equating. Under the equipercentile equating approach, ATI aligns the distribution of student scores on the benchmark assessment to the distribution of scores on the analogous state assessment for the same district (i.e., we align the third grade math benchmark to the most recent third grade state standardized assessment, such as Arizona’s Instrument to Measure Standards [AIMS] in Arizona). We identify the percentile ranks at which students in the district attained the various cut scores on the state standardized assessment, identify the same percentile ranks in the distribution of scores on the benchmark assessment, and the DL scores at those percentile ranks become the cut scores on the benchmark assessment. This allows ATI to identify not only how many students are likely to pass the state standardized assessment at the end of the year, but which students. Analyses of the accuracy of forecasting indicate that the equipercentile equating approach is highly accurate in forecasting which students are likely to show mastery on the state test and which are not.

Learn more by reviewing the Benchmark Assessment in Standards-Based Education research paper.

Experience Galileo for yourself. There are a number of ways to learn first-hand about Galileo K-12 Online. You can:

• Visit the Assessment Technology Incorporated website (ati-online.com)
• Participate in an online demonstration by registering either through the website or by calling 1.877.442.5453 to speak with a Field Services Coordinator
• Visit us at the
• Arizona Department of Education Leading Change Conference June 26  through 28 at the Westin La Paloma in Tucson, Arizona.
• Massachusetts Association of School Superintendents Executive Institute July 11 and 12 at the Mashpee, Massachusetts.
• Arizona Association of School Business Officials 59th Annual Conference and Exposition July 18 through 21 at the JW Marriott Star Pass Resort and Spa.
• Colorado Association of School Executives Conference July 23 through 27 at the Beaver Run Resort, Breckenridge, Colorado.

We look forward to communicating with you online or at events.