New 2015-16 ATI Research Briefs Assist Pre-K Programs to Interpret Developmental Level (DL) scores

Galileo Pre-K is a reliable and valid assessment tool that can be used to define, assess, and track learning. ATI applies procedures based in Item Response Theory (IRT) to information gained through observational assessment to estimate a measure of child learning called the DL score. The DL score indicates a child's position on a developmental path and provides information about which capabilities the child has learned and which capabilities the child is ready to learn. Each year, ATI conducts research to evaluate how a child's DL score relates to both a child's age as well as to the time a child is enrolled in a preschool program. The results of the research are summarized in the following two research briefs and can be used to gauge child growth. 

• Estimated Growth for Children Birth through 5 Years Based on 2015-16 Assessment Data: The change in the DL score over time can be used to evaluate child growth. ATI's research may be used to estimate child growth for various time periods. This information can be used by early childhood providers to evaluate whether children are demonstrating adequate growth over time. Read the full brief. See sample excerpt of data below.

• Predicted Developmental Level Scores for Children Birth through 5 Years Based on 2015-16 Assessment Data: ATI conducts annual research that can be used to predict the DL score for children at various ages. Early childhood providers can use the up-to-date information provided by this research to evaluate whether children are progressing appropriately over time relative to their peers. Read the full brief. See sample excerpt of data below.

Access more research briefs by ATI, click here. 

New and Upcoming Enhancements to Galileo

New – Depth of Knowledge Level Filters in Test Builder and Displays in Reports  
ATI has made it easier for districts and charters to develop assessments addressing appropriate levels of cognitive complexity. Galileo Test Builder has been enhanced to support filtering item searches by Depth of Knowledge (DOK) level while developing assessments. Additionally, when items are administered, DOK information may be easily viewed in several reports (i.e. Item Analysis, Intervention Alert, and Test Monitoring Reports). Soon, DOK levels will also be viewable in the Test Review interface for users with appropriate permissions.  

Filter item searches by one, multiple, or all DOK levels. Click to view larger. 

Technology enhanced DOK level 3 item displayed in the drilldown item view of the Intervention Alert Report.  Click to view larger.

Learn more about searching for items in Test Builder by viewing the Quick Reference Guide.

Coming Soon – Real-Time Developmental Level Scores and Statewide Test Forecasting

Galileo will begin automatically generating Developmental Level (DL) scores, performance levels, and risk levels for Arizona students as soon as they have completed district/charter-wide assessments. The DL score is an Item Response Theory (IRT) based measure of student ability. The DL score takes into account the difficulty of test items and can be placed on a common scale across tests to evaluate growth. The DL score is also used to classify students into Risk Levels that forecast likely student performance on the statewide test at the end of the year. Immediate access to DL scores and Risk Levels will provide districts/charters with real-time actionable information to guide next instructional steps. 

For more information on these or other features of Galileo, contact our friendly and knowledgeable Field Services Coordinators. 

Assessment Based on Instruction or the Benchmark Assessment Series?

Should students be assessed based only on instruction provided during the benchmark period or should all grade-level standards be assessed on each benchmark? This question is asked frequently by ATI clients.

ATI recommends that school districts assess students on standards targeted for instruction if:

· There is an established pacing guide. The school district has a relatively large number of students who will be taking each test. The school district has a pacing guide that includes instruction on all state-tested standards.

There are two particularly important benefits associated with the approach of aligning assessments with a pacing calendar: First, students are assessed only on material that they have been recently taught. Second, it is typically possible to include more items on each standard being assessed than would otherwise be the case. This increases the amount of information on student performance available for each standard that is assessed.

In contrast, when a school district has many different pacing guides, there are challenges associated with the attempt to align assessment with what is currently targeted for instruction. Disagreements about which standards should be assessed on which benchmarks tend to surface when teachers are not all on the same page. It is difficult to make decisions about which classrooms and pacing guides should take precedence over the others.

In addition, analyzing data and making predictions on how students will do on a high stakes tests is most effective when large numbers of students take an assessment. There is an adverse effect on the stability of item parameter estimates used in determining item difficulty, discrimination, and guessing. With many different pacing guides, the numbers of students taking each assessment decreases.

ATI offers an alternative for school districts that don’t have a widely used pacing guide in place or who would only have small number of students taking each assessment. The Benchmark Assessment Series assesses all grade level standards thereby providing Development Level information for students no matter what pacing guide their teachers are following, allows a greater number of students to take the same assessment, and will pinpoint areas where large groups of students have not mastered concepts which will be assessed on the state tests.