The fundamental premise behind instructional effectiveness initiatives is that good teaching and effective educational management can produce enhanced student learning. In order to assess instructional effectiveness, it is not only necessary to show that teachers and administrators have demonstrated skill in instruction and educational management, but it is also essential to demonstrate that effective instruction and educational management have resulted in improvements in student learning.
In 2011, Assessment Technology Incorporated (ATI) was asked to explore how we could support local implementation of instructional effectiveness initiatives. ATI undertook the development of an integrated Instructional Effectiveness Assessment System (IEAS). The initial IEAS pilot began in the 2011-2012 school year and involved 37 school districts. The success of the pilot provided a foundation for the creation of IEAS.
The Galileo IEAS is comprised of two integrated components addressing the key areas in assessing instructional effectiveness, student outcomes and educator proficiency. As part of our continuous efforts to provide schools with a fully integrated and standards-aligned instructional improvement system, the Galileo IEAS has been fully incorporated into the Galileo K-12 Online Instructional Improvement System (IIS). Our plan is to continue to work in collaboration with districts and charter schools in expanding the instructional effectiveness tools.
As part of instructional effectiveness, ATI is answering in a number of ways the urgent need for valid, reliable, and fair items in areas not tested on the statewide assessment. For example, ATI Standards Builder makes it possible to enter any set of standards into the Galileo K-12 Online. The standards for non-tested grades and subjects have been entered into the system, including such diverse subjects as music, dance, and physical education. This means that IE assessments can easily be created and aligned to these subjects within Galileo.
Another way in which ATI is addressing the need for reliable and fair items in grades/ subjects not tested on statewide tests is to provide technology to districts and to organizations working on item creation. One partner in this area is WestEd. WestEd is providing a group of districts with item development services including training in the construction and review of items in art, music, and physical education. ATI is providing technology to manage those items including importing them into Galileo, placing the items in banks, retrieving the items for use in assessments, scheduling and delivering assessments for use in schools, conducting psychometric analyses to establish item parameter estimates and to determine the validity and reliability of the assessments, and providing instructional reports of assessment results.
Still another support of IE assessment in non-tested areas is provided through the use of Galileo Automated Scoring Key (ASK) Technology. Galileo ASK Technology enables schools and districts to administer, score (automatically, if a multiple-choice format is used), and report on an assessment that was created outside of Galileo. Such an assessment may be administered either online or with scannable answer sheets.
Read further about the benefits of IE in the February 20 blog post or ATI’s Instructional Effectiveness Assessment System white paper, stressing the critical issues of measuring instructional effectiveness, the importance of reliability and validity in measurement, and an overview of the measurement procedures needed to be in place to effectively implement and help guide local decision-making.
We welcome the opportunity to speak with you and describe how the Galileo IEAS can be incorporated into your local instructional improvement plans. We can be reached at 877.442.5453.
Tuesday, May 29, 2012
Monday, May 21, 2012
The Galileo School Readiness Scale
Research shows that the social, physical, and cognitive environments that a child is exposed to in the first years of life have a lasting impact on how a child develops. The capabilities that children acquire during their formative preschool years have a profound effect on their successful transition to school as well as maintaining success while attending school. One of the fundamental goals of Assessment Technology Incorporated (ATI) is to assist programs in meeting the school readiness needs of children. To help programs target these valued educational goals, ATI has created the Galileo School Readiness Scale.
What is the Galileo School Readiness Scale?
The Galileo School Readiness Scale is a new developmental scale that consists of capabilities from the essential areas of learning and development reflected in the Galileo G3 Assessment Scales. The capabilities on this new scale reflect valued school readiness goals, which programs may use to benchmark the progress their children are making in getting ready for successful entry into kindergarten.
How were the goals on the Galileo School Readiness Scale selected?
In choosing which capabilities to include on the Galileo School Readiness Scale, ATI analyzed the specific goals in the Head Start Child Development and Early Learning Framework, various state early learning standards, and Common Core Kindergarten Standards. Those capabilities that were common among the Framework and standards were identified as the agreed-upon essential performance objectives for preschool-age children transitioning into kindergarten.
Once these standards were identified and analyzed, the developmentally sequenced capabilities that a child must learn to achieve these valued goals were determined and then included in the Galileo School Readiness Scale.
How do teachers use the Galileo School Readiness Scale?
The Galileo School Readiness Scale should be used in conjunction with the Galileo G3 Assessment Scales. Since the Galileo School Readiness Scale is made up of selected Galileo G3 Assessment Scale capabilities, a teacher may continue to assess their children on the Galileo G3 Assessment Scales. Once a program has access to the Galileo School Readiness Scale, the G3 capabilities a child has already learned will automatically be documented as learned on the Galileo School Readiness Scale. In the future, as a teacher documents a child’s learning on the Galileo G3 Assessment Scales, the like-capabilities on the Galileo School Readiness Scale will also be updated as learned, meaning no additional work for the teacher.
How do administrators use the Galileo School Readiness Scale?
Administrators may aggregate and analyze Galileo school readiness assessment data at multiple points in a year. Periodic data analysis will determine what progress children are making toward meeting these established goals. Reports such as the Progress Report will illustrate the increase in children’s developmental score from one observation period to next, indicating that there has been a gain in ability. The Developmental Profile, when aggregated, will spotlight the particular school readiness capabilities that children have been attaining, and which might need additional attention. Administrators may use the Galileo school readiness assessment data to inform continuous program improvements related to curriculum, teaching, instructional strategies, and the professional development of staff. Furthermore, data from this scale provides programs the opportunity to share with parents and the local community children’s readiness for kindergarten.
Interested in learning more?
Contact your Galileo Field Services Coordinator at 1.877.358.7611 if you would like to learn more.
Kerridan Smith, B.A.
Professional Development Director
What is the Galileo School Readiness Scale?
The Galileo School Readiness Scale is a new developmental scale that consists of capabilities from the essential areas of learning and development reflected in the Galileo G3 Assessment Scales. The capabilities on this new scale reflect valued school readiness goals, which programs may use to benchmark the progress their children are making in getting ready for successful entry into kindergarten.
How were the goals on the Galileo School Readiness Scale selected?
In choosing which capabilities to include on the Galileo School Readiness Scale, ATI analyzed the specific goals in the Head Start Child Development and Early Learning Framework, various state early learning standards, and Common Core Kindergarten Standards. Those capabilities that were common among the Framework and standards were identified as the agreed-upon essential performance objectives for preschool-age children transitioning into kindergarten.
Once these standards were identified and analyzed, the developmentally sequenced capabilities that a child must learn to achieve these valued goals were determined and then included in the Galileo School Readiness Scale.
How do teachers use the Galileo School Readiness Scale?
The Galileo School Readiness Scale should be used in conjunction with the Galileo G3 Assessment Scales. Since the Galileo School Readiness Scale is made up of selected Galileo G3 Assessment Scale capabilities, a teacher may continue to assess their children on the Galileo G3 Assessment Scales. Once a program has access to the Galileo School Readiness Scale, the G3 capabilities a child has already learned will automatically be documented as learned on the Galileo School Readiness Scale. In the future, as a teacher documents a child’s learning on the Galileo G3 Assessment Scales, the like-capabilities on the Galileo School Readiness Scale will also be updated as learned, meaning no additional work for the teacher.
How do administrators use the Galileo School Readiness Scale?
Administrators may aggregate and analyze Galileo school readiness assessment data at multiple points in a year. Periodic data analysis will determine what progress children are making toward meeting these established goals. Reports such as the Progress Report will illustrate the increase in children’s developmental score from one observation period to next, indicating that there has been a gain in ability. The Developmental Profile, when aggregated, will spotlight the particular school readiness capabilities that children have been attaining, and which might need additional attention. Administrators may use the Galileo school readiness assessment data to inform continuous program improvements related to curriculum, teaching, instructional strategies, and the professional development of staff. Furthermore, data from this scale provides programs the opportunity to share with parents and the local community children’s readiness for kindergarten.
Interested in learning more?
Contact your Galileo Field Services Coordinator at 1.877.358.7611 if you would like to learn more.
Kerridan Smith, B.A.
Professional Development Director
Wednesday, May 16, 2012
ATI Item Development: Expertise and Experience
ATI item banks, currently containing more than 80,000 items, are among the largest item banks aligned to standards in the nation. The items within these banks have been developed by experts in the field of assessment and instructional design – our content specialists are highly trained and often former educators themselves. We also value the input of those actively in the education field, thus, during the course of a single school year, ATI items are reviewed by hundreds of educators. The combination of internal processes and continuous review by educators means ATI produces assessments that meet local needs and are highly effective in forecasting student mastery of performance standards and in guiding instruction to enhance student achievement.
The following is a glimpse into ATI’s item and assessment construction process.
Item Specifications
The first step in writing a new item to be included in the ATI item banks is to review the standard to be assessed with the item. The standard is broken down into the component skills that make up the standard. These skills are the starting point for developing a list of item specifications that define the characteristics of the particular class of item to be written.
Item Construction
After specifications have been written, items are constructed corresponding to the specifications using the online Bank Builder tool. Bank Builder includes features that promote item quality, such as allowing for the use of item families, or groups of items, that refer to the same contextual material.
To ensure reliable and efficient measurement of standards, passage length is controlled to ensure that assessments are an appropriate length to fit within the time limits typically found in school settings. Reliability is a direct function of assessment length.
The item construction process also includes attention to important principles of item writing. For example, when writing selected-response items, incorrect alternatives must be plausible answers that represent common mistakes or misunderstandings. By developing constructed-response, interactive, and performance items an opportunity exists for more authentic assessment of student proficiencies. The nature of the standard being assessed is evaluated to find the best alignment for the task (e.g., using constructed-response to measure summarizing; using inclusive performance assessment items to measure a broader standard; using interactive items to practice skills like measuring and observing.)
Careful attention is paid to all text and images related to items to ensure that the material is at the appropriate reading level. One measure of readability used routinely is the Flesch-Kincaid Index, a formula based on the number of words per sentence and the number of syllables per word. Item text and images are also analyzed to ensure appropriate use of technical language and for sensitivity to diversity in backgrounds.
Item Review and Certification
The internal review process for item specifications, items, and item families occurs in three steps. Step one is an independent review using the online bank review tool in Galileo K-12 Online. Reviewers may accept the material being reviewed, reject it, or accept it with modifications. Following the initial review, the material is subjected to a final review. When consensus is reached between the two reviewers and the writer, the material under review is accepted for use. In the case of item review, the item is certified. Only certified items are provided by ATI for use by school districts.
The item review process continues once a certified item is made available for use in an assessment. Items placed in district-designed assessments are subjected to review by each member of the district’s review team during the assessment review process. The development of new items based on district suggestions is one source of the continuing expansion of ATI item banks.
The following is a glimpse into ATI’s item and assessment construction process.
Item Specifications
The first step in writing a new item to be included in the ATI item banks is to review the standard to be assessed with the item. The standard is broken down into the component skills that make up the standard. These skills are the starting point for developing a list of item specifications that define the characteristics of the particular class of item to be written.
Item Construction
After specifications have been written, items are constructed corresponding to the specifications using the online Bank Builder tool. Bank Builder includes features that promote item quality, such as allowing for the use of item families, or groups of items, that refer to the same contextual material.
To ensure reliable and efficient measurement of standards, passage length is controlled to ensure that assessments are an appropriate length to fit within the time limits typically found in school settings. Reliability is a direct function of assessment length.
The item construction process also includes attention to important principles of item writing. For example, when writing selected-response items, incorrect alternatives must be plausible answers that represent common mistakes or misunderstandings. By developing constructed-response, interactive, and performance items an opportunity exists for more authentic assessment of student proficiencies. The nature of the standard being assessed is evaluated to find the best alignment for the task (e.g., using constructed-response to measure summarizing; using inclusive performance assessment items to measure a broader standard; using interactive items to practice skills like measuring and observing.)
Careful attention is paid to all text and images related to items to ensure that the material is at the appropriate reading level. One measure of readability used routinely is the Flesch-Kincaid Index, a formula based on the number of words per sentence and the number of syllables per word. Item text and images are also analyzed to ensure appropriate use of technical language and for sensitivity to diversity in backgrounds.
Item Review and Certification
The internal review process for item specifications, items, and item families occurs in three steps. Step one is an independent review using the online bank review tool in Galileo K-12 Online. Reviewers may accept the material being reviewed, reject it, or accept it with modifications. Following the initial review, the material is subjected to a final review. When consensus is reached between the two reviewers and the writer, the material under review is accepted for use. In the case of item review, the item is certified. Only certified items are provided by ATI for use by school districts.
The item review process continues once a certified item is made available for use in an assessment. Items placed in district-designed assessments are subjected to review by each member of the district’s review team during the assessment review process. The development of new items based on district suggestions is one source of the continuing expansion of ATI item banks.
Subscribe to:
Posts (Atom)