A comprehensive assessment system serving the needs of students from preschool to the 12th grade must invariably include many different types of assessment. The various assessment types serve different purposes. Benchmark and formative assessments are designed to inform instruction. Pretests and posttests are useful for measuring academic progress. Screening instruments are useful in identifying students at risk for learning problems. Placement tests inform grade-level placements, and advanced course placements. Computerized Adaptive Tests provide efficient measures of academic proficiency. Automated construction of Computerized Adaptive Tests increases testing options and enhances the capability to meet unique local needs. Observational assessments provide authentic measures of competencies in the environment in which those competencies are used. In addition, they provide immediate information that can be used to guide instruction. Value-Added Assessments can identify classes and schools that are highly successful as well as those needing additional assistance.
As the extensive list in the preceding paragraph implies, the most obvious benefit of a comprehensive assessment system is that it helps to ensure that all of the school’s assessment needs are met. Insofar as possible this is accomplished by including all of the various required types of assessment within the system. The vast majority of assessment types will be native to the system. However, there may be some types that are imported into the system. Statewide test results provide a familiar example.
An additional benefit to a comprehensive assessment system is that it creates economies in the assessment process including the construction, publication, administration, scoring, and reporting technology necessary for system implementation. This is the case because the same technology can be used for many different types of assessments.
Perhaps the most important benefit of a comprehensive assessment system is that it supports the ability to adapt to continuous change, which is a hallmark of education in our times. Assessment needs change continuously. A well designed comprehensive assessment system includes technology capable of accommodating change. For example, a well designed system should include the ability accommodate continuously changing standards. It should have the capability to rapidly align items to those standards. It should include dynamic item banks that expand continuously. It should have the capability to generate innovative item types that will be required as the transition to online assessment accelerates, and it should be capable of incorporating new types of assessments to meet changing needs. A comprehensive assessment system should always be and will always be a work in progress. Nothing less will meet the educational challenges of the 21st century.
Wednesday, January 5, 2011
Friday, October 15, 2010
Progress on Common Core State Standards
With the decision by the Kansas State Board of Education to adopt the Common Core State Standards, there are now 38 states participating in this cooperative effort to standardize high expectations for all students in the United States. Assessment Technology, Inc. is responding to this coordinated effort by the states to establish clear, common guidelines as we work to develop the next generation of instructional and assessment technology.
Users of Galileo K-12 services are already familiar with the alignment of our instructional materials and assessments to the specific state standards that have been in use. Now, as we transition into the Common Core State Standards, we are drawing upon the resources from each state that helped inform the development of the Common Core State Standards while also expanding and innovating to address the greater breadth and specificity in the new standards.
To provide an example, in third grade language arts, states where Galileo K-12 is used had as few as 16 and as many as 68 standards, sometimes incorporating writing as part of the standards, and in other states having separate standards for writing. In the Common Core State Standards, third grade English/Language Arts has 92 standards for instruction and assessment. We all know that third-graders aren't being expected to learn an additional 50% more than their predecessors. The Common Core State Standards are instead providing a greater degree of specificity in identifying the skills and knowledge that a student in the third grade is supposed to master to match the progress expected of all third graders in the Common Core State Standards.
Each state adopting the Common Core State Standards will develop crosswalk materials that align its existing high-stakes assessment items and prior standards to the new standards. ATI is already engaged in that process, drawing from our nationwide experience to incorporate the proven materials in our item and activity banks in specific alignment to the Common Core State Standards expectations. These items and activities will then be available to teachers to use in identifying initial student performance on the new standards and then evaluating student needs for instruction on standards that may be different or more detailed than was present in the previous version of state standards. This is where our extensive content from multiple states will prove valuable, having a ready resource for assessment and instruction in these newly emphasized capabilities.
We are looking forward to working with all of our client districts as this new era of educational cooperation and heightened expectations of student success unfolds.
Users of Galileo K-12 services are already familiar with the alignment of our instructional materials and assessments to the specific state standards that have been in use. Now, as we transition into the Common Core State Standards, we are drawing upon the resources from each state that helped inform the development of the Common Core State Standards while also expanding and innovating to address the greater breadth and specificity in the new standards.
To provide an example, in third grade language arts, states where Galileo K-12 is used had as few as 16 and as many as 68 standards, sometimes incorporating writing as part of the standards, and in other states having separate standards for writing. In the Common Core State Standards, third grade English/Language Arts has 92 standards for instruction and assessment. We all know that third-graders aren't being expected to learn an additional 50% more than their predecessors. The Common Core State Standards are instead providing a greater degree of specificity in identifying the skills and knowledge that a student in the third grade is supposed to master to match the progress expected of all third graders in the Common Core State Standards.
Each state adopting the Common Core State Standards will develop crosswalk materials that align its existing high-stakes assessment items and prior standards to the new standards. ATI is already engaged in that process, drawing from our nationwide experience to incorporate the proven materials in our item and activity banks in specific alignment to the Common Core State Standards expectations. These items and activities will then be available to teachers to use in identifying initial student performance on the new standards and then evaluating student needs for instruction on standards that may be different or more detailed than was present in the previous version of state standards. This is where our extensive content from multiple states will prove valuable, having a ready resource for assessment and instruction in these newly emphasized capabilities.
We are looking forward to working with all of our client districts as this new era of educational cooperation and heightened expectations of student success unfolds.
Tuesday, September 7, 2010
Five Steps Forward in Implementing Educational Reform in Your District
By now, it should come as no surprise that the recent proliferation of new educational policies and funding at federal and state levels is designed to facilitate a new era of reform in America. The overall effort to channel American education in new and innovative directions has presented an array of challenges and opportunities at all levels of the education continuum. Yet, when all is said and done, it is within the local community and school district where the broad sweeping ideas of educational reform are carefully vetted in the context of local reality. It is here where the local school board, administrators, educators, students, families, volunteers, and community partners systematically work together to develop plans for implementing reform.
It is axiomatic that within this local context, success in implementing and sustaining reform efforts over the long-term is best achieved when districts and governing boards exercise local control across a broad range of decision-making activities. Among the most salient of these decision-making activities is the selection and adoption of technology-based instructional improvement systems that provide a “best fit” with the district’s educational reform and student achievement goals. An instructional improvement system is generally conceived of as a technology-based, data-driven, standards-aligned, integrated system comprised of assessment, reporting, and instructional tools that can be used to support district efforts to elevate student achievement.
As you engage in the process of selecting a new instructional improvement system or replacing an old one, here are five fundamental steps you can take to help pave the way:
Building Consensus and Identifying Roles
As the old adage goes, “buy-in is critical.” In this regard, consensus building will need to take into consideration a variety of factors including district stakeholder interests, skills and expertise, and the impact of instructional improvement system adoption on established routines, to name a few. For some stakeholders, integration of an instructional improvement system into daily routines might represent a welcome enhancement. For others, it may initially evoke a sense of uncertainty about what specific kinds of changes will likely occur within the classroom and concern about how these changes may affect established routines and approaches to ongoing instruction. Consensus building in support of efforts to select and adopt an instructional improvement system helps to ensure that the course of action is one that everyone is generally comfortable with. At the outset, consensus building activities help to identify common ground for the importance of these efforts. Moreover, consensus building activities provide a collaborative context for: 1) identifying realistic and relevant goals in relation to system use; 2) developing an agreed to set of plans and procedures for implementation; 3) managing and monitoring implementation over time; and 4) evaluating and utilizing results to inform ongoing decision-making and implementation revisions. To get thing going, formulate some basic questions:
• How will input from stakeholders be gathered and taken into consideration?
• How will stakeholders be informed of the instructional improvement system options available?
• Will there be a formal review committee or will administrators seek stakeholder input?
Determine District Goals
In order to effectively evaluate an instructional improvement system for local deployment it is essential to identify immediate and long-term educational reform goals. A starting point for this process is the formulation of a few basic questions:
• What are the overall goals of our district in using an instructional improvement system?
• What are our assessment and reporting goals?
• What are our goals for the students?
• Teachers? Administrators?
• What are our goals related to providing educational content and monitoring curriculum implementation?
Develop Criteria for Consideration
Criteria to be considered in evaluating the “best fit” between an instructional improvement system and district goals might include taking a close look at the system’s assessment, scoring and reporting, and instructional capabilities. Consideration should also be given to the extent to which the system can accommodate professional learning communities and provide flexibility both in system implementation options and adaptability to changes in standards and government requirements. Finally, the security measures, data management features, and technical requirements of the system should be thoroughly vetted.
Establish Timelines and Determine the Selection Process
Timely implementation of a new instructional improvement system is the key to promoting successful outcomes. Consequently your district will want to work backwards from the desired implementation date in planning a realistic timeline for evaluating options and selecting an instructional improvement system. To facilitate this process, a number of questions might be considered:
• Will the selection process require a request for proposal (RFP) and if so, how much time is needed to review RFP responses and move the process through district channels?
• If an RFP is not required, what selection process will be used, who is involved, and how much time is needed?
• How will our governing board be included in the process and how will their role impact the timeline?
• What post-selection tasks and customization (district pacing calendar aligned benchmarks) might impact the timeline?
Compare Technology Solutions
Local control and empowerment over educational reform is fully exercised during the evaluation of the instructional improvement system options available and the eventual decision to select and adopt a particular instructional improvement system. Becoming fully informed about the options available, their strengths and limitations is essential to the success of this process. This can be accomplished in a number of ways:
• Gather information and materials including background research and referrals.
• Interview each organization offering an instructional improvement system solution and request an online walkthrough. This will allow you to preview systems and create a short list for presentations at the district.
• Request on-site presentations so that you can see firsthand the extent to which the various options meet your district’s needs.
• Ask for access to the system and assign staff to “try out” the system.
• Carefully review written proposals and cost estimates against your criteria and talk to other districts that use the instructional improvement system under consideration.
It is axiomatic that within this local context, success in implementing and sustaining reform efforts over the long-term is best achieved when districts and governing boards exercise local control across a broad range of decision-making activities. Among the most salient of these decision-making activities is the selection and adoption of technology-based instructional improvement systems that provide a “best fit” with the district’s educational reform and student achievement goals. An instructional improvement system is generally conceived of as a technology-based, data-driven, standards-aligned, integrated system comprised of assessment, reporting, and instructional tools that can be used to support district efforts to elevate student achievement.
As you engage in the process of selecting a new instructional improvement system or replacing an old one, here are five fundamental steps you can take to help pave the way:
Building Consensus and Identifying Roles
As the old adage goes, “buy-in is critical.” In this regard, consensus building will need to take into consideration a variety of factors including district stakeholder interests, skills and expertise, and the impact of instructional improvement system adoption on established routines, to name a few. For some stakeholders, integration of an instructional improvement system into daily routines might represent a welcome enhancement. For others, it may initially evoke a sense of uncertainty about what specific kinds of changes will likely occur within the classroom and concern about how these changes may affect established routines and approaches to ongoing instruction. Consensus building in support of efforts to select and adopt an instructional improvement system helps to ensure that the course of action is one that everyone is generally comfortable with. At the outset, consensus building activities help to identify common ground for the importance of these efforts. Moreover, consensus building activities provide a collaborative context for: 1) identifying realistic and relevant goals in relation to system use; 2) developing an agreed to set of plans and procedures for implementation; 3) managing and monitoring implementation over time; and 4) evaluating and utilizing results to inform ongoing decision-making and implementation revisions. To get thing going, formulate some basic questions:
• How will input from stakeholders be gathered and taken into consideration?
• How will stakeholders be informed of the instructional improvement system options available?
• Will there be a formal review committee or will administrators seek stakeholder input?
Determine District Goals
In order to effectively evaluate an instructional improvement system for local deployment it is essential to identify immediate and long-term educational reform goals. A starting point for this process is the formulation of a few basic questions:
• What are the overall goals of our district in using an instructional improvement system?
• What are our assessment and reporting goals?
• What are our goals for the students?
• Teachers? Administrators?
• What are our goals related to providing educational content and monitoring curriculum implementation?
Develop Criteria for Consideration
Criteria to be considered in evaluating the “best fit” between an instructional improvement system and district goals might include taking a close look at the system’s assessment, scoring and reporting, and instructional capabilities. Consideration should also be given to the extent to which the system can accommodate professional learning communities and provide flexibility both in system implementation options and adaptability to changes in standards and government requirements. Finally, the security measures, data management features, and technical requirements of the system should be thoroughly vetted.
Establish Timelines and Determine the Selection Process
Timely implementation of a new instructional improvement system is the key to promoting successful outcomes. Consequently your district will want to work backwards from the desired implementation date in planning a realistic timeline for evaluating options and selecting an instructional improvement system. To facilitate this process, a number of questions might be considered:
• Will the selection process require a request for proposal (RFP) and if so, how much time is needed to review RFP responses and move the process through district channels?
• If an RFP is not required, what selection process will be used, who is involved, and how much time is needed?
• How will our governing board be included in the process and how will their role impact the timeline?
• What post-selection tasks and customization (district pacing calendar aligned benchmarks) might impact the timeline?
Compare Technology Solutions
Local control and empowerment over educational reform is fully exercised during the evaluation of the instructional improvement system options available and the eventual decision to select and adopt a particular instructional improvement system. Becoming fully informed about the options available, their strengths and limitations is essential to the success of this process. This can be accomplished in a number of ways:
• Gather information and materials including background research and referrals.
• Interview each organization offering an instructional improvement system solution and request an online walkthrough. This will allow you to preview systems and create a short list for presentations at the district.
• Request on-site presentations so that you can see firsthand the extent to which the various options meet your district’s needs.
• Ask for access to the system and assign staff to “try out” the system.
• Carefully review written proposals and cost estimates against your criteria and talk to other districts that use the instructional improvement system under consideration.
Tuesday, August 31, 2010
Adaptive Testing
While adaptive testing has been around for a long time, it has recently been gaining in popularity. Many have been looking at it as a magic bullet to meet the data driven decision making requirements of the modern accountability era. The approach can greatly shorten the time required to deliver an assessment with an acceptable level of measurement precision. While it is certainly true that adaptive testing has a number of potential benefits that provide it a place in the assessment bag of tools, it also has a number of limiting characteristics that have to be considered in selecting it for a particular use.
How does an adaptive test adapt? The approaches that are currently used fall into two basic categories based on whether adaptation decisions are made after every item or after a group of items. In the first case, a cumulative score is calculated each time a student completes an item. The next item is typically selected by determining the item that will provide the optimal level of information regarding the student’s ability. Once a score has been reached that provides an adequate level of measurement precision, the assessment is ended. Typically this endpoint is reached after far fewer questions than would be required to achieve a comparable level of precision on a standard or “linear” test , thus making the approach extremely efficient.
In the second case, decisions are made after administering preconfigured blocks of items rather than on an item by item basis. A typical design might specify a routing test followed by two stages, each of which involves a decision between pre-constructed item blocks optimized for lower, average, and high ability. In this design, students would complete the routing test and then be assigned to the appropriate second stage item block based on their score. After completing the second stage they would again be routed to the appropriate block of items in stage three based on their cumulative score on the routing test and the second stage. After completing the third stage the assessment would be ended and an overall score determined based on all the items completed.
What are the benefits of adaptive testing? One of the most notable is its efficiency. This is particularly true with the item by item approach. As I mentioned, an adequately precise measure of a student’s ability may typically be achieved with far fewer items than would be required for a typical “linear” test. This efficiency can make the approach far easier to use when large groups of students must be tested in a short time.
What considerations go along with the benefits? Two of the most fundamental are content control and item exposure. Both are controlled to some degree by the multi-stage design. The multi-stage design is not quite as efficient as the single-stage design. However, in those cases in which content control and/or item exposure are important considerations, the multi-stage approach may be preferred.
Content control is especially important in standards-based education. While the item by item approach is extremely efficient, the ability to ensure that all students are exposed to content reflecting standards targeted for instruction may be compromised in the item by item approach. Johnny, Suzy, and Javier may all sit down together and not be exposed to any of the same items. Suzy may have to multiply fractions while Johnny and Javier don’t. This may not be cause for concern if the test is being used for purposes of screening, but it may limit the utility of the measure if the results are being used for very specific decisions about student instructional needs. For instance, if there are instructional decisions to be made about providing instructional coaching on multiplication of fractions for Johnny and Javier, it would likely be most informative they had actually been asked to respond to the questions that Suzy was given. While Item Response Theory (IRT) may be used to estimate the probability that students have mastered these skills even if they haven’t been given the questions, actual administration of the items can afford the additional benefit allowing mistakes to be analyzed. The pre-constructed design of the multi-stage approach affords control of this issue at the expense of some efficiency.
Item exposure is particularly important when item security is a concern. The focus of the item selection algorithms on picking the optimal item for a given ability level can mean that the same limited set of items come to the top of the heap repeatedly. This is particularly true for the item by item approach. This issue can limit the utility of the assessment for repeated use because students are likely to see the same items again and again. In high stakes situations, the door is open for cheating because the items that are likely to show up become predictable. As with the content consideration, the multi-stage approach provides some control over this issue at the expense of efficiency.
The combination of strengths and unique considerations involved with adaptive testing point to some particular uses for which it would be a strong choice. One of the most notable is screening. The efficiency with which an adaptive test can provide a reliable measure of ability make it a good choice when the issue at hand is deciding whether a student should enter into a particular instructional group or class. The degree of content control provided by a multi-stage approach could provide more granular information when the decision at hand is whether a child should be given additional help on a specific skill or set of skills.
ATI is current working on a module to support adaptive testing. The module will support the the item by item and multi-stage approaches. Provision of these options will allow districts to take advantage of the strengths provided by adaptive testing when they are designing an assessment program that will best meet their needs.
How does an adaptive test adapt? The approaches that are currently used fall into two basic categories based on whether adaptation decisions are made after every item or after a group of items. In the first case, a cumulative score is calculated each time a student completes an item. The next item is typically selected by determining the item that will provide the optimal level of information regarding the student’s ability. Once a score has been reached that provides an adequate level of measurement precision, the assessment is ended. Typically this endpoint is reached after far fewer questions than would be required to achieve a comparable level of precision on a standard or “linear” test , thus making the approach extremely efficient.
In the second case, decisions are made after administering preconfigured blocks of items rather than on an item by item basis. A typical design might specify a routing test followed by two stages, each of which involves a decision between pre-constructed item blocks optimized for lower, average, and high ability. In this design, students would complete the routing test and then be assigned to the appropriate second stage item block based on their score. After completing the second stage they would again be routed to the appropriate block of items in stage three based on their cumulative score on the routing test and the second stage. After completing the third stage the assessment would be ended and an overall score determined based on all the items completed.
What are the benefits of adaptive testing? One of the most notable is its efficiency. This is particularly true with the item by item approach. As I mentioned, an adequately precise measure of a student’s ability may typically be achieved with far fewer items than would be required for a typical “linear” test. This efficiency can make the approach far easier to use when large groups of students must be tested in a short time.
What considerations go along with the benefits? Two of the most fundamental are content control and item exposure. Both are controlled to some degree by the multi-stage design. The multi-stage design is not quite as efficient as the single-stage design. However, in those cases in which content control and/or item exposure are important considerations, the multi-stage approach may be preferred.
Content control is especially important in standards-based education. While the item by item approach is extremely efficient, the ability to ensure that all students are exposed to content reflecting standards targeted for instruction may be compromised in the item by item approach. Johnny, Suzy, and Javier may all sit down together and not be exposed to any of the same items. Suzy may have to multiply fractions while Johnny and Javier don’t. This may not be cause for concern if the test is being used for purposes of screening, but it may limit the utility of the measure if the results are being used for very specific decisions about student instructional needs. For instance, if there are instructional decisions to be made about providing instructional coaching on multiplication of fractions for Johnny and Javier, it would likely be most informative they had actually been asked to respond to the questions that Suzy was given. While Item Response Theory (IRT) may be used to estimate the probability that students have mastered these skills even if they haven’t been given the questions, actual administration of the items can afford the additional benefit allowing mistakes to be analyzed. The pre-constructed design of the multi-stage approach affords control of this issue at the expense of some efficiency.
Item exposure is particularly important when item security is a concern. The focus of the item selection algorithms on picking the optimal item for a given ability level can mean that the same limited set of items come to the top of the heap repeatedly. This is particularly true for the item by item approach. This issue can limit the utility of the assessment for repeated use because students are likely to see the same items again and again. In high stakes situations, the door is open for cheating because the items that are likely to show up become predictable. As with the content consideration, the multi-stage approach provides some control over this issue at the expense of efficiency.
The combination of strengths and unique considerations involved with adaptive testing point to some particular uses for which it would be a strong choice. One of the most notable is screening. The efficiency with which an adaptive test can provide a reliable measure of ability make it a good choice when the issue at hand is deciding whether a student should enter into a particular instructional group or class. The degree of content control provided by a multi-stage approach could provide more granular information when the decision at hand is whether a child should be given additional help on a specific skill or set of skills.
ATI is current working on a module to support adaptive testing. The module will support the the item by item and multi-stage approaches. Provision of these options will allow districts to take advantage of the strengths provided by adaptive testing when they are designing an assessment program that will best meet their needs.
Tuesday, August 17, 2010
ATI Develops New Technological Capabilities to Assess Literacy in Early Childhood
During the period from early childhood to third grade, children are developing the skills necessary to become successful readers such as an awareness of how the sounds in spoken language correspond to written language. As children progress to fourth grade and beyond, their ability to successfully read and comprehend text becomes critical for learning in other domains like math, science, and social studies. Unfortunately, although literacy is a major focus of early childhood education, not all children become successful readers. According to the 2009 National Assessment of Educational Progress (NAEP), one in three fourth-graders nationwide cannot read above a basic level.
Over the last decade, childhood literacy has become an important topic on a national and state level. In 1997 and 2002, Congress convened two national panels (the National Reading Panel and the National Early Literacy Panel) to provide research-based recommendations on how to improve reading achievement in early childhood. The reports issued by these panels are available at www.nationalreadingpanel.org and www.nifl.gov. Individual states are also implementing initiatives related to early childhood literacy. For example, Arizona recently passed a bill which calls upon a statewide task force to provide recommendations for a set of statewide assessments to measure students’ reading abilities in grades one and two. The bill also requires school districts to screen students in preschool through second grade for reading deficiencies, providing an opportunity for early intervention during this critical time period. Along with these evaluations, the bill requires that students reading far below grade level at the end of third grade are not promoted to fourth grade. Retained students must be provided with targeted intervention such as summer school reading instruction, online reading instruction, a different reading teacher, or intensive reading instruction during the next academic year.
To support national and state early childhood literacy initiatives, ATI has been working to develop a set of assessments targeting the critical aspects of reading for grades K through three. By providing valid, reliable, standards aligned assessments, ATI can assist districts and schools by identifying students with reading deficiencies and suggesting appropriate areas for intervention. ATI also supports targeted interventions and online reading instruction through Instructional Dialogs and other intervention materials that allow students to practice early literacy skills. Assessing the early literacy skills of very young children presents some unique challenges. For example, many standards related to early literacy are not best assessed using a standard text-only, multiple-choice item. In addition, very young children often cannot read even simple text. For these reasons, the development of new technological capabilities and innovative item types has been an important part of ATI’s work.
One new and exciting technological capability developed by ATI for these assessments is the ability to include audio material in items. The audio capability makes it possible to create computer-administered items for children who are not yet able to read. Instead of the teacher reading the item aloud, the child can listen to pre-recorded instructions, questions, and answer choices. This capability enables a more standardized presentation of items and makes the assessment easier to administer. In addition, the audio capability allows the direct assessment of standards that address the awareness or manipulation of the sounds in spoken language (phonemic awareness) without teacher involvement.
Another newly developed ATI technological capability is the ability to present text for a predetermined time period. This capability has been crucial to the development of a new innovative item type designed to assess reading fluency (a concept related to reading rate). Previously, assessment options for reading fluency were limited to one-on-one testing and scoring procedures requiring subjective judgment. This new capability represents a major advance in ATI’s coverage of early childhood standards by enabling online assessment of reading fluency and automated scoring.
Two more technological capabilities have been developed by ATI to tailor the assessment process for very young children. First, to keep children engaged and to encourage them to progress through the assessment, visually engaging scenes have been developed that slowly appear piece by piece as a reward after each question is answered. Second, the new assessments and item types have been designed to be compatible with portable tablet computers that employ touch screen technology as well as standard desktop computers. By facilitating the assessment of young children, these new technological capabilities developed by ATI will support districts and schools in implementing literacy assessments and interventions in early childhood.
Over the last decade, childhood literacy has become an important topic on a national and state level. In 1997 and 2002, Congress convened two national panels (the National Reading Panel and the National Early Literacy Panel) to provide research-based recommendations on how to improve reading achievement in early childhood. The reports issued by these panels are available at www.nationalreadingpanel.org and www.nifl.gov. Individual states are also implementing initiatives related to early childhood literacy. For example, Arizona recently passed a bill which calls upon a statewide task force to provide recommendations for a set of statewide assessments to measure students’ reading abilities in grades one and two. The bill also requires school districts to screen students in preschool through second grade for reading deficiencies, providing an opportunity for early intervention during this critical time period. Along with these evaluations, the bill requires that students reading far below grade level at the end of third grade are not promoted to fourth grade. Retained students must be provided with targeted intervention such as summer school reading instruction, online reading instruction, a different reading teacher, or intensive reading instruction during the next academic year.
To support national and state early childhood literacy initiatives, ATI has been working to develop a set of assessments targeting the critical aspects of reading for grades K through three. By providing valid, reliable, standards aligned assessments, ATI can assist districts and schools by identifying students with reading deficiencies and suggesting appropriate areas for intervention. ATI also supports targeted interventions and online reading instruction through Instructional Dialogs and other intervention materials that allow students to practice early literacy skills. Assessing the early literacy skills of very young children presents some unique challenges. For example, many standards related to early literacy are not best assessed using a standard text-only, multiple-choice item. In addition, very young children often cannot read even simple text. For these reasons, the development of new technological capabilities and innovative item types has been an important part of ATI’s work.
One new and exciting technological capability developed by ATI for these assessments is the ability to include audio material in items. The audio capability makes it possible to create computer-administered items for children who are not yet able to read. Instead of the teacher reading the item aloud, the child can listen to pre-recorded instructions, questions, and answer choices. This capability enables a more standardized presentation of items and makes the assessment easier to administer. In addition, the audio capability allows the direct assessment of standards that address the awareness or manipulation of the sounds in spoken language (phonemic awareness) without teacher involvement.
Another newly developed ATI technological capability is the ability to present text for a predetermined time period. This capability has been crucial to the development of a new innovative item type designed to assess reading fluency (a concept related to reading rate). Previously, assessment options for reading fluency were limited to one-on-one testing and scoring procedures requiring subjective judgment. This new capability represents a major advance in ATI’s coverage of early childhood standards by enabling online assessment of reading fluency and automated scoring.
Two more technological capabilities have been developed by ATI to tailor the assessment process for very young children. First, to keep children engaged and to encourage them to progress through the assessment, visually engaging scenes have been developed that slowly appear piece by piece as a reward after each question is answered. Second, the new assessments and item types have been designed to be compatible with portable tablet computers that employ touch screen technology as well as standard desktop computers. By facilitating the assessment of young children, these new technological capabilities developed by ATI will support districts and schools in implementing literacy assessments and interventions in early childhood.
Tuesday, July 20, 2010
District Suggestions for New Items
ATI welcomes suggestions for new items. Such suggestions help to ensure that assessments will meet district needs. ATI currently has one of the largest item banks in the nation containing over seventy-thousand certified items. Moreover, as mentioned above, the banks are continuously growing.
ATI has stringent guidelines for creating items. When making suggestions for new items, there are a number of considerations that a district may find useful. Suggestions for new items are likely to benefit the district most when they focus on the measurement of a specific skill or capability. If the district curriculum includes a particular capability that is not currently well represented in ATI item banks, suggesting the inclusion of additional items assessing the skill in question will be beneficial as long as the skill suggested aligns to the standard. At times, textbooks contain concepts which do not align to any state standards. In order to maintain the validity of ATI assessments, only items which align to a specific state standard may be added to ATI’s assessment bank. The more specific a request provided by a district, the more likely that ATI’s writers will be able to produce the exact items a district desires.
Suggesting stylistic changes is generally less useful. ATI does develop items specifically aligned to the various styles of statewide assessments. ATI, as well as states testing companies, have specific guidelines for measuring reading level. Many items use specific wording in order to maintain appropriate grade-level readability. For example, the phrase “past tense” will not be used on a third grade test since the word “tense” has a reading level ranking of sixth grade. It is important to ensure that items on local benchmark and formative assessments not be limited to a particular style. Only practicing items using specific wording when learning a new concept may cause challenges for students later. Even state released items change their wordings from year to year. One year the first grade word problem may include the phrase “in all” and the following year it may include the phrase “altogether”. It is important for students to practice items using both phrases. In addition, as items become closer in style to a statewide test, the danger that test outcomes will be contaminated by the phenomenon of “teaching to the test” increases. Benchmark and formative assessments are intended to measure the mastery of standards, not merely the ability to respond to items written in one particular style.
ATI has stringent guidelines for creating items. When making suggestions for new items, there are a number of considerations that a district may find useful. Suggestions for new items are likely to benefit the district most when they focus on the measurement of a specific skill or capability. If the district curriculum includes a particular capability that is not currently well represented in ATI item banks, suggesting the inclusion of additional items assessing the skill in question will be beneficial as long as the skill suggested aligns to the standard. At times, textbooks contain concepts which do not align to any state standards. In order to maintain the validity of ATI assessments, only items which align to a specific state standard may be added to ATI’s assessment bank. The more specific a request provided by a district, the more likely that ATI’s writers will be able to produce the exact items a district desires.
Suggesting stylistic changes is generally less useful. ATI does develop items specifically aligned to the various styles of statewide assessments. ATI, as well as states testing companies, have specific guidelines for measuring reading level. Many items use specific wording in order to maintain appropriate grade-level readability. For example, the phrase “past tense” will not be used on a third grade test since the word “tense” has a reading level ranking of sixth grade. It is important to ensure that items on local benchmark and formative assessments not be limited to a particular style. Only practicing items using specific wording when learning a new concept may cause challenges for students later. Even state released items change their wordings from year to year. One year the first grade word problem may include the phrase “in all” and the following year it may include the phrase “altogether”. It is important for students to practice items using both phrases. In addition, as items become closer in style to a statewide test, the danger that test outcomes will be contaminated by the phenomenon of “teaching to the test” increases. Benchmark and formative assessments are intended to measure the mastery of standards, not merely the ability to respond to items written in one particular style.
Wednesday, July 7, 2010
Student File Attachment
Galileo users now have the option of adding attachments to a student’s record. These attachments can be accessed through the K-12 Student/Parent Center. Districts quite often mail out announcements or test results to the student and/or family. In certain states, the local department of education has turned over the responsibility of mailing out state test results to the school districts. In these budget conscious times, Galileo can save districts money on postage as well as labor, by mailing out announcements or test results.
On the Student Demographics page, there’s a link to “Add Student Files…” In addition, there’s an indicator as to how many files are attached so users don’t have to open the interface to find out.
When you click the "Add Student Files..." link, the Student Files page opens. There's no limitation as to the file type that can be added, and, currently, no size limit. The option is given to add a file description. The final step is attaching the file and selecting the “Add” button.
If the view/binocular is selected, the file attachment will open.
On the Student Demographics page, there’s a link to “Add Student Files…” In addition, there’s an indicator as to how many files are attached so users don’t have to open the interface to find out.
When you click the "Add Student Files..." link, the Student Files page opens. There's no limitation as to the file type that can be added, and, currently, no size limit. The option is given to add a file description. The final step is attaching the file and selecting the “Add” button.
The attachments/files are viewed in the K-12 Student/Parent Center. Once the student/parent logs in with the unique student user account, select the “more options” label/“Student Files” link to view the attachments.
If the view/binocular is selected, the file attachment will open.
Subscribe to:
Posts (Atom)