Monday, June 21, 2010

Preparing Your 2010-2011 Rosters

With the arrival of summer ATI staff members responsible for Galileo student data management shift their focus to, of all things, fall. This focus is primarily on new classes, rosters, and accommodating all staff changes a district may undertake throughout the summer.

For districts new to Galileo, the best way to create your class lists and rosters is through Galileo Data Import (GDI). The GDI process involves three basic steps:

  1. Districts provide an export from their Student Information System (SIS) that lists all classes, teachers, and students within the district.
  2. ATI staff import this data to a test database and perform a thorough quality assurance review – any problems are resolved with the district before the data is imported (step 3) into the live Galileo database.
  3. ATI staff import the district-provided information directly into Galileo K-12 Online.

Detailed instructions for GDI can be found in the Tech Support section of Galileo K-12 (and Preschool) Online, as well as at the following links:

Preschool: http://www.ati-online.com/pdfs/ImportInstructionsPre-K.pdf

K-12: http://www.ati-online.com/pdfs/ImportInstructionsK-12.pdf

As you prepare your 2010-2011 program year data for import, please remember the following important points:

  1. Be sure to include all required information in your import.
  2. Optional information is not required in the Galileo database, but failure to include this information may adversely affect future report filtering.
    • Any omitted optional data can be imported at any time throughout the program year, either as part of GDI or as a separate process – contact ATI for more information.
  3. If TeacherID or StudentID fields change within your SIS, please notify ATI prior to providing any import files to ensure proper transition within the Galileo database. Large-scale ID changes may require extra processing time so please notify ATI as far in advance as possible so we can help you plan accordingly.
  4. Due to new class structures and teacher assignments, the quality assurance process is typically longest during the first upload of the year. Getting uploads underway as soon as data is available will help ensure adequate processing time before your first assessments of the year.
    • Providing ‘shell’ data with class structures and teachers as soon as they are entered into your SIS this summer will provide opportunity to perform thorough review of class alignment, course codes and teacher assignments. This will speed the processing of files containing student enrollment in the fall.

Please refer to the links above for more details about the import process.

Friday, June 11, 2010

ATI and the Common Core Standards

With the June 2, 2010 release of the Common Core Standards by the National Governors Association and the Council of Chief State School Officers, a new set of opportunities has been presented to educators to establish more rigorous and beneficial academic expectations for students and schools.

The development of these standards has already influenced the standards revision process in many states. Those states currently in the process of revising and updating their local standards will now have an opportunity to consider how the Common Core Standards will inform their new educational objectives, whether through direct adoption or mapping of common goals, as part of a collective effort to raise student achievement across the nation.

Assessment Technology, Inc. is eagerly anticipating the opportunity to assist districts using Galileo K-12 Online in the transition to new state standards and in aligning their standards with the Core Curriculum Standards. ATI’s existing process of specification to develop precise assessment and instructional content is the key factor in the process to manage this transition.

As we develop materials and questions to instruct and measure student success on the state standards we write specifications that not only accurately and effectively measure the standard, but further refine the process by defining the particular skills or knowledge inherent in components of that standard. This fundamental, systemic approach places the full content of the Galileo K-12 Online banks in a format that allows for detailed and precise mapping, or in this case remapping, of assessment items and instructional materials to particular characteristics of state standards.

The diversity of our 74,000+ item bank and 1,500+ Instructional Dialogs draws on our experience writing to measure the standards present in each state and then finding the exact mapping appropriate to those items in other states where that same skill or knowledge is demanded. Rather than applying generic items at the most broad levels, we have defined the specifications to allow lessons, activities, and items to be used exactly where they are needed. This expertise will be a tremendous asset as ATI works with our partner-districts in the transition from the generation of standards we have all been using over the last five to eight years, to the new state or Common Core Standards as they are adopted by the states.

Tuesday, June 8, 2010

Interim and End-of-Course Examinations in Standards-Based Education

Interim and end-of-course examinations are among the oldest and most common forms of assessment in education. For as long as they have been in use, interim and end-of-course examinations have been designed to hold students accountable for the mastery of course material. However, these examinations have also been used to guide instruction to promote further learning. With the advent of the education reform movement, the guidance function of course examinations is receiving increased attention. The education reform movement (e.g., Ravitch, 2001) has placed increased responsibility on educators to raise the level of student academic accomplishment. As part of the reform effort, the movement has promoted standards-based assessments including benchmark and formative assessments, which provide information to guide instruction. The contrasting purposes of accountability and guidance require special attention in light of the significant changes that have occurred in assessment practice and instruction promoted through the education reform movement. Historically, the accountability function served by course examinations has focused on identifying and separating able students from those less able. Since its inception, the educational reform movement has been concerned with identifying able teachers and effective educational practices that can raise the level of achievement for all students. In order to achieve the goals of the education reform movement, it is important to reconcile the contrasting purposes of accountability associated with course examinations and guidance associated with standards-based assessments. This post recounts the purposes of course examinations and details the ways in which these types of assessment can be adapted for use in standards-based education to enhance student learning.

Interim and End-of-Course Examinations

The purposes of interim and end-of-course assessments have been understood for a very long time. We revisit them here to differentiate course assessments from standards-based assessments. The discussion of differentiation provides a foundation for integrating the two forms of assessment to promote student learning.

Interim and end-of-course examinations are designed to assess knowledge and skills targeted in course instruction. Interim examinations assess knowledge and skills covered during part of a course. For example, an instructor may decide to administer a mid-term examination half way through a course. The mid-term examination will cover material addressed in the first half of the course. The end-of-course or final examination may cover material reflecting the entire course, or it may cover material presented since the previous interim examination.

Course Examinations and Accountability

Although course examinations have always played a role in guiding instruction, their role in accountability has also been extensive. Moreover, the importance of accountability increases as children progress through school. Interim and end-of-course assessments hold the student accountable for learning what has been taught. As students advance through the grades, teachers place increasing responsibility for learning on student shoulders. The teacher’s responsibility is to provide opportunities to learn the material presented. The student is responsible for learning the material. Interim and end-of-course examinations provide information on the extent to which students have effectively met their responsibility for learning.

As more and more responsibility for learning is assigned to the student, interim and end-of-course examinations increasingly become high stakes assessments. The consequences associated with different performance levels on these assessments significantly impact future instructional options that may be available to the student. Students who fail course examinations will likely fail the course and as a consequence not receive credit for taking the course. Failure may require that the student take the course a second time. In some cases, poor performance on course examinations precludes the opportunity to take certain advanced courses. Poor performance on course examinations may also limit options for higher education. Sometimes failure experiences lead students to conclude that continued pursuit of education is not worth the effort. One consequence associated with this conclusion is an unacceptably large high school dropout rate.

Standards-based educational practice offers an alternative to traditional practice related to course examinations. In standards-based education, high stakes assessment is typically limited to consequences associated with performance on a statewide test. The principal purpose of local standards-based assessments is to provide information that can be used to guide instruction. Data from benchmark and formative assessments are used to plan interventions to promote further learning. The student has the ultimate responsibility for learning. However, the teacher assumes greater responsibility than that typically required in the absence of a standards-based initiative. For example, if a benchmark assessment reveals that a student is at risk for not meeting standards on a statewide test, an intervention may be undertaken to bring the student on course to meet standards. This is in direct contrast to the situation associated with interim and end-of-course assessments, in which the responsibility of mastering material is primarily on the shoulders of the student, and the interim or end-of-course assessment simply indicates to the student whether or not he or she was successful in achieving that goal. The discussion that follows suggests changes in the use of course examinations that make it possible to employ these examinations within the context of standards-based initiatives designed to elevate student achievement.

Standards-Based Course Examinations

Typically interim and end-of-course examinations are informal examinations constructed at the classroom level for the purpose of assessing student proficiency related to material covered in the course and presented in the class. Additional benefits can be derived from these assessments to the extent that they can be linked to standards-based procedures designed to enhance learning.

Informal interim course examinations are like the short formative assessments used in standards-based education in that the psychometric properties of both forms of assessment are generally not examined. In both cases the number of students taking the assessments is generally too small to support statistical analyses necessary to establish the psychometric properties of the instruments. Another similarity is that both forms of assessment are generally constructed by classroom teachers. The fact that the assessments are teacher-constructed helps to ensure that the assessments reflect what has been taught. Major differences include the fact that course examinations are not required to be aligned with standards whereas formative assessments used in standards-based education are required to be aligned to standards. In addition, course examinations are not necessarily used extensively to guide instruction whereas the central purpose of formative assessments is to guide subsequent instructional planning.
In order to apply informal course examinations in standards-based education, it would be necessary to ensure that the assessments were used not only to document what had been learned, but also to inform instructional decisions designed to elevate learning. To make this happen, the items in the examinations would have to be aligned to standards. In addition, instruction would have to be planned based on assessment results. Typically an upcoming course examination is preceded by instruction including a review of material previously presented. This instruction-assessment sequence is consistent with standards-based educational practice. However, in standards-based education, it is also important to ensure that assessment results are used to guide future instruction. Inclusion of this component of the assessment-instruction sequence increases the potential contribution of course examinations to student learning.

In some cases, interim and end-of-course examinations are administered to large numbers of students enrolled in multiple sections of the same course. These examinations are often taken by sufficient numbers of students to support psychometric analyses examining reliability and validity. Interim and end-of-course examinations that are subjected to psychometric analyses are like benchmark assessments used in standards-based education. Benchmarks are designed to provide reliable and valid assessments aligned to standards reflecting the district curriculum. Benchmarks indicate the extent to which standards targeted for instruction at successive time periods have been mastered. They also are used to forecast performance on upcoming statewide assessments. Interim and end-of-course examinations can function as benchmarks if the items included in the examination are aligned to standards and if reliability and validity are established. For example, if an interimor end-of-course assessment is to be used to forecast performance on a statewide assessment, then the assessment must be long enough to achieve acceptable reliability levels, and its validity, or effectiveness in forecasting statewide assessment performance, must be established.

Conclusion

Interim and end-of-course examinations support long-held views regarding the responsibilities of teachers and students related to learning. The responsibility of the teacher is to provide opportunities for students to learn. The responsibility of students is to take advantage of those opportunities in ways that will enable them to achieve their full potential. These views have made a significant contribution to the foundations of American education, and they remain critical components of our educational system. Yet, rapid advances in technology, the explosion of knowledge in the information age, and the advent of the global economy have called for new approaches to education that will lead to elevated achievement necessary to maintain a competitive edge in the new world that confronts us. The challenge of today is not merely to judge the achievements of students who have been given learning opportunities. It is to elevate the achievement of all students.

Standards-based education has clarified the nature of this challenge and the possible approaches that can be taken to meet the challenge. It has provided standards representing valued educational accomplishments. It has produced evolving assessment technology to measure the attainment of standards. It has introduced the power of measurement technology into local assessment programs, and it has made it clear that the national goal of elevating student achievement is attainable. The task ahead related to interim and end-of-course assessments is to link assessment to measure student achievement with assessment to guide instruction in ways that will raise achievement to new levels. This post has described some of the ways in which linking might occur.