Next-Generation Data Importation: Flexibility for the Future of Your School

The Arizona Framework for Measuring Teacher Effectiveness, Colorado State Council for Educator Effectiveness, The Massachusetts Task Force on the Evaluation of Teachers and Administrators… that which we call “instructional effectiveness” (IE) by any other name would still require accurate student enrollment and demographic information. With apologies to William Shakespeare, widespread interest in measuring the effectiveness of teachers has spawned legislation in many states outlining assessment requirements to guide the evaluation process for effectiveness of instructors and school administrators.

The need to measure progress regularly lies at the heart of instructional effectiveness initiatives and the ability to achieve positive results. To do so, districts will need to ensure accountability for changes within the instructional improvement and instructional effectiveness system to match the student information system (SIS) regularly, throughout the annual assessment cycle. Many state education systems face the question of how to achieve measurable, positive IE results and it can be daunting. But never fear, the Galileo Data Import (GDI) module is here to help! GDI is time-tested, having imported in excess of one million student records since being introduced over ten years ago. In addition to student enrollment and demographic information, GDI can also be used to update Galileo K-12 Online with teacher information, class-level and student user accounts, and IE-relevant course information such as subject area and grade level.

As part of the Galileo K-12 Online Instructional Improvement and Instructional Effectiveness System, all client districts are availed of the GDI module. The purpose of GDI is to allow districts to update student enrollment and demographic information with the frequency needed to support classroom initiatives – whether monthly or daily updates, GDI can meet the need.

Major advantages of the GDI module include:

1. Cost: In the increasingly challenging financial climate many state education systems face questions about how to fund worthy initiatives like IE. Client districts are able to update student enrollment with the frequency necessary to ensure classroom success at no additional cost over the annual Galileo K-12 Online subscription rate.


2. Data Pre-Validation: To start each program year, every district in need of frequent data importation will go through a validation process to ensure the export format meets their needs and the requirements of the Galileo K-12 Online Instructional Improvement and Instructional Effectiveness System. Once this is complete, SIS export files generated using a consistent format are assured of uploading efficiently and without risk to existing Galileo data.


3. Extensibility: Unlike many data exchange frameworks, GDI is flexible enough to accommodate the changing needs of the district SIS and data storage needs – at any point in the program year. This includes changes such as altering the format of a particular field (e.g., class name) up to, and including, the addition of new data fields at any time.


• A key advantage of this flexibility with regards to IE is the ability to add course subject data to the Galileo K-12 Online Instructional Improvement and Instructional Effectiveness System. Whether the information is stored in your SIS or you export a list of courses and update the subject manually, accurate alignment of district courses to the subject being taught is possible with Galileo and GDI.


4. Automation: Many student information systems are capable of scripted export. Add to these a batch FTP transfer to ATI servers and the entire import process can be “hands free” for the district once the initial pre-validation is completed.

The measurement of IE and student growth present the opportunity to perform timely intervention and create dynamic, customized assessments. Continuous measurement, evaluation, and adaptation of the assessment system enable a district to enhance learning for all students. The GDI module sets the table with up-to-date, accurate student data. If you would you like to learn more about frequent updates through GDI, please refer to the K-12 Importation Instructions or contact your Field Services Coordinator at 877.358.7617.

Community Assessment and Item Banking Initiative

ATI is answering educators’ need for valid, reliable, and fair items and assessments for use in evaluating the instructional effectiveness of teachers in areas not tested on statewide assessments with the Community Assessment and Item Banking Initiative. The initiative provides a way for users to develop and to share within and across districts both assessment items and assessments. As part of the initiative, ATI is assisting in the preparation of educators to develop high-quality items and is providing services related to assessment development and data analysis. Participating districts and charters are provided access to a continually growing repository of shared, district-written, high-quality items and customized assessments in areas such as physical education, music, art, social studies and foreign languages.

It Takes a Village

As the newest member of ATI’s Field Services team, I’ve discovered that “It takes a village to raise a child” doesn’t apply exclusively to raising children. The team members from all departments, I have discovered, are truly committed to creating a village here at ATI to help Galileo users at all levels – clients, students, parents and co-workers. The scope of knowledge and expertise that I’ve encountered as I train to represent ATI to districts and charter school organizations is impressive, from the members of the Educational Management team, to the Professional Development and Field Services department members. I have been invited to participate in trainings, sales presentations, webinars and conferences. Without exception, every individual has been happy to help me understand the huge body of information available about education, assessments, psychometrics, instructional effectiveness and the entire Galileo Online system. These professionals are truly committed to ensuring that we all have access to the information and expertise to help us raise this and future generations, to help them all achieve at higher and higher levels, so they too, can continue to raise their children in a global village.

- Nancy Auslander
Field Services Coordinator

Clarifying the Role of Text Readability Scores in Benchmark and Formative Assessment

I want to take this opportunity to address some questions that have come up regarding readability metrics related to the texts appearing in Galileo K-12 Online assessments. One of the interesting aspects of readability metrics is the attempt to quantify, mathematically, the literary and informational expression found in text. These measures provide a handy guideline for determining if the structural complexity of a text is relevant to typical readers at a grade level. However, there are some limitations to the numerical categorization of text, which will be covered in this blog post.

First, it’s important to address the purpose of testing students using text-based items.

The ATI Assessment and Instructional Design team’s goal in developing the texts and items is to provide a basis for clear measurement of students’ abilities to comprehend and analyze text in a way that informs educators on each individual student’s progress toward standards mastery. To make sure that all students being assessed are challenged, the most effective strategy is to include texts and items of variable difficulty. This means that some items are achievable by all students, some by most students, and some only by students who are proficient in the expectations required to complete those tasks. Note that Common Core State Standards information recommends a broad text readability range in measuring performance on Common Core State Standards-aligned assessments.

All texts used in Galileo Online since 2002 have been analyzed using the Flesch-Kincaid Readability measure. 

The Flesch-Kincaid formula:

A Flesch-Kincaid score can be calculated manually or by utilizing the Microsoft Word Spelling and Grammar tool with the Readability option enabled to get the score. As can be seen in the formula, the number of syllables/words and words/sentences in an analyzed text are the key determiners of the grade level rating of a text.

Below is an example of the paragraph above, analyzed using Flesch-Kincaid, and then revised using an understanding of the nature of the formula.

A Flesch-Kincaid score can be calculated manually or by utilizing the Microsoft Word Spelling and Grammar tool with the Readability option enabled to get the score. As can be seen in the formula, the number of syllables/words and words/sentences in an analyzed text are the key determiners of the grade level rating of a text. Flesch-Kincaid Grade Level as written: 14.3 (post high school)

A Flesch-Kincaid score can be calculated manually. It can also be found utilizing the Microsoft Word Spelling and Grammar tool. This is done with the Readability option enabled to get the score. The formula calculates the number of syllables/words and words/sentences in an analyzed text. These are the key determiners of the grade level rating of a text. Flesch-Kincaid Grade Level with sentences simplified: 8.1 (middle school)

A Flesch-Kincaid score can be found by hand. It can also be found using the MS Word Spelling and Grammar tool. This is done with the review option set to get the score. F.K. uses the number of syllables/words and words/sentences in a text. These are the key parts of the grade level rating of a text. Flesch-Kincaid Grade Level with sentences simplified and polysyllabic words reduced: 4.0 (elementary school)

As can be seen from the three examples above, the readability number, while useful, is simply an analysis of the structure of the passage, not the appropriateness of content. Edits made the text simpler for less-accomplished readers because they are more familiar with simpler sentences and words, but did not simplify the topic. Sometimes compound sentences and polysyllabic words are unavoidable, even at earlier grades. The presence of these complex sentences and more difficult words will affect the readability statistics of the text, but may or may not affect the students’ ability to comprehend the text. For example, longer words that the students are familiar with will not make the text more difficult to read even though the readability formula will interpret them that way. Awareness of these limitations makes this quantitative measure simply a guideline to avoid overly complex sentence structures and longer, unfamiliar words.

One question that is often asked is the meaning of the decimal points. A text with a score of 4.4 is not intended to be aligned to the fourth month of fourth grade, but simply to be  structurally more complex than a text with a score of 4.2.

In addition to Flesch-Kincaid readability measures, Galileo K-12 Online incorporates The Lexile® Framework for Reading developed by MetaMetrics® into our benchmark texts. The model used in Lexile measures has some similarity to Flesch-Kincaid. More information on the formula used to compute Lexile measures can be found at the MetaMetrics Web site.  While some grade-level equivalent information is identified, MetaMetrics has made the following statement on grade equivalence:

There is no direct correspondence between a specific Lexile measure and a specific grade level. Within any classroom or grade, there will be a range of readers and a range of reading materials. For example, in a fifth-grade classroom there will be some readers who are ahead of the typical reader (about 250L above) and some readers who are behind the typical reader (about 250L below). To say that some books are "just right" for fifth graders assumes that all fifth graders are reading at the same level. The Lexile® Framework for Reading is intended to match readers with texts at whatever level the reader is reading.

We all recognize that the quantitative structural scores of text are just one aspect of readability. The other aspect is the qualitative element of the topic and ideas presented therein. ATI’s content specialists and each district’s teachers choose topics and themes that are meaningful to the students. We encourage all of our district partners to engage in the assessment review process, which allows the replacement of items and texts to better serve the needs of each district’s students. ATI develops items aligned to texts to serve a broad range of student abilities, and is glad to work with our partners to make sure that the texts presented in the final version of custom assessments have the support of the educators who will use them to inform instruction.

Further questions on readability and text appropriateness? Please comment below.