What Should an Assessment System Look Like?

Leila Morsy, Michael Kieffer, Catherine Snow
The developmental nature of reading means that diagnosing the reading comprehension ability of adolescents is more challenging than diagnosing reading comprehension among third graders. In particular, assessments should not only capture the increased sophistication of the reading task in the middle and high school years, but should also capture the specialization of the many tasks that comprise reading comprehension for older readers. Educators must think carefully not only about what the assessments they use consider “grade-level” text, but also how those assessments capture or fail to capture the processes involved in reading in different content-area classes.

Assessment do’s

Assessing comprehension is not a simple task. Moreover, a review of commonly used assessments confirms what many have suggested — no single test can serve all purposes. A system of assessment is required. Catherine Snow describes several requirements for such a system: the capacity to identify individual children as poor comprehenders (i.e., screening), and the capacity to identify subtypes of poor comprehenders for the purposes of differentiating instruction (i.e., diagnosis). She also indicates that an assessment system must reflect the outcomes in reading that educators believe are important — that is, educators who seek to promote critical thinking in their middle school students cannot use sixth grade assessments that emphasize only factual recall.

An assessment system should:

  • Include off-the-shelf standardized assessments, but would certainly not end with them.
  • Emphasize the ongoing formal and informal assessment (ranging from end-of-unit exams to writing conferences) that skilled teachers use on daily basis to guide their instruction.1
  • Provide relevant and up-to-date information for teachers at strategically useful times.
  • Requires the collaboration of educators at various levels (including district curriculum leaders, school literacy coaches, and classroom teachers) to ensure that they have assessments to meet each of their specific purposes (e.g., screening, diagnosis, monitoring progress) and that educators can interpret the results of these assessments in systematic ways that can inform instructional decisions.

Readers who want to learn more about creating such systems can look to Boudett, City, & Murnane (2005) for more advice, as well as Deshler, Palinscar, Biancarosa, & Nair (2007) for insight into how assessment data fits into a larger instructional plan for adolescent struggling readers.

Assessment don'ts

Educators will be able to identify their own examples of instances when schools and districts have used assessments without a clear sense of purpose, used assessments for purposes other than those for which they were designed, or used them without a purpose in mind at all. Here is a list of inappropriate uses of assessments:

  • Using last year’s state standards test scores to decide which students require which interventions. Under pressure to align instruction with the content assessed on state standards tests, some schools have decided that these tests should be the only basis on which instructional decisions should be made. Unfortunately, while these tests may tell educators who to teach, they rarely provide information about what to teach, especially in the case of reading comprehension.
  • Using tests designed to look as similar as possible to the state standards tests to decide which students require which interventions. This practice involves districts investing resources and time to create “interim” assessments that indicate how well students will do on the end-of-year state tests. Because these are designed to be as similar as possible to the summative end-of-year tests, these tests will provide only limited information about what skills to teach or about which interventions to provide to which students.
  • Providing teachers with a wide range of assessment choices without training or direction about which to use for what purposes. Choosing which assessments to use for what purposes is not a simple or straight-forward task. Moreover, administering and analyzing the data provided by reading assessments can require substantial teacher expertise. When districts provide teachers with boxes of assessments but little training on how to use them, teachers are likely to waste instructional time on testing without gaining useable information about students’ skills and needs.
  • Providing teachers with no standardized screening or diagnostic assessments in the interest of furthering their use of informal, formative techniques. Although teachers should be supported and encouraged to use a range of informal formative assessments to drive their daily instructional decisions, they should also be provided with tools with which they can gain measures of students’ skills that are objective, reliable, and comparable across classrooms and schools.

Avoiding these inappropriate uses of assessments and meeting the requirements above for creating a rational system of assessment requires thoughtful and reflective leadership on the part of the district and school officials in charge.

Morsy, L., Kieffer, M., and Snow, C.E. (2010). Measure for measure: A critical consumers’ guide to reading comprehension assessments for adolescents. New York, NY: Carnegie Corporation of New York.