During my first month as a teacher, I attended a professional development session on Depth of Knowledge (DOK). During the training, we sat in a cold meeting room and looked at assessment items, evaluating them for their levels, 1, 2, 3, and 4. Then, we planned out how we would teach students to get to the level of thinking required by those level-4 questions. I remember sitting at my table thinking, “But I am a reading teacher! What do I do when the text itself is what is really hard for my kiddos?” For all of its strengths, DOK is a content-agnostic framework. For literacy this is tough because the text plays such an important role in the demand any reading question places on students.

Fast forward many (more than I care to admit) years, and a new framework, one that takes the demands of reading a text into account, is finally available. The Framework to Evaluate Cognitive Complexity in Reading Assessments is based on the expectations of Criterion B.4 of the CCSSO Criteria for Procuring and Evaluating High-Quality Assessments. This criterion requires that summative assessments ensure that items reflect the range of cognitive demand required by the standards. Typically, this has meant using Bloom’s or DOK to get a list of verbs or a range of 1-4s on a test. But the Criteria suggest another option, a content-specific framework, which, for reading, is outlined as:

• The complexity of the text on which an item is based;
• The range of textual evidence an item requires (how many parts of text[s] students must locate and use to respond to the item correctly);
• The level of inference required; and
• The mode of student response (e.g., selected-response, constructed-response)

Achieve, a nonprofit dedicated to making sure every student graduates from high school ready to succeed in the college or career of his/her choice, took this guidance and used it as the basis for a new reading framework. To use the framework, assessment developers fill out a tool, rating each item on an assessment against three factors: Text Complexity (how complex the text the item asks about is), Evidence (how much of the text a student must use to respond to the item), and Reasoning (what kind of thinking is needed to respond to the item):

Then, developers look across the test form and ensure that there is not just a variety of one variable, but that there is a range of complexity within all variables:

But, what does it actually look like? How do we evaluate a question for its Text Complexity, Evidence, and Reasoning? Check out this example below:

The idea behind this framework is that reading items derive their complexity from a variety of features, so when we evaluate the items, we should note that. Instead of looking for one numerical score, each item now gets rated for all of the variables that contribute to its complexity. Text is placed above evidence and reasoning to signal its importance: the research makes clear that the complexity of the text is the key factor impacting student performance on assessment items. As such, the tool associated with the framework emphasizes text complexity to ensure that its importance is noted.

Ultimately, the CCSSO criteria require summative assessments to reflect the complexity of college- and career-ready (CCR) standards. Summative assessments should also capture the range of complexity within the standards, and this framework allows assessment developers to do just that. Previous evaluation methods focused on applying somewhat arbitrary numerical scores, but with the Achieve tool, each item is rated against ALL of the features of literacy that make it complex. This way, assessment vendors can look across the form to ensure that there are items that reflect a range of complexity, as well as items that are complex for a variety of reasons.

Though this tool was developed with the expectations of summative assessments in mind, we all know that assessment tools often make their way to instruction and professional learning for teachers, just as Bloom’s and DOK did for me as an early teacher. As I have worked with the tool, I have found myself thinking back to that training I attended my first month of teaching. What would a training that focused on this new framework have signaled to me as a new educator? What would I have inferred was “important” for my students? As opposed to the content-agnostic framework I learned about back then, text complexity is front and center in this tool–not lists of verbs and thinking skills. And that would have changed my instructional focus and my impact on my students. Materials, assessment, professional learning, and classroom instruction are all part of a cohesive system that creates strong learning experiences–and when it comes to teaching reading, the texts we select as worthy of students’ time must always be front and center.