Rebecca Kockler, Louisiana’s Assistant Superintendent of Academic Content, and her team at the Louisiana Department of Education had a clear mission: conduct curriculum reviews that would give districts information they could use to make smart decisions. After the Louisiana team selected the IMET as the tool that could get them that information, they began the process of putting it to use: building a review team, training reviewers, and conducting reviews.
The first task for the 2013-2014 review process was assembling the team that would do the work. As they had in previous review processes, they selected a group of educators from around the state who would bring their expertise and complete an initial review on every textbook. Supporting those educators were math and ELA content experts who would conduct the second set of reviews and help norm the outcomes.
Before anyone reviewed a single vendor offering, however, they conducted four months of training on Louisiana’s College and Career Ready Standards, the Shifts, the IMET, and what students would see on new, standards-aligned assessments. Kockler and her team wanted everyone to be clear about how and why these reviews would be different so that they could fully evaluate standards-alignment and provide simple and clear information for districts.
To become familiar with the IMET, the team looked for free samples of curriculum options online and then used them to practice conducting reviews. As they practiced, they found ways they could tweak the tool to make it a better fit for their review team and for the outcomes they wanted to achieve. For example, Louisiana wanted to provide more context around review findings to help their districts make decisions. “We knew, from our formal review that year, that nothing was fully aligned at the time,” said Kockler, “but we also knew that there were materials that were certainly better than others and saying simply ‘it’s either aligned or it’s not aligned,’ just felt unhelpful to districts.” Thus, Louisiana created a system of tiers: Tier 1 meant the option was fully aligned; Tier 2 meant it was partially aligned (these options had clear strengths, but were also missing elements), and Tier 3 meant unaligned.
After this preliminary work, it was time to conduct the reviews. Vendors had a six-month window to submit materials and the teams reviewed options on a rolling basis as they were submitted. Teachers worked in subject- and grade-band-specific groups of three to five teachers to conduct the reviews. Multiple cohorts of teachers would review the same materials to make sure their findings were similar. When findings were different among the teachers, the content experts would review the materials during a second round of reviews. When needed, the reviewers would discuss differences via follow-up calls to understand each perspective.
While the initial months of training for reviewers were certainly helpful, Kockler noted that the richest training occurred during the conversations between teacher reviewers and the content experts. There were math- and ELA-specific challenges related to misinterpretations of the IMET criteria, failure to cite concrete evidence, and a lack of clear rationales for findings; all of these issues were best discussed in real-time by the reviewers and content experts. The patterns that emerged from these real-time conversations helped Kockler and her team better tailor subsequent trainings for new reviewers that occurred in early 2015.
After finishing the reviews, the team’s next step was to convey the findings to curriculum vendors and build tools to help districts fill the consistent gaps found in programs. Continue to follow this series to find out how vendors reacted to the team’s findings and how Louisiana took action based on what they found.
