Evolving EdReports’ Review Process to Meet Emerging and Future Needs

At EdReports, we believe, and research demonstrates, that instructional materials matter for student success. It mattered before the COVID-19 health crisis—and it will matter even more as schools transition into new teaching formats to ensure all students are getting the content they need to be college and career-ready. 

In the five years since we released our first reports we have seen the curriculum market evolve. Educators are demanding better materials, in part because they now have access to critical information about a program’s alignment and usability. To date, districts representing 11 million students have used EdReports when choosing materials. We have reviewed more than 90 percent of the known English language arts (ELA) and mathematics materials market and have catalyzed 30 publishers to improve their products as a result of our reviews. And materials are improving, with nearly half of K-12 ELA programs and one-third of mathematics programs now meeting expectations for alignment. Educators have more choices than ever before as they begin selection processes. 

We are witnessing in real time the critical need for coherent, standards-aligned curricula that support students, teachers, and parents to know what content will be taught two, four, and even eight months down the line. Digital specifications, and now the ability to use materials remotely, has never been more important. There is growing demand and need for curriculum to better support the needs of different students, especially as schools reopen.

In order to continue producing high-quality reviews to meet emerging and future needs, EdReports is announcing two sets of updates that will be made to our reviews of mathematics, ELA, and science reviews. 

1. This summer, EdReports will begin providing more information about the components of curriculum that support critical needs, such as remote learning and digital components, for more than 200 existing grade-level materials that meet expectations for alignment. This is in response to the growing need from school districts for more precise information and guidance about specific characteristics of materials (e.g., ease of implementation, student and parent facing components, availability of digital components) so that districts can learn more about these components during their curriculum adoptions.
    
2. EdReports is releasing revised review tools after more than five years of use in the field. As a learning organization, we knew it was time to reexamine the market and revise our indicators to provide more fine-grained information to help districts make choices about instructional materials. 

As we make these transitions, I’d like to take a moment to talk through what the changes are, why they were made, and how our decisions will better support districts selecting instructional materials. 

Supporting School Districts to Meet Emerging Curricula Needs

The COVID-19 crisis has illuminated inequities in curriculum availability, quality, and use within and across districts and schools. Districts without comprehensive curricula are scrambling to find multiple weeks’ worth of content to share either in packets or in online classrooms. Other districts may have quality, standards aligned curricula, but are struggling with implementation in a distance learning environment. 

As districts plan for immediate and long term needs, EdReports will provide more information about the components of curriculum that support distance learning on our website this summer. To accomplish this, we will conduct an audit of our current technology indicators, interview educators and district leaders to identify what additional information would be the most helpful, and work with publishers to gather and verify provided information to include with our reports. 

Revising Our Review Tools to Meet Long Term Needs

Since the very beginning, EdReports has solicited feedback of our reviews and tracked innovations in the market to ensure that the reports we create continue to be useful to educators—especially as materials evolve. To ensure that we’re providing educators with the information they need, we have revised our review tools across all content areas (which includes criteria, corresponding evidence guides, and the review process) for the first time. 

The revised tools will be used on reviews beginning in late spring/early summer 2020. Reviews take about six months to complete at which point they will be published on EdReports.org. 

We approached these revisions the same way we approach reviewing a new content area. We started with a listening and learning tour in the summer of 2019 where we spoke with classroom educators, districts, states, researchers, nonprofits, publishers, and other organizations to receive feedback on our current tools. We also conducted an internal audit of our tools and coordinated with experts and organizations with deep experience in working with students with learning differences and English Learners.

What’s Staying the Same with Our Reviews

Before we jump into changes, we want to focus on what will stay the same. Alignment to college and career-ready standards remains the foundation of our reviews. The report design and presentation will also look familiar to what readers are accustomed to now. 

Additionally, because our ELA foundational skills review tools were recently created, we have not updated them. Science reports will continue to have the same criteria and evidence for standards alignment (gateways 1 and 2) and will incorporate universal changes to usability indicators (gateway 3) to align with other content areas. 

The total points that determine whether a program meets, partially meets, or does not meet expectations for standards alignment will remain the same. We stand by the reports reviewed and released using the original tools; the information provided remains accurate and applicable even as future review processes adapt to meet evolving needs. There are no changes to determinations of alignment on materials that have already been reviewed.

What’s Changing in Our Reviews

Mathematics and ELA Standards Alignment Indicators (Gateways 1 and 2)

• Mathematics K-8: For our K-8 mathematics criteria, we are making language and scoring updates to indicators around coherence, and the connections between content and the mathematical practices. The total points available in each gateway has not changed, and the key elements of each criterion remain consistent. 
• ELA K-12: New indicators in K-8 ELA will help districts identify program “bloat”—when a program is challenging to implement and teach because there is more content than can be feasibly taught in a single school year. For K-5, we have also made some revisions to our treatment of foundational skills. The changes are to better align our comprehensive criteria to our new supplemental tool providing readers more information on critical components such as phonics and phonemic awareness. Similar to mathematics, the total points available in each gateway have not changed.

You can read in much greater detail about the specific changes in each content area in our tool revision frequently asked questions.

Mathematics, ELA, and Science: Adjustments to Usability and Design Indicators (Gateway 3)

A constant in our review process is that materials must first meet expectations for alignment to the standards (gateways 1 and 2) in order to be reviewed and attain a rating for gateway 3. Our updates to this gateway are a direct response to how much materials have improved since our organizational launch in 2015. Then, only 1 out of the 19 programs reviewed met expectations for alignment. Now, educators have more aligned programs to choose from. We realized that we could provide more information about product usability to support a district’s selection and implementation process. 

Primary changes to usability and design criteria address teacher supports and diverse student populations that require support for language acquisition and learner variance. New indicators were created to address the needs of English learners with an emphasis that English learner content and lesson objectives are grade-level/age appropriate and of equal rigor to help English learners meet grade-level standards. 

Other updates to usability criteria address technology issues such as data privacy and interoperability with learning management systems as well as supports to help teachers utilize the materials and ensure students are receiving standards-aligned content. These will be contained in a separate, unscored form. The form will be published this summer along with new information about aligned programs. 

Our aim with these revisions is to make certain that districts have the information they need to choose aligned materials that will also meet diverse students’ needs and address local priorities.

Read about revisions to usability and design criteria in detail in our frequently asked questions. 

What’s Next?

At EdReports we will always believe that our best report is yet to be written. As a learning organization committed to producing high-quality educator-led and evidence-rich reviews of instructional materials, we will continue to monitor research and work closely with both districts and publishers to inform any new tool revisions.

We know that districts, states, and teachers rely on our reports as critical resources to help them make important decisions. Our goal is to continue delivering trusted, research-based, credible information about programs so educators can compare options and select materials that work best for their students. 

The trust we have developed has come in large part from the consistency in our tool and the ability it brings to calibrate and compare materials; however, this cannot be an excuse to not innovate. 

We must be willing to learn and grow so that the field gets the answers they need to the questions they are asking. Ultimately, our mission is to do all we can to empower educators with the highest quality information so that they can make the best choices for their students!