Educator-led Teams Develop Rubrics

Our review rubrics are developed by educators from across the country with deep content knowledge and expertise in conducting materials reviews. Each rubric is developed following these steps:

  • LEARN – Rubric development teams absorb the findings from a national learning tour of content experts and educators
  • EXAMINE – Teams examine existing rubrics from the field
  • COLLABORATE – Educator teams develop content-specific rubrics for EdReports review processes
  • FINALIZE – With input from experts in the field, including teacher membership organizations, state departments of education, researchers and leading policy voices, rubrics are finalized

The educators on rubric development teams also develop detailed evidence guides that accompany the rubrics. The guides can also provide support as you read the reports or incorporate our rubrics as part of your own review process.

Educator-led Teams Review Materials

Each report found on EdReports.org represents hundreds of hours of work by educator reviewers.

Team Composition Reviewers work in teams of four to five people using the review rubrics, evidence guides, and key documents to thoroughly examine their sets of instructional materials. These team members are expert educators that are well versed in the standards and have completed/passed/been vetted through a thorough application and selection process.

Time Commitment Reviewers receive more than 25 hours of virtual and face-to-face training on the EdReports.org review rubric and process and meet weekly over the course of several months to develop final reports. Reviewers independently gather evidence for specific indicators each week before meeting in teams to share evidence, come to consensus on scoring, and write the draft that ultimately is shared on the website.

Process All team members look at every grade and indicator, ensuring that the entire team considers the program in full. Teams touch every page of the series and look for the presence of standards, how well the standards are sequenced, the depth with which standards are included, and other important characteristics of high-quality instructional materials. These comprehensive reviews are evidence-rich. No other review process provides as much detail about how materials meet or do not meet the criteria, including specific examples from the programs.

Advisory Panels Each content area has a dedicated set of volunteer advisors—researchers, professors, and content leaders in the field—that provide guidance to the review teams. These advisors are available to help educator review teams when they encounter examples within the instructional materials where they need other experts to weigh in on alignment or quality.

Publishers All publishers are invited to provide an hour-long orientation to the review team. The review teams can also ask the publishers clarifying questions about their program. Publishers may also post two sets of information on the EdReports.org website: a response to the educator report and a document that includes relevant any background information or research on the instructional materials.

Rubric Design

The EdReports.org’s rubric supports a sequential review process through three gateways. These gateways reflect the importance of standards alignment to the fundamental design elements of the materials and considers other attributes of high-quality curriculum as recommended by educators.

Step 1 Step 2 Result: Alignment Rating
The reviewed materials Meet Expectations for Gateway 1 AND Meet Expectations for
Gateway 2
Meet Expectations
BUT either Partially Meet or Do Not Meet the expectations for Gateway 2 Partially Meet Expectations
The reviewed materials Partially Meet Expectations for Gateway 1 AND Meet Expectations for Gateway 2 Partially Meet Expectations
BUT Partially Meets expectations for Gateway 2 Partially Meet Expectations
BUT Do Not Meet the expectations for Gateway 2 Do Not Meet Expectations
The reviewed materials Do Not Meet the expectations for Gateway 1 If it Does Not Meet Expectations for Gateway 1 the material is not reviewed for Gateway 2 Do Not Meet Expectations

Advancing Through Gateways

  • Materials must meet or partially meet expectations for the first set of indicators to move along the process. Gateways 1 and 2 focus on questions of alignment. Are the instructional materials aligned to the standards? Are all standards present and treated with appropriate depth and quality required to support student learning?
  • Gateway 3 focuses on the question of usability. Are the instructional materials user-friendly for students and educators? Materials must be well designed to facilitate student learning and enhance a teacher’s ability to differentiate and build knowledge within the classroom. In order to be reviewed and attain a rating for usability (Gateway 3), the instructional materials must first meet expectations for alignment (Gateways 1 and 2).

Key Terms Used throughout Review Rubric and Reports

  • Indicator Specific item that reviewers look for in materials.
  • Criterion Combination of all of the individual indicators for a single focus area.
  • Gateway Organizing feature of the evaluation rubric that combines criteria and prioritizes order for sequential review.
  • Alignment Rating Degree to which materials meet expectations for alignment, including that all standards are present and treated with the appropriate depth to support students in learning the skills and knowledge that they need to be ready for college and career.
  • Usability Degree to which materials are consistent with effective practices for use and design, teacher planning and learning, assessment, and differentiated instruction.
X