About ED OVERVIEW
Assessment Design Toolkit

About the Assessment Design Toolkit

Welcome to the Reform Support Network (RSN) Assessment Design Toolkit (Toolkit). The Toolkit includes videos and supplemental materials to help teachers write and select well-designed assessments. Though the primary audience is teachers and principals, district and State leaders can use the Toolkit to design professional development opportunities. The Center on Standards & Assessment Implementation (CSAI) also hosts a version of the Assessment Design Toolkit here. In addition to Toolkit resources, the CSAI page will highlight and support States (and interested Districts) that customize the Toolkit. Please visit the CSAI Assessment Design Toolkit page to learn more about how States and Districts adapt and use the Toolkit.

Scope of the Toolkit

The Toolkit includes 13 "modules" divided into four parts: (1) key concepts, (2) five elements of assessment design, (3) writing and selecting assessments and (4) reflecting on assessment design. The modules address how to plan, write and select well-designed assessments. The modules do not cover how to use assessments to measure student growth.

Why the RSN Developed the Toolkit

The RSN developed the Assessment Design Toolkit to respond to State requests for help to improve assessment literacy among the teacher corps. Assessment literacy is important for all teachers, especially for those teachers of non-tested grades and subjects who do not have State assessments to help them measure student growth.

After you have viewed the modules, repurpose the Toolkit

Repurpose the Toolkit

Scroll to the bottom of this page to learn how to make the Toolkit your own.

View the Toolkit Modules

Explore the modules via the menu below. We suggest that you print four key documents before you begin. Then start with the orientation module, and work through the modules at your own pace.

Print before you begin
Assessment Blueprint Assessment Blueprint Example Assessment Checklist
Start Here
Orientation: How to Navigate the Modules
Part 1: Key Concepts Part 2: Five Elements of Assessment Design Part 3: Writing and Selecting Assessments Part 4: Reflecting on Assessment Design
1. Introduction to Assessment Design 4. Alignment 9. Selected Responsive Items 13. Reflecting on Assessment Design
2. Purposes of Assessment 5. Rigor 10. Constructed-Response Items
3. Types of Assessment Items 6. Precision 11. Performance Tasks
7. Bias 12. Portfolio Assessments
8. Scoring Make it your own

Introduction

The modules are designed to empower educators, pre-kindergarten through grade 12, to become master assessment designers so they can do an even better job of helping their students learn the course content throughout the school year. Each module includes a summary, a video and supplemental materials. You can view the videos alone, in teams or with entire faculties.

The run time of the videos is shorter than the time we suggest you take to learn the videos' content. You can pause the videos whenever you need an extra moment to think about what you have learned, and you can rewind them to revisit key concepts.

This module will orient you to how the modules work.

Return to top

Part 1: Key Concepts

Introduction

Measuring what students know and can do is an essential part of teaching, and, like much of teaching, designing assessments that measure what we want them to measure is sophisticated work. By completing these modules, you will be able to plan, write and select assessments in which you are confident and that give you a clear sense of what your students are learning.

By the end of this module, you should be able to define several key terms and concepts that are foundational to this series of modules—including what the modules identify as five elements of assessment design and validity and reliability—and explain why teachers should focus on these five elements of assessment design and not the statistical concepts associated with validity and reliability as researchers and test makers apply them. You will also be able to explain the purpose of two tools that the Toolkit introduces: an assessment blueprint template and an assessment blueprint filled out with an example.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

Now that you've finished the module and met the objectives, let's move on to others.

Return to top

Introduction

Now that you've had a chance to view the introductory module and learn or get a refresher on some important terms, let's discuss the multiple purposes of assessment.

By the end of this module, you should be able to identify the different purposes of assessment and understand how to use the assessment blueprint to document the primary purpose of an assessment.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

You are well on your way to becoming a master designer of assessments. The next module focuses on the types of items you can use when you design assessments, such as selected response, constructed response and performance tasks. Let's move on to this or other modules of your choosing.

Return to top

Introduction

This module focuses on the types of assessment items teachers design all the time.

By the end of this module, you should be able to identify, describe and tell the difference between three different types of assessment items.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

We've finished our discussion of assessment items. Now that we've reviewed or built our understanding of the purposes of assessment and the types of assessment items, we're ready to apply that knowledge to the five elements of assessment design that we discussed in the first module.

Return to top

Part 2: Five Elements of Assessment Design

Introduction to Part II: Five Elements of Assessment Design

In the introductory module, we introduced the concept of five elements of assessment design alignment, rigor, precision, bias and scoring, and suggested that if the assessments you write or find elsewhere address these five elements effectively, those assessments stand a great chance of having an appropriate level of validity and reliability for use in your classroom. In this and the next four modules, you will learn how to recognize the effective use of these five elements and be well on your way to writing or selecting well-designed assessments for your students.

Introduction

The focus of this module is the first element of assessment design—alignment.

By the end of this module, you should be able to define alignment for the purpose of the modules and explain why it is important. You should also be able to explain how to "unpack" a standard to understand its content and use the assessment blueprint to document the skills embedded within it.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

Now that you have a good understanding of one element of assessment design—alignment—let's move on to the second of the five elements: rigor.

Return to top

Introduction

Rigor. That's what you bring to your instruction every day. You make sure that your students work hard and think hard each and every lesson. You also want to bring rigor to your assessments, which is why rigor is an element of assessment design. Rigor is the focus of this module.

This module has several goals. By the end of this module, you should be able to (1) define what rigor means for the purpose of these modules; (2) use the verbs in standards and other tools that you have available to you to design assessment items that match the cognitive complexity of the relevant standards; (3) explain why assessments with an appropriate level of rigor also measure for a range of student thinking and understanding; and (4) use the assessment blueprint to document the level of rigor of each skill you are measuring in assessments you write or select. The level of rigor of this module is high!

To understand the key concepts in this module, you will need to reference two of the supplemental materials: Bloom's Taxonomy and Webb's Depth of Knowledge Chart. You can click on the links below to access them.

Supplemental Materials:

Bloom's Taxonomy
Webb's Depth of Knowledge
Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

The Rigor video and supplemental materials use Bloom's Taxonomy as the primary tool to determine rigor level. For those who prefer to use Webb's Depth of Knowledge (DOK) as a measure of rigor, the video and accompanying supplemental materials linked under the "Alternative Rigor module" header below provide an alternative version of the Rigor module that uses Webb’s DOK levels.

Alternative Rigor module: Webb's DOK Levels

Video
Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

Now that you have a good understanding of two elements of assessment design—alignment and rigor—let's move on to the third of the five elements: precision.

Return to top

Introduction

You've now learned about alignment and rigor, the first two of five elements of assessment design, and you are well on your way toward writing or selecting well-designed assessments for your students.

The focus of this module is the third element of assessment design—precision.

By the end of this module, you should be able to describe what precision means for the purpose of these modules and make an imprecise item more precise. The video in this module is shorter than many other modules. We do, however, provide additional examples of how to design precise selected- and constructed-response items in Part III of this Toolkit.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

You have completed modules on three of five assessment design elements. Let's move on to the fourth element—bias.

Return to top

Introduction

As you know, teaching is hard work. Designing assessments that measure what you want them to measure is an acquired skill. There is a knowledge base for great assessment development, and your acquiring that knowledge and putting it into practice will help you draw accurate conclusions about your students' learning. Knowing and putting into practice five elements of assessment design will go a long way toward meeting this goal.

You have been working your way through a series of modules to develop the knowledge you need to design great assessments. You've already learned about three of the elements—alignment, rigor and precision. The focus of this module is the fourth element of assessment design—bias.

By the end of this module, you should be able to describe what bias means for the purpose of these modules and detect potential bias in assessment items.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

You have completed this module on bias, which all assessment designers should avoid when they write or select assessments. In the next module, we take up the fifth element of assessment design—scoring.

Return to top

Introduction

Grading papers and quizzes, assigning grades and scores to projects and performance tasks. Teachers do these things every single day. And so the fifth component of assessment design—scoring—will likely be the most familiar to you. This module, we hope, will help make you an even more effective scorer of your students' work and demonstrate for you how the more effective your scoring practices are, the more likely your assessment results will be reliable.

By the end of this module, you should be able to define what scoring means for the purpose of these modules and explain how and why you should use well-designed tools, such as answer keys, scoring guides and rubrics, to score many assessments. You should also be able to explain what distinguishes one tool from another.

Mastering how to write a well-designed rubric is beyond the scope of this module. We recommend the following resources and examples of well-designed rubrics to help you continue your learning about rubrics. (Available June 2015)

Partnership for Assessment of Readiness of College and CareersPerformance Level Descriptors: Mathematics
These rubric describe four levels of student performance for all math Common Core standards grades 3-12.

Kansas State Department of Education. "Assessment Literacy Project."
This resource includes a video and supplemental to help educators learn about the different types of rubrics, how to recognize a well-designed rubric and how to design and score rubrics.

New York State Education Department. "Teaching Is the Core Assessment Literacy Webinar Series – Part 5: Rubrics and Other Scoring Methods."
This resource includes a video and associated PowerPoint presentation to help educators learn to write and select rubrics that match intended learning outcomes and to check for consistency among scorers.

Smarter Balanced Assessment Consortium. "General Scoring Rubrics Mathematics."
This holistic scoring rubric describes levels of student performance on 4-, 3-, 2-, and 1-point mathematics assessments items.

Student Achievement Partners. "Scoring Rubric for Text-Based Writing Prompts."
This rubric describes four levels of student performance for text-based writing prompts. It is primarily for writing about a text, but you could use a subset for another writing task.

Partnership for Assessment of Readiness for Colleges and Careers. 2014. "Grades 6-11 Condensed Scoring Rubric for Prose Constructed Response Items Research Simulation Task and Literary Analysis Task."
These rubrics describe five levels of performance for PARCC's research simulation, literary analysis, and narrative tasks.

Elk Grove Unified School District (EGUSD). "K-12 Rubrics."
This resource includes writing rubrics for every grade level aligned with college- and career-ready standards, including informational, opinion, narrative writing rubrics and research rubrics. (Note that rubrics are the same within K-2, 3-5, and 6-12 with the exception of the standards).

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

We've finished our discussions of five elements of assessment design, including scoring. If you address these five elements as you design assessments for your students, you should feel confident that they measure what you want them to measure and that the results are—to use those terms we discussed in the first module—valid and reliable. Now let's move on to discuss the types of items you write when you design your own assessments.

Return to top

Part 3: Writing & Selecting Assessments

Introduction to Part III: Writing & Selecting Assessments

Since you started your career as a teacher, you've been designing and administering assessments. You've probably written multiple-choice questions, asked students to fill in the blanks with correct answers, requested short responses to prompts or designed performance tasks, such as essays, or, if you are a physical education teacher, perhaps dance routines, or, if a science teacher, perhaps experiments.

The next series of modules focuses on the types of items and assessments you can write or select: selected-response items, constructed-response items, performance tasks and portfolio assessments. For some of you, this will be a review. For others, we might be covering new ground. Whether you are a novice or experienced assessment developer, we believe there's enough helpful information here to help you make even more informed decisions about when to use specific types of items and assessments and how to improve the assessments you've already been designing so that the results of your assessments truly reflect all that your students are learning.

Introduction

We're going to start with selected-response items, which, if done well, are a lot more difficult to write than your students probably realize.

By the end of this module, you should be able to define what a selected-response item is, identify the benefits and challenges selected-response items present, know the four parts of a well-designed multiple-choice item and use the assessment blueprint to organize items in an assessment.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

When you're ready, let's move on to the next module. It focuses on constructed-response items.

Return to top

Introduction

In the last module, you learned about selected-response assessment items and, in particular, multiple-choice items. The module explored how to write selected-response items, when to use them, and what their strengths and weaknesses are for classroom assessment. You learned about how to write effective multiple-choice questions, right down to the level of how to create meaningful distractors.

The focus of this module is a different type of item—the constructed-response item. Again, you have probably written many constructed-response items. Our hope is that this module will help you take what you know and can do to the next level and that the completion of this module will be one more step you take on your journey to becoming a master assessment designer for your classroom.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

When you're ready, let's move on to the third type of item: performance tasks.

Return to top

Introduction

You've looked at two types of assessment items—selected response and constructed response. Now let's move on to performance tasks. You likely design simple and complex performance tasks with regularity. They provide you with tremendous flexibility and they allow you to check your students' higher-level thinking.

By the end of this module, you should be able to define what a performance task is, list a variety of performance tasks, identify the benefits and challenges of performance tasks, know that there is a "what-who-how" framework that you can use to design performance tasks, and use the assessment blueprint to design assessment items.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

You now have completed modules on the three types of assessment items. We hope you will use what you've learned when you write or select your next assessment. Don't hesitate to return to the modules or associated materials if you need to brush up on an item type. Now, let's move on to the next module on portfolio assessments.

Return to top

Introduction

You have finished watching videos about three different types of assessment items. Next we will discuss portfolio assessments. Portfolios are not a particular type of assessment item; they are full-bodied assessments, capable of housing a collection of other assessments that come together as a powerful display of student learning. Portfolios should be part of every masterful classroom assessment designer's toolkit.

By the end of this module, you should be able to define a portfolio assessment, distinguish between its two types, identify the benefits and challenges of using portfolio assessments, and know that there is a "what-who-how" framework that you can use to design them.

Supplemental Materials:

Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

You have completed the module on portfolios and are ready to move on to the final module of this series on assessment design. You are very close to addressing, and hopefully mastering, all of the objectives that we have established for this series of modules.

Return to top

Part 4: Reflecting on Assessment Design

Introduction

You have invested a lot of time in mastering the skills of assessment design. You have learned about the purposes of assessment, five elements of assessment design, the three different types of assessment items and portfolio assessments. You've learned how to use an assessment blueprint to help you design assessments.

In this final module we focus on a tool that you can use to evaluate an assessment you have written or selected. We simply call it the "Assessment Checklist." It focuses you on the five elements of assessment design, which help you design classroom assessments that have an appropriate level of validity and reliability.

By the end of this module, you should be able to use the Assessment Checklist to help you determine whether an assessment you design appropriately addresses the five elements of assessment design featured in this series of modules.

Supplemental Materials:

Assessment Checklist
Worksheet
Note-Taking Template
Narrator's Script
PowerPoint Deck
List of Sources

Conclusion

Congratulations are in order! You have officially completed all 13 modules in the Assessment Design Toolkit and mastered most, if not all, of its content. Your hard work has paid off, and you should feel even better prepared than ever to write and select assessments that measure the knowledge and skills you want them to measure. Your students will benefit greatly from your hard work and assessment expertise. Good luck as you continue your career as a teacher. Among the many complex skills you will demonstrate in every unit of instruction, count among them the capacity to write and select well-designed assessments.

Return to top

How Can Educators Use the Toolkit?

The Toolkit is designed so that educators can repurpose its module(s). For example, the modules can be part of a robust professional development plan. State and district leaders can post the modules to online learning platforms for teachers to access independently to fit their schedules. Teams of teachers, Professional Learning Communities and school departments can use the modules to improve assessment literacy together in groups.

Read Vignettes of How to Use the Toolkit

State, district, and school-level educators will use the modules in different ways. To learn about how the Toolkit might bolster professional development at each of these levels, you can read three vignettes that describe how a State, district and school leader might use the modules. You can also read a list of ideas for how to use the Toolkit developed by State, district and school leaders who attended a convening about assessment design in May 2015.

Vignette of a State Leader
Vignette of a District leader
Vignette of a School Leader
List of Ideas for How to Use the Toolkit

How Can Educators Repurpose the Toolkit?

Repost Hyperlinks and Videos

We encourage educators to repost the link to the Toolkit and any links within the Toolkit to public forums, websites, or online communities. For example, a district leader leading an in-person professional development session on assessments might include a link the Toolkit in his or her training resources so that educators can follow up with independent study.

Educators can also embed any one individual video or a subset of videos on a different Web page. To embed a video on a blog or webpage:

  1. Hover over the video you wish to embed
  2. Right click (for PCs) or CTRL + click (for Macs)
  3. Select "Get embed code"
  4. Copy the code
  5. Paste the code into the new Web page

Download the Supplemental Materials for Later or Offline Use

Select the links below for zipped files that contain editable Microsoft Word & PowerPoint documents or zipped files that contain PDF 508 Compliant documents.

Part I: Key Concepts

Part III: Writing & Selecting Assessments

Part IV: Reflecting on Assessment Design


An earlier version of this document was developed under the auspices of the Reform Support Network, with funding from the U.S. Department of Education under contract #GS-23F-8182H. This publication features information from public and private organizations and links to additional information created by those organizations. Inclusion of this information does not constitute an endorsement by the U.S. Department of Education of any products or services offered or views expressed, nor does the Department of Education control its accuracy, relevance, timeliness or completeness.



   
Last Modified: 07/17/2015