Sunday, November 29, 2020

Our tests are the problem, not the students

The “progressive” physics teacher

You have just finished teaching a unit of physics. Great fun was had by many. Many struggled. Everyone seemed to see some level of success. There was a lot of feedback and interaction between students and each other and students and you. They explored, explained, solved, presented, discussed, and applied. You have a pretty good idea who is going to do well on the test and who won’t. You usually do, because you have a pretty good idea every time. You are fairly well dialed into your classes by the nature of your approach, including robust formative assessment.

You have students take the test. They all sit down with a pencil and a calculator and try to solve the same problems and answer the same questions. You have different versions for each class, but it’s mostly just different numbers and graphs. Some of them get extra time because they have IEPs. 

You are nice enough to allow retakes until students see success or give up. The retakes are different questions, which is a fair amount of work to create, but they are testing in the same way, so it’s a justifiable replacement grade. You are focused on growth, after all, not trying to punish them for their first attempts.

You use standards-based grading, or maybe a hybridized version of it. Much of what they do is performance tracked to content standards. Students can work on showing mastery of a standard until they achieve as high as they want. Again, it’s about growth, and this gives every learner a chance to continue working on something until they get it. You have an array of content assessment items for each standard ready for use as assessment.

I think that probably describes the system of a lot of good physics teachers. Those teachers are working hard to do what they think is best, creating a system in which students can thrive and grow at their own pace without the common overwhelming stress an unfairness of a more traditional system with one-off timed tests that just get entered into a grade book and everyone moves on.

These teachers tend to have classes full of fun inquiry and are well-liked by students. They are highly collaborative with other physics teachers in sharing and adopting ideas and tools for making the process make most sense.

The problems with this model

The root problem (not what this post is really about)

While I have great respect for the teachers who use systems similar to those as described above, and I have taken many great ideas from them, I believe the systems they use are highly flawed, enough so that I would label them ableist and unjust. 

The root problem is in the use of course content retention and mastery as the metrics for success. I have other blog posts about why I think the goal of standardized content evaluation through performance testing is an awful way to determine grades. I won’t rehash all of it. Summarizing it — students won’t remember a long list of content items after they leave our courses. There are no compelling reasons for them to do so, and the brain doesn’t even work that way. So why do we base their grades on their ability to do that? It’s not useful to them. The content details are not the greatest take-ways from a good course, so we should not be ranking and sorting them based on how well they remember those.

Let’s pretend like a lot of physics teachers disagree with my assessment of the root problem with our courses, because I think they probably do. They would possibly come back with obvious questions like, “You want me to teach physics but not grade based on how well they learn physics? What are we teaching physics for then?” 

That is good question. I also have a handful of blog posts about what I believe to be the answer to that. We should be teaching transferable skills using physics. Physics is just the fun delivery vehicle in which we happen to be experts at making opportunity for exploring the nature of science and the skills needed to be successful as a technical learner.

Let’s pretend like we don’t see eye-to-eye on the root problem, which I understand, and I think that can be okay for now. I’ll go forward with the rest of this assuming I can’t change your mind about what matters most as course take-aways and that grades should be based on those. There still are major problem with many of the systems that more progressive teachers are using, and they are problems inherited from traditional systems.

“Take as many attempts as you need.”

The assessment problem

All students show “mastery” in the same way on the same assessments by themselves without resources, and their grades are heavily based on their performance on them. That’s a problem in most classes. I believe it tends to stem from assessing the wrong things, but that might not be the only reason. Why is this a problem?

It’s a problem because it’s ableist. 

It’s a problem because it’s racist. It benefits the privileged the most. We are all aware of the socioeconomic disparities associated with race in this country. Any system that benefits the privileged much more than others is racist. 

It’s a problem because it doesn’t involve the skills needed to be successful in situations outside of the classroom. It’s just the warehouse model of “education” take from large colleges and universities and applied in smaller settings.

Are we trying to see how well they can do our assessments, or are we trying to measure how well they understand material? Creating opportunities for students to repeatedly assess in the exact same way rewards only the students who have the abilities to be successful on those type of assessments in those settings.

You know who tends to do well on those assessments? The kids without ADHD. The kids who don’t have trouble writing and drawing. The kids who don’t have anxiety. The kids with families who can afford tutors. The kids who don’t have to work jobs to help support their families so they have time to practice those types of assessment. The kids without stresses of life not knowing if they will have food to eat, or a roof over their head. The kids in the schools with the most wealthy populations where they have all of the resources and teachers and experience.

Why do we do it then?

Why do we insist on using the same individual assessments for each student? I think there are many answers. 

Many would say because that’s how its done in college. But colleges do it because it is economical. It greatly reduces person hours in grading. They do it to save money, not because it’s better for students.

Others would claim it’s for data. The physics education research (PER) community values all students taking the same assessments, because they have universal data to use. They can get survey data from students too, about how well the students feel they learned and performed. They can compare groups and subgroups. This ignores the fact that making decisions based on unjust methods for determining outcomes is a horrible idea. The survey asks students about how well they think learning something and being assessed in a pigeon-holed manner went for them.

Others might claim it’s fair. Every student has the same opportunity so they take the assessments. Anything else would be unfair. This ignores the fact that every student is not the same. They have different  abilities and interests, so it’s not the same opportunity. Nothing about what they do for class up to the day of the test is the same for each student, including the assessment.

All of those answers for why we do it are rooted in the assumption that it makes sense for every student to approach learning and assessment in the same manner. That is not true, nor is it just. But we still do it, and what we end up doing is ranking and sorting them by social class.

“Let’s talk about how you might go about this.”

What do we do instead?

Give students the opportunity to explore your subject in the way that makes most sense for them, and assess them accordingly. I have a collection blog posts about how one might take constructivism to the level of meeting students at their abilities and interests. I’m not sure if any of the ideas are best or even good, but they are at least attempts to create a system in which every student can thrive and succeed. They still require great effort and are designed to challenge each student. They incorporate student interest and choice, with protocols and opportunity built in for teacher feedback to make sure the methods and assessments are relevant and robust.

If my ideas don’t seem that great, then ask your students ”How would you like to explore this topic? How would you prefer to show mastery?” You might get some blank stares if they aren’t used to being asked that. But provide them with some guidance and examples, and help them understand their own interests and abilities, and you might just stumble on a system that works for every student.

The job as the teacher can then become one of a coach, with constant feedback to make sure what they pursue is appropriately challenging and relevant. They will need a teacher with expertise in the subject, and they will need help developing and using methods for evaluation, for evaluating themselves and for evaluation by their peers and teacher. 

In summary

I believe we focus on assessing the wrong things. This tends to push us to assess in the wrong way — the same way for each student. Whether or not most people agree with me about what we assess, and whatever we decide to assess, most of us don’t do it in a fair and just manner. In order to accomplish this it is important to meet students where they are, both in terms of ability and interests. I don’t have the answers for this. I have some systems in place that might be a good start, but the real answers are in the students.

Scott @BrunnerPhysics


  1. I've come full circle on this Scott. Began teaching in urban lower-income LA for 13 years, then the last 21 in higher demographic San Diego in a crazy competitive district. But I agree, we focus on assessing the wrong things. I'm back to recreating a skills/improvement based system. But families are like "so I am earning 4/5 and 5/5s is that an A?" Trying to find peace with a method to coordinate skill/content mastery individually and keep it student improvement focused but it comes back to them and 'A's. Still looking for ideas from others. Old dog = new tricks!

    1. Thanks for your thoughts, Bob. Translating skills growth system evaluations to grades is probably the hardest part about the whole system design process. I have found that developing evaluation tools with student input helps, and having them evaluate themselves. This is actually unavoidable if they are all doing different things, like designing their own projects. I just don’t have time to fully write their rubrics for achievement for that many projects. I also use digital portfolios with pretty clear checklists for requirements for demonstrating skill mastery. It’s all a work in progress. Students get annoyed that they have to reflect sometimes, but in the end they value the process and it’s fairness and focus on growth of useful skills. Hopefully this experiment will fit nicely in the portfolio system. Good luck!