As she flipped to the open-response questions on the physics MCAS last week, Abby Cooney felt a wave of relief.
“I was like, ‘Oh, I already know these,'" she said. "In the moment, it was really nice.”
Cooney, 15 and a first-year student at Brookline High School, said she’s pretty comfortable with the material in physics class. But her relief on test day stemmed from having walked through, she says, half of the test’s open-response questions with her teacher and classmates just a day earlier.
Those repeat questions raise concerns, especially if they were widespread. Experts say if too many questions were available ahead of time, it can make the meaning of a test's results a lot less clear. The episode comes at the end of an already chaotic school year, during which many states sought federal waivers from mandated standardized testing only to be denied.
As Cooney recalls it, three of the test’s six open-response questions were drawn verbatim from ten-year-old practice tests the state had previously published for review. “All of them were exactly the same — same pictures, same ABC order, as far as I can tell,” she said. After testing ended, Cooney says her class was abuzz with talk of the repeats.
WBUR spoke to two other Brookline students who also say they recognized multiple questions from practice exams. The students requested anonymity for fear of violating the state's policies regarding testing security.
Ed Wiser, the science curriculum coordinator at Brookline High, never saw test materials himself — but he heard from his students and reported the alleged repeats to the state’s Department of Elementary and Secondary Education last week. He said he simply wanted to make clear to the state that his students and teachers hadn’t done anything wrong. Several other educators told WBUR they heard similar reports from students.
State officials declined to comment on potential repeated questions, although this year's testing has concluded. But they confirmed that items from this year’s science MCAS tests will not be published after the school year.
In a January memo sent to district superintendents, Jeff Riley, the state’s education commissioner, stressed the need to administer MCAS exams to give the state “critical insight into academic losses that need to be addressed” after over a year of disrupted learning.
But one testing expert says if this year’s questions were available to students and teachers before test day, it poses a threat to the validity of the results.
"It just gave my whole class leverage. I think it’s definitely unfair."Abby Cooney, a first-year student at Brookline High School, on repeat MCAS questions
As executive director of the New Hampshire-based Center for Assessment, Scott Marion works with education officials and test publishers to develop and administer effective standardized tests.
As test-makers write new questions, they submit them to field testing and fine-tuning — something that the pandemic’s uncertainties made practically impossible. So Marion and the Center's staff encouraged test-makers to use shortened exams this year, drawing tried-and-true questions from their “item bank."
But Marion said it is critical that most if not all of those questions weren't available verbatim beforehand, given that students will have a "differential" level of access to them. In his view, what happened in Abby Cooney’s classroom looks like an exaggerated version of the norm in which some students get detailed test prep and others don’t.
“The inference that we want to make is, if you get more questions right than me, you know more about this content than I do. But if you got more questions right than me because you had the answers beforehand it [means] I can’t carry that inference,” Marion said.
For all of Cooney’s relief in the moment of her assessment, she agreed.
“At my school, teachers go through past tests, picking out questions — maybe teachers at other schools aren’t doing that,” Cooney said. “It just gave my whole class leverage. I think it’s definitely unfair.”
Achieving a passing score on one of MCAS’s four science, technology and engineering tests is part of the state’s competency determination — which has been modified for the classes of 2020 and 2021 — and is therefore a graduation requirement.