Skip to content

Test analysis beyond scoring

At this point in the year, many teachers wonder if there is anything they can do to improve student scores.  The answer is simple but takes some explaining – solve the right problem and quit wasting time solving the wrong ones.

So, how do we figure out what the “right problem” is?  The first step in this process is to which students are underperforming, as well as the nature of the underperformance. Ed Directions coaches utilize what we call a “step one” analysis of student performance to begin digging at the root of the problem. This establishes who missed a specific question and what the incorrect response tells us about their work. An example of the tool we use is included below.

MULTIPLE CHOICE

QUESTION #                       CONTENT                                    TASK(S)

ANSWERSRESPONSE KEY# WHO ANSWEREDSTUDENTS WHO ANSWERED
A   
B   
C   
D   
E   

The chart above is set up to analyze a multiple choice question.  The first column refers to the bubble answer. The second column is our key to reference what you (as the teacher) think was the probable cause of breakdown, resulting in the incorrect answer. The Third column is a count of the number of students who chose that option as their answer on the test, and the final column is used to identify the names of the students who chose each answer.

The chart below is a filled in example of how this chart was used during one of our coaching sessions.  The response key shows all the probable reasons for missing the question. To be noted, none of the response keys rule out that a student was guessing to get an answer, except for “A” which is noted as, “unrelated answer” in which a student was most likely guessing or simply filling in bubbles.

Question #    4      Content: add together three two-digit numbersTask(s): add three two-digit numbers with an answer that requires carrying into the 10’s column
ANSWERSRESPONSE KEY# WHO ANSWEREDSTUDENTS WHO ANSWERED
AUnrelated answer16
BFailed to carry to 10’s41,4,8,12
CComputation incorrect in one’s column22,9
DCorrect answer33,10,11
EComputation incorrect in tens column25, 7

When the teacher of this particular class analyzed results they were able to interview the students with the student work in hand, and ask each student why they made the decision that they made. She was able to identify the flaw in the student processing, and addressed their specific error one support session.  This process allowed her to attack the right problem, and not use a one-size-fits-all support for students who obviously had different needs.

(An aside – in interviewing students five and seven she found out that they were doing the addition exactly the way they’d been taught – except – when they totaled the 1’s column they carried 1 one and then added the 10’s column. Unfortunately they were required to carry 2 to the  tens column. They had the ritual down (“X plus Y, carry my one) but their understanding of the ritual was not adequate)

In the sample social studies question included below, the teacher was confident the students would be able to identify the branch of government described by function in a reading.

Question #    11      Content: separation of powers – federal governmentTask(s): ID the branch of government (executive, legislative, judicial) described in a reading
ANSWERSRESPONSE KEY# WHO ANSWEREDSTUDENTS WHO ANSWERED
AWrong branch (J)0 
BWrong branch (E)95,8,9,11,12,14,15,16,18
CUnrelated answer37,13,19
DWrong level – state51,2,6,10,20
ECorrect answer (L)33,4,17

In this case, it was a little more difficult for the teacher to put initial reasons for missing the question in the response key. In the students who answered B or D it required additional strategies to determine that some of the students misread the question while others misread the reading that was provided. The incorrect performance was due not to problem of social studies knowledge but to a problem in the students’ inability to do critical or close reading.

In schools where we have coaches, teachers give sample questions every day until the week before the test to identify real problems causing students to underperform. Most of them will be short answer, multiple-choice, or constructive response since those are the types most often found on state assessments. This preparation work allows teachers to apply strategic interventions until mid-May to maximize the majority of students’ peformance, and then afterwards to apply Band-Aid strategies to those who have not benefited from the strategic interventions.

About Post Author