Test analysis beyond scoring

At this point in the year, many teachers wonder if there is anything they can do to improve student scores.  The answer is simple but takes some explaining – solve the right problem and quit wasting time solving the wrong ones.

So, how do we figure out what the “right problem” is?  The first step in this process is to which students are underperforming, as well as the nature of the underperformance. Ed Directions coaches utilize what we call a “step one” analysis of student performance to begin digging at the root of the problem. This establishes who missed a specific question and what the incorrect response tells us about their work. An example of the tool we use is included below.

MULTIPLE CHOICE

QUESTION #                       CONTENT                                    TASK(S)

The chart above is set up to analyze a multiple choice question.  The first column refers to the bubble answer. The second column is our key to reference what you (as the teacher) think was the probable cause of breakdown, resulting in the incorrect answer. The Third column is a count of the number of students who chose that option as their answer on the test, and the final column is used to identify the names of the students who chose each answer.

The chart below is a filled in example of how this chart was used during one of our coaching sessions.  The response key shows all the probable reasons for missing the question. To be noted, none of the response keys rule out that a student was guessing to get an answer, except for “A” which is noted as, “unrelated answer” in which a student was most likely guessing or simply filling in bubbles.

When the teacher of this particular class analyzed results they were able to interview the students with the student work in hand, and ask each student why they made the decision that they made. She was able to identify the flaw in the student processing, and addressed their specific error one support session.  This process allowed her to attack the right problem, and not use a one-size-fits-all support for students who obviously had different needs.

(An aside – in interviewing students five and seven she found out that they were doing the addition exactly the way they’d been taught – except – when they totaled the 1’s column they carried 1 one and then added the 10’s column. Unfortunately they were required to carry 2 to the  tens column. They had the ritual down (“X plus Y, carry my one) but their understanding of the ritual was not adequate)

In the sample social studies question included below, the teacher was confident the students would be able to identify the branch of government described by function in a reading.

In this case, it was a little more difficult for the teacher to put initial reasons for missing the question in the response key. In the students who answered B or D it required additional strategies to determine that some of the students misread the question while others misread the reading that was provided. The incorrect performance was due not to problem of social studies knowledge but to a problem in the students’ inability to do critical or close reading.

In schools where we have coaches, teachers give sample questions every day until the week before the test to identify real problems causing students to underperform. Most of them will be short answer, multiple-choice, or constructive response since those are the types most often found on state assessments. This preparation work allows teachers to apply strategic interventions until mid-May to maximize the majority of students’ peformance, and then afterwards to apply Band-Aid strategies to those who have not benefited from the strategic interventions.

About Post Author

Frank Desensi

Frank DeSensi is the founder and Chief Innovation Officer of Educational Directions, LLC. which consults with schools and school districts in the southeastern and mid-western United States. A retired educator, Frank spent 35 years in a variety of teaching and administrative positions. He taught at the university, college, secondary, and middle-school levels; worked in the central office as a curriculum specialist; and held both principal and assistant principal positions. From 1993 to 1998, Frank served as a Kentucky Distinguished Educator, helping to turn around schools that were labeled in decline or in crisis under the provisions of the Kentucky Education Reform Act. Frank helped develop the STAR training program for new DE’s and served as a trainer in the Kentucky Leadership Academy. He jointly holds patents for three data-management systems for schools.

See author's posts