Using state test scores to mine data for your SIP

Last week I had a chance to talk with a school principal about his school improvement plan. His leadership team had just read the schools state report and built a strategic plan for raising their score to the goal that have been set for them by the state. The principal told me the total score for their school, what the state goal score was, and then a bit about their plan for reaching that goal this year.  That was followed by the principal asking for my feedback.

When I asked him how they decided on their plan priorities and activities he mentioned that they had talked to their state support contact who gave them some ideas for improving the scores, and they put these in their school improvement plan. While this was a good idea, I suggested an added strategy.

At Educational Directions, although we believe that the state report cards are very important data points, they are not decision points that can be used to drive a school or district improvement plan. We believe that by analyzing the state report card, what we call a “data dive,” we are merely beginning the process of strategic planning.

So, what is a data dive. Let’s cover the high points.

The first step is to establish how the school grade/score is determined. So, you must begin by establishing what “counts” in a school score. This varies from state to state and can include both academic and nonacademic “cells.” The scores in these cells are then totaled to determine the school/district raw scores.

In most states, once the tests have been scored, some process (usually a norming process) is used to determine a “cut score” that represents the lowest number of points needed to be considered “successful.” Again, this varies from state to state and can be performance referenced to represent gains made from prior school or district performance levels.

The second step is to perform a red flag analysis. Let’s walk through a simplified version of this process.  You start with establishing how a school earns a score through the state test.  Compare the total number of points available to the number of points your school scored.

If this is the first time you have worked with a table like this, it’s natural to start to wonder how this comparison can help you start to plan.  The basic idea here is to start to understand where a school “lost” points, and to help determine priority areas for improvement.  To help with this, you need to add some data to your chart to help determine the “relative” status of your scores.

Once you obtain data for the top three rows, you can begin to “red flag” areas that are significantly below the state, district, or reference (comparable) school scores. Sit down with your leadership team and discuss the scores that were red flagged.  Pay close attention to the flagged scores that were lower than what your school leadership team expected, and have honest discussions about what your leaders think the scores should have been. This activity gives you a chance to identify relative strengths and weaknesses and to use the “red flags” to determine where your school “lost points” they needed to retrieve to reach their averages or their goals.

Next, you need to numerically define the task at hand. This involves playing with the mathematics of reaching your goal. We use the chart below to quantify the task.

At this point you must think about where your strategic plan is supposed to take your school, and to “play with the numbers” to see what it would take to get there. It’s important at this stage that the leadership team be candid about their ability to raise certain scores, but it’s imperative for the team to understand that you are building a pathway to “success.”

Once your data dive is complete, now you can compare the completed red flag analysis with your current School Improvement Plan. This will allow you to audit your plan priorities, plan for new areas to address your weaknesses, and pull back from initiatives that may not be contributing as greatly to retrieving your lost points.

In many cases, when Educational Directions partners with schools and districts to provide these services, it only takes reviewing one or two red flag areas before we start to see that any relationship between their SIP priorities and their quantitative goals was indirect at best.

We ended our discussion at this point so that they could complete a red flag analysis and develop some quantitative goals. Their task was to complete the audit and prioritize their concerns and conclusions.