Translating school improvement into numbers

By Joan Richardson

SCHOOL TEAM INNOVATOR - February 1997

Vickie Weseman is a self-described bean-counting, number- crunching educator.

She's one of a new breed of educators who believe that looking at the numbers will reveal secrets about a school's strengths and weaknesses. Numbers, she says, can provide insight about where to focus energies to beef up weak areas. Numbers also can highlight successes that educators can study to improve other areas.

As principal of Prairie Park Elementary School in Lawrence, Kansas, Weseman is also required to study the data as she develops her school's annual improvement plans and to prepare a school profile that quantifies the achievements in her school. "You try to show with numbers that something's happening with kids,'' says Weseman.

Central office staff have looked at test scores and other data for years. But principals and teachers often didn't bother to give the test results more than a cursory glance. For years, nobody expected them to do more than that.

"Most data was sitting in the central office and never used. It was stuck on some state report and put over in a corner somewhere and forgotten. Nobody ever looked at it,'' says Steve Nolte, who wrote the Kansas guidelines for developing data-based school profiles.

In Kansas, that changed in 1991 when the state legislature passed a school reform act that, among other measures, required schools to look at data on student performance in key areas and to use that information to guide their school improvement planning.

Now, the Education Commission of the States reports that 35 states require schools to make some kind of report about their performance to their communities. In some states, the reports are little more than just telling the public about results on statewide tests. But, increasingly, states are requiring schools to use data-based decisions to guide their school improvement efforts.

In his book, Results,: The Key to Continuous School Improvement, Mike Schmoker argues that "data should be an essential feature of how schools do business.''

"Do we want schools to continue merely adopting innovations? Or we do want schools to improve?... If we collectively focus on goals and regularly measure the impact of the methods..., then we will get better results,'' writes Schmoker, who is a research analyst for the Amphitheater School District in Tucson, Ariz.

For educators who want to do more work with data, the question, then is how to do it. Sandee Crowther, who oversees evaluation and standards in Lawrence, Kansas, points to a circular cycle for data-based decisions.

1) Determine what data you need and assemble it. Standardized tests are generally part of every school's package of data. Schools also ought to collect data on attendance, discipline, scores on the SAT and ACT, graduation rates and any other key indicators in the district. Teachers can be recruited to collect data in their classrooms about individual schools issues. For example, at Weseman's school, teachers were particularly concerned about the number of student visits to the health clinic, so they began to tally those to see what they might learn.

Crowther urges districts to use multiple sources of data. Her preference is at least three sources for each area of focus.

Using a computer is the easiest way to organize the data. The person in charge of collecting the information needs to identify which spreadsheet program will be used and that all data is reported on the appropriate spreadsheet.

Pick a spreadsheet that converts easily into a graphical program so any information you've collected can be displayed conveniently in your final product.

2) Analyze the data. Disaggregate the data according to various subgroups. The most common subgroups are race and ethnic background, sex, and socioeconomic factors.

You can probe the data further by linking various pieces of information. For example, compare standardized tests scores with attendance and discipline information.

What conclusions can you draw as you examine the data?

3) Establish priorities for focus. In some cases, as in Kansas, the state establishes priorities. For example, Kansas schools must focus on reading and math. But the state has left it up to each school to determine how they will do that. In states and districts with more flexibility for determining goals, school staff members need to discuss the results together before establishing the school's priorities.

4) Set goals. By disaggregating the data, principals and teachers can identify groups that need the most help. "You have to target the whole population but we watch a certain group to see if we can raise them even more,'' Crowther says.

Even schools that are performing at the 90th percentile can find room for improvement, Crowther says. "You will find pockets of students who are not doing as well. You concentrate on what it takes to improve the performance of those students,'' she says.

5) Identify strategies and interventions. For each goal, identify the specific strategies and interventions that teachers will use to improve performance.

For example, if fourth graders are doing poorly on space, dimensionality, and measurement portion of a statewide assessment, strategies for improvement might include encouraging more portfolio entries on the topics and encouraging more consistent use of manipulatives in everyday activities.

6) Evaluate the results. When the data once again becomes available, the principal and teacher need to compare the new set of results to the previous data. What changed since the last evaluation?

7) Revisit. Revise and start the process over. Crowther says schools should revisit the entire collection of data at least once a year to determine which areas to refine.

Weseman urges other principals to involve teachers in analyzing the data but to avoid overusing teachers in the collection phase. "I want teachers heavily involved in the decision making as a result of what's in the data. I want them interpreting the data and using that data to guide their decisions. But that doesn't necessarily mean they have to be involved in collecting the data,'' she says.

As teachers get closer to the numbers, she says they will support setting school goals that focus on student achievement. "Before we started using data, a lot of our goals were pretty climate-oriented goals. This says very clearly that we need to be focused on what the kids are doing,'' she says.

Crowther agrees. "People used to make intuitive decisions about what they thought would make a difference. But their interventions didn't make a difference in student achievement.

"Data kind of hits you right in the face with how you're doing. It gets you focused real fast,'' Crowther says.

Nolte says educators also need to consider the public relations value of using data to guide their decisions. "People have been making assumptions and taking shots at schools without any data to back up their assertions. Schools have been saying 'we're doing good.' But they haven't been able to show it. Well, here's the way to show it,'' he says.