Valid email addresses are required to post comments. If your comment is not posted, I will send you an email with an explanation.

Monday, February 14, 2011

Sound Bites and Bumper Sticker Philosophies

Andrew Cuomo, recently elected Governor of New York, outlined several concerns during his State of the State address last month. Among the issues he raised and the perceptions he articulated was his critical view of educational performance levels in public schools across the state. Cuomo shared an opinion that served as a combustible sound bite when he claimed that New York spends more than any other state on public school education (based on per pupil expenditures) yet achieves at a level placing New York at 34th of the 50 states.

I consider Cuomo's statement to be an opinion because there are as many different ways of measuring actual costs of education as there are methods of calculating performance levels. One must carefully define input and output if they expect to make a viable point. For instance, when assessing costs for education, does the examiner take into consideration the high cost of heating school buildings in this part of the country? Are the financial figures placed in the context of regional and state indices of cost of living when comparing statistics among states with widely different economies? What aspects of learning are you measuring - graduation rates? college admissions? collective grade point averages? scores on common tests (there aren't any tests for the general population of learners that are common to all 50 states - or even half of them!) participation in advanced placement classes? number of credits required to graduate? rigor of the curriculum?

I'm certainly not calling Governor Cuomo a liar. Let me make that absolutely clear. However his free and casual use of a caustic citation without a carefully constructed context calls to mind an adage I remember from a statistics class in college: "Figures don't lie, but liars can figure."

I recognize that the 24 hour 7 day a week news cycle of radio, television, print, and internet sources literally buries people beneath an avalanche of data and news stories. The intense competition for the attention of the audience overwhelmed by information produces a plethora of sound bites and bumper sticker statements (like, "1st in spending, 34th in performance!") that reduce content to a minimal amount of packaging with hope that people will remember a catchy phrase or an important number. I am sure those rankings remain in the memory of many people who followed the State of the State speech. However, although it enabled the Governor to make a case for subsequently calling for a sharp decrease in state aid to public schools, per his contention that New York tax payers were not getting appropriate value for their money, it was not entirely accurate. Or, at the least, it did not begin to offer a comprehensive perspective on a very complex issue.

It seems that for every study suggesting that public education is operating at an inferior rate, at the national level of state level, there is a correspondingly converse study that asserts public education is presently meeting the challenging demands of production and performance.

For instance, here's a summary of extensive research conducted by the Center for American Progress  a not for profit organization which examined a number of factors impacting the education process in terms of Return On Investment. In other words, they meticulously explored publicly available vital statistics on each of more than 9,000 school districts throughout America to determine the value of performance related to cost and resource allocation using data from the 2007-2008 school year  (it should be noted that since the 2007-08 school year Green Island has improved its achievement level and is no longer considered by the state to be a School In Need of Improvement). The researchers reviewed "academic achievement relative to a district's educational spending, while controlling for factors outside of their control such as cost of living and degree of student poverty." The critical question driving the research was - What are the taxpayers getting in return for their investment in their local public school district, when demographic factors (poverty rate, special education, English language learners, regional cost of living [comparable wage index]) are used as metrics to provide a proper context for the study.

The three different productivity measures are featured and defined below in italics. There are six different levels used in the evaluation matrix to arrive at Return On Investment (ROI) rankings from 1-6.

Basic Return on Investment index: This measure rates school districts on how much academic achievement a district gets for each dollar spent relative to other districts in their state. We adjusted for a variety of factors including cost-of-living differences as well as higher concentrations of low-income, non-English-speaking, and special education students to avoid penalizing districts where education costs are higher. Green Island received a ranking in the orange - we were considered in the bottom third in spending and the bottom third in achievement. This is the fourth level of six levels - with red representing the lowest (1) and green the highest (6).

Adjusted Return on Investment index: Measure that uses the same approach as the Basic Return on Investment index or Basic ROI (see entry below), but applies a statistical method called a state-level regression analysis to adjust each district’s spending for the different costs associated with serving larger concentrations of low-income, non-English-speaking, and special education students in its state. The adjustments or weights used in the Basic ROI are often not sensitive enough to account for spending differences within states. Green Island once again received the orange ranking, meaning that after adjustments were made to account for demographics, we remained with a score of 4 out of 6.

Predicted efficiency index: Index measuring whether a district's achievement is higher or lower than would be predicted after accounting for its per-pupil spending and percentage of students in special programs such as students receiving subsidized lunches. Under this approach a low-achieving district could get high marks if it performed better than predicted. Lowering academic expectations for students from disadvantaged backgrounds, however, is not a policy position the Center for American Progress supports.
The first two measures rate districts based on the achievement that school systems produce compared to their expenditures after controlling for factors outside the district’s control. In contrast, the predicted efficiency measure doesn’t compare achievement to spending. Instead, the approach rates districts on the results of their predicted achievement after controlling for factors outside their control. This distinction is important. The first two approaches attempt to measure how much “bang for the buck” a school district gets.
Technically, then, this approach does not evaluate districts against an evaluation matrix, nor does it weight or predict the amount that a school district spends on education. Instead, we used a regression analysis to predict what achievement a district should have relative to other districts in the state given its spending and percentage of students in special programs. Green Island received a darker orange ranking, showing that the district actually performed at a more efficient rate than expected. That is, when accounting for the demographics of the district, and the spending of the district, Green Island performed at a rate of efficiency higher than one would predict because the anticipated ranking was either medium cost with low achievement, or higher costs and medium achievement - neither of which are an efficient use of funds.

This was one of several methods used to assess school performance. Judging educational outcomes is a particularly difficult analysis - more than can be described in a single sentence or in one breath - like a sound bite or bumper sticker philosophy.

The bottom line --- the voters within each community should have the right to exercise their analysis on the Return On Investment for their tax dollars when they vote on their local school budget in May to determine whether the district budget is approved and what level they find appropriate -- with the right to exceed a 2% tax cap if they feel it is justified.

No comments:

Post a Comment