Dice Rolling Activity for New Jersey Teachers

Yesterday, New Jersey’s Education Commissioner announced his plans for how teachers should be evaluated, what teachers should have to do to achieve tenure, and on what basis a teacher could be relieved of tenure. In short, Commissioner Cerf borrowed from the Colorado teacher tenure and evaluation plan which includes a few key elements (Colorado version outlined at end of post):

1. Evaluations based 50% on teacher effectiveness ratings generated with student assessment data – or value-added modeling (though not stated in those specific terms)

2. Teachers must receive 3 positive evaluations in a row in order to achieve tenure.

3. Teachers can lose tenure status or be placed at risk of losing tenure status if they receive 2 negative evaluations in a row.

This post is intended to illustrate just how ill-conceived – how poorly thought out – the above parameters are. This all seems logical on its face, to anyone who knows little or nothing about the fallibility of measuring teacher effectiveness or probability and statistics more generally. Of course we only want to tenure “good” teachers and we want a simple mechanism to get rid of bad ones. If it was only that easy to set up simple parameters of goodness and badness and put such a system into place. Well, it’s not.

Here’s an activity for teachers to try today. It may take more than a day to get it done.

MATERIALS: DICE (well, really just one Die)! That’s all you need!

STEP 1: Roll Die. Record result. Roll again. Record result. Keep rolling until you get the same number 3 times in a row. STOP. Write down the total number of rolls.

STEP 2: Roll Die. Record result. Roll again. Record result. Keep rolling until you get the same number 2 times in a row. STOP. Write down the total number of rolls.

Post your results in the comments section below.

Now, what the heck does this all mean? Well, as I’ve written on multiple occasions, the year to year instability of teacher ratings based on student assessment scores is huge. Alternatively stated, the relationship between a teacher’s rating in one year and the next is pretty weak. The likelihood of getting the same rating two straight years is pretty low, and three straight years is very low. The year to year correlation, whether we are talking about the recent Gates/Kane studies or previous work, is about .2 to .3. There’s about a 35% chance that an average teacher in any year is misidentified as poor, given one year of data, and 25% chance given two years of data. That’s very high error rate and very low year to year relationship. This is noise. Error. Teachers – this is not something over which you have control! Teachers have little control over whether they can get 3 good years in a row. AND IN THIS CASE, I’M TALKING ONLY ABOUT THE NOISE IN THE DATA, NOT THE BIAS RESULTING FROM WHICH STUDENTS YOU HAVE!

What does this mean for teachers being tenured and de-tenured under the above parameters? Given the random error, instability alone, it could take quite a long time, a damn long time for any teacher to actually string together 3 good years of value added ratings. And even if one does, we can’t be that confident that he/she is really a good teacher. The dice rolling activity above may actually provide a reasonable estimate of how long it would take a teacher to get tenure (depending on how high or low teacher ratings have to be to achieve or lose tenure). In that case, you’ve got a 1/6 chance with each roll that you get the same number you got on the previous. Of course, getting the same number as your first roll two more times is a much lower probability than getting that number only one more time. You can play it more conservatively by just seeing how long it takes to get 3 rolls in a row where you get a 4, 5 or 6 (above average rating), and then how long it takes to get only two in a row of a 1, 2, or 3.

What does that mean? That means that it could take a damn long time to string together the ratings to get tenure, and not very long to be on the chopping block for losing it. Try the activity. Report your results below.

Each roll above is one year of experience. How many rolls did it take you to get tenure? And how long to lose it?

Now, I’ve actually given you a break here, because I’ve assumed that when you got the first of three in a row, that the number you got was equivalent to a “good” teacher rating. It might have been a bad, or just average rating. So, when you got three in a row, those three in a row might get you fired instead of tenured. So, let’s assume a 5 or a 6 represent a good rating. Try the exercise again and see how long it takes to get three 5s or three 6s in a row. (or increase your odds of either success or failure by lumping together any 5 or 6 as successful and any 1 or 2 as unsuccessful, or counting any roll of 1-3 as unsuccessful and any roll of 4 -6 as successful)

Of course, this change has to work both ways too. See how long it takes to get two 1s or two 2s in a row, assuming those represent bad ratings.

Now, defenders of this approach will likely argue that they are putting only 50% of the weight of evaluations on these measures. The rest will include a mix of other objective and subjective measures. The reality of an evaluation that includes a single large, or even significant weight, placed on a single quantified factor is that that specific factor necessarily becomes the tipping point, or trigger mechanism. It may be 50% of the evaluation weight, but it becomes 100% of the decision, because it’s a fixed, clearly defined (though poorly estimated) metric.

In short, based on the instability of measures alone, the average time to tenure will be quite long, and highly unpredictable. And, those who actually get tenure may not be much more effective, or any more, than those who don’t. It’s a crap shoot. Literally!

Then, losing tenure will be pretty easy… also on a crap shoot… but your odds of losing are much greater than your odds were of winning.

And who’s going to be lining up for these jobs?

Summary of research on “intertemporal instability” and “error rates”

The assumption in value-added modeling for estimating teacher “effectiveness” is that if one uses data on enough students passing through a given teacher each year, one can generate a stable estimate of the contribution of that teacher to those children’s achievement gains.[1] However, this assumption is problematic because of the concept of inter-temporal instability: that is, the same teacher is highly likely to get a very different value-added rating from one year to the next. Tim Sass notes that the year-to-year correlation for a teacher’s value-added rating is only about 0.2 or 0.3 – at best a very modest correlation. Sass also notes that:

About one quarter to one third of the teachers in the bottom and top quintiles stay in the same quintile from one year to the next while roughly 10 to 15 percent of teachers move all the way from the bottom quintile to the top and an equal proportion fall from the top quintile to the lowest quintile in the next year.[2]

Further, most of the change or difference in the teacher’s value-added rating from one year to the next is unexplainable – not by differences in observed student characteristics, peer characteristics or school characteristics.[3]

Similarly, preliminary analyses from the Measures of Effective Teaching Project, funded by the Bill and Melinda Gates Foundation found:

When the between-section or between-year correlation in teacher value-added is below .5, the implication is that more than half of the observed variation is due to transitory effects rather than stable differences between teachers. That is the case for all of the measures of value-added we calculated.[4]

While some statistical corrections and multi-year analysis might help, it is hard to guarantee or even be reasonably sure that a teacher would not be dismissed simply as a function of unexplainable low performance for two or three years in a row.

Classification & Model Prediction Error

Another technical problem of VAM teacher evaluation systems is classification and/or model prediction error. Researchers at Mathematica Policy Research Institute in a study funded by the U.S. Department of Education carried out a series of statistical tests and reviews of existing studies to determine the identification “error” rates for ineffective teachers when using typical value-added modeling methods.[5] The report found:

Type I and II error rates for comparing a teacher’s performance to the average are likely to be about 25 percent with three years of data and 35 percent with one year of data. Corresponding error rates for overall false positive and negative errors are 10 and 20 percent, respectively.[6]

Type I error refers to the probability that based on a certain number of years of data, the model will find that a truly average teacher performed significantly worse than average.[7] So, that means that there is about a 25% chance, if using three years of data or 35% chance if using one year of data that a teacher who is “average” would be identified as “significantly worse than average” and potentially be fired. Of particular concern is the likelihood that a “good teacher” is falsely identified as a “bad” teacher, in this case a “false positive” identification. According to the study, this occurs one in ten times (given three years of data) and two in ten (given only one year of data).

Same Teachers, Different Tests, Different Results

Determining whether a teacher is effective may vary depending on the assessment used for a specific subject area and not whether that teacher is a generally effective teacher in that subject area. For example, Houston uses two standardized test each year to measure student achievement: the state Texas Assessment of Knowledge and Skills (TAKS) and the nationally-normed Stanford Achievement Test.[8] Corcoran and colleagues used Houston Independent School District (HISD) data from each test to calculate separate value-added measures for fourth and fifth grade teachers.[9] The authors found that a teacher’s value-added can vary considerably depending on which test is used.[10] Specifically:

among those who ranked in the top category (5) on the TAKS reading test, more than 17 percent ranked among the lowest two categories on the Stanford test. Similarly, more than 15 percent of the lowest value-added teachers on the TAKS were in the highest two categories on the Stanford.[11]

Similar issues apply to tests on different scales – different possible ranges of scores, or different statistical modification or treatment of raw scores, for example, whether student test scores are first converted into standardized scores relative to an average score, or expressed on some other scale such as percentile rank (which is done is some cases but would generally be considered inappropriate). For instance, if a teacher is typically assigned higher performing students and the scaling of a test is such that it becomes very difficult for students with high starting scores to improve over time, that teacher will be at a disadvantage. But, another test of the same content or simply with different scaling of scores (so that smaller gains are adjusted to reflect the relative difficulty of achieving those gains) may produce an entirely different rating for that teacher.

Brief Description of Colorado Model

Colorado, Louisiana, and Tennessee have teacher evaluation systems proposed that will require 50% or more of the evaluations to be based on their students’ academic growth. This section summarizes the evaluation systems in these states as well as the procedural protections that are provided for teachers.

Colorado’s statute creates a state council for educator effectiveness that advises the state board of education.[12] A major goal of these councils is to aid in the creation of teacher evaluation systems that “every teacher is evaluated using multiple fair, transparent, timely, rigorous, and valid methods.”[13] Considerations of student academic growth must comprise at least 50% of each evaluation.[14] Quality measures for teachers must include “measures of student longitudinal academic growth” such as “interim assessments results or evidence of student work, provided that all are rigorous and comparable across classrooms and aligned with state model content standards and performance standards.”[15] These quality standards must take diverse factors into account including “special education, student mobility, and classrooms with a student population in which ninety-five percent meet the definition of high-risk student.”[16]

Colorado’s statute also calls for school districts to develop appeals procedures. A teacher or principal who is deemed ineffective must receive written notice, documentation used for making this determination, and identification of deficiency.[17] Further, the school district must ensure that a tenured teacher who disagrees with this designation has “an opportunity to appeal that rating, in accordance with a fair and transparent process, where applicable, through collective bargaining.”[18] If no collective bargaining agreement is in place, then the teacher may request a review “by a mutually agreed-upon third party.”[19] The school district or board for cooperative services must develop a remediation plan to correct these deficiencies, which will include professional development opportunities that are intended to help the teacher achieve an effective rating in her next evaluation.[20] The teacher or principal must receive a reasonable amount of time to correct such deficiencies.[21]

[1] Tim R. Sass, The Stability of Value-Added Measures of Teacher Quality and Implications for Teacher Compensation Policy, Urban Institute (2008), available at http://www.urban.org/UploadedPDF/1001266_stabilityofvalue.pdf. See also Daniel F. McCaffrey et al., The Intertemporal Variability of Teacher Effect Estimates, 4 Educ. Fin. & Pol’y, 572 (2009).

[2] Sass, supra note 27.

[3] Id.

[4] Bill & Melinda Gates Foundation, supra note 26.

[5] Peter Z. Schochet & Hanley S. Chiang, Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains (NCEE 2010-4004). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education (2010).

[6] Id.

[7] Id. at 12.

[8] Sean P. Corcoran, Jennifer L. Jennings & Andrew A. Beveridge, Teacher Effectiveness on High- and Low-Stakes Tests, Paper presented at the Institute for Research on Poverty summer workshop, Madison, WI (2010).

[9] Id.

[10] Id.

[11] Id.

[12] Co. Rev. Stat. § 22-9-105.5(2)(a) (2010).

[13] Id. § 22-9-105.5(3)(a).

[14] Id.

[15] Id.

[16] Id. The statute also calls for the creation of performance evaluation councils that advise school districts. Id. § 22-9-107(1). The performance evaluation councils also help school districts develop teacher evaluation systems that must be based on the same measures as that developed by the state council for educator effectiveness. Id. § 22-9-106(1)(e)(II). However, the performance evaluation councils lose their authority to set standards once the state board has promulgated rules and the initial phase of statewide implementation has been completed. Id. § 22-9-106(1)(e)(I).

[17] Id. § 22-9-106(3.5)(b)(II).

[18] Id.

[19] Id.

[20] Id.

[21] Id.

School Funding Myths & Stepping Outside the “New Normal”

I’ve been writing quite a bit lately about rather complex state school finance formula issues. That is much the point of this blog. But it may be hard for some to see how my recent posts on school finance relate back to the broader reform agenda, and to understand the implications of these posts for state policies. Let me try to summarize these posts – posts on spending bubbles, the “New Normal” and school finance PORK. My overarching goal in these posts is to explain that much of the reformy rhetoric about budget cuts and the “New Normal” is based on myth about how school funding works and what we should be doing in these catastrophic economic times.

Here are the myths and some of the realities.

Reformy myth #1: That every state has done its part and more, to pour money into high need, especially poor urban districts. It hasn’t worked, mainly because teachers are lazy and overpaid and not judged on effectiveness, measured by value-added scores. So, now is the time to slash the budgets of those high need districts, where all of the state aid is flowing, and fire the worst teachers. And, it will only help, not hurt.

Reality: Only a handful of states have actually targeted substantial additional resources to high need districts. See www.schoolfundingfairness.org. And the effort of states to finance their elementary and secondary education systems varies widely. Some states have in fact systematically reduced their effort to finance local public schools for decades. That is, the tax burden to finance public schools in some states is much lower now than it was decades ago. Very few states apply much higher effort than in the past. See: https://schoolfinance101.wordpress.com/2010/12/23/is-it-the-new-normal-or-the-new-stupid/

Reformy myth #2: The only aid to be cut, the aid that should be cut, and the aid that must be cut in the name of the public good, is aid to high need, large urban districts in particular. The argument appears to be that handing down state aid cuts as a flat percent of state aid is the definition of “shared sacrifice.” And the garbage analysis of district Return on Investment by the Center for American Progress, of course, validates that high need urban districts tend to be least efficient anyway. Therefore, levying the largest cuts on those districts is entirely appropriate.

Reality: As I have discussed in my series of recent posts, if there are going to be cuts – if states really believe that cuts to state aid are absolutely necessary, many state aid formulas include aid that is more appropriate to cut. That is, aid to districts who really don’t need that aid. Aid to districts that can already spend well above all others with less local effort. Aid to districts that will readily replace their losses in state aid with additional local revenues (or even private contributions). That’s the pork, and that’s where cuts, if necessary, should occur.

Reformy myth #3: The general public is fed up and don’t want to waste any more of their hard earned tax dollars on public schools. They are fed up with greedy teachers with gold plated benefits and fed up with high paid administrators. They don’t care about small class sizes and…well… are just fed up with all of this taxing and spending on public schools that stink. As a result, the only answer is to cut that spending and simultaneously make schools better.

Reality: The reality is that local voters in a multitude of surveys rate their own local public schools quite highly and that local voters when given the opportunity, even during the recent economic downturn, show very high rates of support for school budgets – including budgets with significant increased property taxes (the most hated tax). As I noted in a previous post, when New Jersey handed down state aid cuts to 2010-2011 school budgets and when- for the first time in a long time- the majority of local district budgets statewide failed to achieve approval from local voters, it was still the case that the vast majority (72%) of local budgets passed in affluent communities – in most cases raising sufficient local property tax resources to cover the state aid cuts. In another case, local residents in an affluent suburban community raised privately $420,000 to save full day kindergarten programs. Meghan Murphy’s analysis of Hudson Valley school districts shows that New York State districts also have attempted to counterbalance state aid cuts with property tax increases, but that the districts have widely varied capacity to pull this off. Parents in a Kansas district are suing in federal court requesting injunctive relief to allow them to raise their taxes for their schools (they use faulty logic and legal arguments, but their desire for better schools should be acknowledged!).

Reformy myth #4: None of this school funding stuff matters anyway. It doesn’t matter what the overall level of funding is and it doesn’t matter how that funding is distributed. As evidence of this truthiness, reformers point to 30+ years of huge spending growth coupled with massive class size reduction and they argue… flat NAEP scores, low international performance and flat SAT scores. Therefore, if we simply cut funding back to 1980 levels (adjusted only for the CPI) and fire bad teachers, we can achieve the same level of outcomes for one heck of a lot less money.

Reality: First of all, these comparisons of spending now to spending then are bogus. I address the various factors that influence the changing costs of achieving desired educational outcomes in this post: https://schoolfinance101.wordpress.com/2011/01/12/understanding-education-costs-versus-inflation/. Second, rigorous peer reviewed studies do show that state school finance reforms matter. Shifting the level of funding can improve the quality of teacher workforce and ultimately the level of student outcomes and shifting the distribution of resources can shift the distribution of outcomes. http://www.tcrecord.org/content.asp?contentid=16106 Similarly, constraining education spending growth over the long term can significantly harm the quality of public schools. See: https://schoolfinance101.wordpress.com/2010/04/22/a-few-quick-notes-on-tax-and-expenditure-limits-tels/

An opportunity for states?

I would argue that now… right now… represents a real opportunity for those states who actually want to step up, and really invest in the quality of their education systems and use the quality of their education systems to drive their economic futures.

I stumbled across this article http://www.foxnews.com/us/2011/02/13/states-offer-tax-breaks-guarantee-jobs/ on the Fox News website the other day, and it presents some useful insights for state policy makers regarding tax policy decisions and economic growth. I’ve written about the same point very early in my blogging (that the Small Business Survival Index in particular, misses some big points about location selection). Here’s a short section of the Fox News piece:

But there’s a catch to the anti-tax, pro-business rhetoric: Businesses consider a range of factors when deciding where to locate, including the quality of schools, roads and programs that rely on a certain level of public spending and regulation. And evidence suggests there is little correlation between a state’s tax rate and its overall economic health.

“Concerns about taxes are overstated,” said Matt Murray, a professor of economics at the University of Tennessee who studies state finance. “Labor costs, K-12 education and infrastructure availability are all part of a good business climate. And you can’t have those without some degree of taxation.”

States’ tax rates also do not predict their resilience during an economic downturn.

Arguably, no time is better than now. Other states are jumping on board with the “New Normal” reformy logic that slashing education budgets, increasing class sizes and narrowing curriculum around tested content areas is the only way to go. Yet, educated parents invariably want small class sizes (often topping the list of preferences for private or public schools) and want their children to have intellectually stimulating and broad, enriched curriculum. The current environment presents a great opportunity for some states to step outside the “New Normal” and truly race to the top with real investment in their public schooling systems.

Reformy Disconnect: “Quality Based” RIF?

I addressed this point previously in my post on cost-effectiveness of quality based layoffs, but it was buried deep in the post.

Reformers are increasingly calling for quality based layoffs versus seniority based layoffs, as if a simple dichotomy. Sounds like a no brainer when framed in these distorted terms.

I pointed out in the previous post that if the proposal on the table is really about using value-added teacher effect estimates versus years of service, we’re really talking about the choice between significantly biased and error prone – largely random – layoffs versus using years of service. It doesn’t sound as much like a no brainer when put in those terms, does it? While reformers might argue that seniority based layoffs are still more “error prone” than effectiveness rating layoffs, it is actually quite difficult to determine which, in this case, is more error prone. Existing simulation studies identifying value-added estimates as the less bad option, use value-added estimates to determine which option is better. Circular logic (as I previously wrote)?

We’re having this policy conversation about layoffs now because states are choosing (yes choosing, not forced, not by necessity) to slash aid to high need school districts that are highly dependent on state aid, and will likely be implementing reduction in force (RIF) policies. That is, laying off teachers. So, reformy pundits argue that they should be laying off those dead wood teachers – those with bad effectiveness ratings, instead of those young, energetic highly qualified ones.

So, here are the basic parameters for quality-based RIF:

1. We must mandate test-score based teacher effectiveness ratings as a basis for teacher layoffs.

2. But, we acknowledge that those effectiveness ratings can at best be applied to less than 20% of teachers in our districts, specifically teachers of record – classroom teachers – responsible for teaching math and reading in grades 3 to 8 (4 to 8 if only annual assessment data)

3. Districts are going to be faced with significant budget cuts which may require laying off around 5% or somewhat more of their total staff, including teaching staff.

4. But, districts should make efforts to layoff staff (teachers) not responsible for the teaching of core subject areas.

Is anyone else seeing the disconnect here? Yeah, there are many levels of it, some more obvious than others. Let’s take this from the district administrator’s/local board of education perspective:

“Okay, so I’m supposed to use effectiveness measures to decide which teachers to lay off. But, I only have effectiveness measures for those teachers who are supposed to be last on my list for lay offs? Those in core areas. The tested areas. How is that supposed to work?”

Indeed the point of the various “quality based layoff” simulations that have been presented (the logic of which is problematic) is to layoff teachers in core content areas and rely on improved average quality of core content teachers over time to drive system wide improvements. These simulations rely on heroic assumptions of a long waiting list of higher quality teacher applicants just frothing at the mouth to take those jobs from which they too might be fired within a few years due to random statistical error (or biased estimates) alone.

That aside, reduction in force isn’t about choosing which teachers to be dismissed so that you can replace them with better ones. It’s about budgetary crisis mode and reduction of total staffing costs. And reduction in force is not implemented in a synthetic scenario where the choice only exists to lay off either core classroom teachers based on seniority, or core classroom teachers based on effectiveness ratings (the constructed reality of the layoff simulations). Reduction in force is implemented with consideration for the full array of teaching positions that exist in any school or district. “Last in, first out” or LIFO as reformy types call it, does not mean ranking all teachers systemwide by experience and RIF-ing the newest teachers regardless of what they teach, or the program they are in. Specific programs and positions can be cut, and typically are.

And it is unlikely that local district administrators in high need districts would, or even should, look first to cut deeply into core content area teachers. So, a 5% staffing cut might be accomplished before ever cutting a single teacher for whom an effectiveness rating occurs – or very few. So, in the context of RIF, layoffs actually based on effectiveness ratings are a drop in the bucket.

So now I’m confused. Why is this such a pressing policy issue here and now? Does chipping away at seniority based provisions really have much to do with improving the implementation of RIF policies? Perhaps some are using the current economic environment and reformy momentum to achieve other long-run objectives?

Pork Hunting 101: Shredding the Pork

I love a good pulled pork sandwich. In fact, it’s one of my favorite foods. And the best damn pork sandwich I can think of is the Hog Heaven at Oklahoma Joe’s in the side of a freakin’ gas station in Kansas City, Kansas. I expect that this statement will perhaps be the most controversial statement I’ve made on this blog thus far. Really… it’s awesome… and one of the few things I really miss about being in Kansas City.

That aside, I’ve been blogging lately about a different kind of PORK – school finance pork. I started this pork campaign with a posting on state aid in New York after the announcement of Governor Cuomo’s proposed budget and education cuts. Despite large amounts of aid still being allocated to some of the wealthiest school districts in the nation, the good democratic Governor of New York decided it was somehow most appropriate to largely protect that aid, and instead slam the highest need, large urban, mid-sized city and poor rural districts in the state.

This post provides a primer on finding pork in state school finance formulas. It’s a two part process that begins with screening for pork, and then involves more intensive investigative research.

What is School Finance Pork? School finance pork is state aid that is currently being allocated to districts that otherwise don’t really need that aid. In this case, need is defined in terms of the needs of the students to be served AND in terms of the ability of the local public school districts and its residents and property owners to pay the cost of those services. In overly simple terms, some local public school districts can easily pay for the full cost of their needed educational programs and services on their own and with much less effort (tax effort) than others. Allocating state aid to these districts while depriving others with greater student needs and the inability to meet those needs is inexcusable. Cutting aid to needier communities who are unable to replace those lost revenues, while retaining aid to the wealthy is inexcusable. That’s PORK. And like other political pork-barrel spending, it exists because state legislators negotiate for state aid formulas that bring something home to their own districts.

How do legislators generate PORK? Pork can be generated by at least two different approaches. In the first approach, legislators actually try to manipulate the general state aid formula to find ways to argue that it’s actually more expensive to educate kids in wealthier communities, or alternatively that its cheaper to educate kids in poor communities. If legislators can raise the foundation funding targets to wealthy communities they can increase the likelihood that they can drive state support to those districts. Seems a bit of a stretch, but it’s been done by many. A second “within the formula” approach to generating pork is to adopt “minimum state aid” and/or hold harmless aid provisions that guarantee that no matter how wealthy a district is, that the state will still pick up X% of its formula funding. That is, the state will cover X% even if the district can raise double what it needs with very little tax effort. Another type of pork is Outside the Formula pork. It can be tricky (though not impossible) for legislators to sufficiently manipulate a state aid formula to provide more aid to wealthier districts. It’s relatively easy for state legislators to adopt outside the formula aid programs that allocate aid in very different ways – like flat allocations per pupil across all districts, regardless of local wealth, or even like New York’s property tax relief targeted to wealthier communities.

How do we screen for Pork? Here, I provide a few examples of screening for pork, using a national data set – the U.S. Census Bureau’s Fiscal Survey of Local Governments. As we stated in our Is School Funding Fair? A bare minimum goal of a state school finance formula would be to achieve a FLAT relationship between poverty and state and local revenue per pupil. Ideally, state school finance formulas would result in additional resources targeted to higher poverty districts – that is – systematically higher state and local revenue per pupil in higher poverty districts. One can take and state’s school finance data – here from 2007-08 – and make a graph of the a) local revenue per pupil, b) state aid per pupil and c) total revenue per pupil across districts by poverty. Here’s Texas:

I’ve dropped very small districts because they tend to scatter the pattern for a variety of legitimate cost related reasons. Blue circles represent the state and local revenue per pupil, which on average in Texas has a slight downward slope. Red triangles are the local revenue. Clearly, the lower poverty districts are raising quite a bit in local revenue but the higher poverty districts aren’t raising much. On average the state aid in green squares is being allocated in inverse proportion to the local revenue… but not that aggresively. For example, if we look way to the upper left…. we’ve got some red triangles – local revenue per pupil – above $10,000 per pupil. That is, some of these very low poverty districts are able to raise on their own, over $10k per pupil. But look at the trajectory of the state aid allotments – those districts are still getting a significant amount of state aid – enough to still raise their totals even higher as pointed out by yellow arrows. Would it not make more sense for this aid to be allocated to the districts at the far right hand side of the picture? Amazingly, the state aid distribution slope in Texas is quite gradual – to be kind. It would appear to include significant “flat” distributions of either or both minimum aid and/or outside the formula aid which goes to lower poverty and/or higher wealth districts the expense of higher need, lower wealth districts. THAT’s PORK!

Here’s New York, when using the same data set:

And here’s Pennsylvania:

In Pennsylvania, minimum aid provisions and flat distribution of Special Education aid are partly to blame.

And here’s Illinois:

Illinois also has minimum aid provisions in the form of alternative formulas for districts otherwise too wealthy to receive aid through the primary foundation formula. And Illinois allocates numerous categorical aids outside the formula in ways that reinforce the aggregate disparities.

So, here are four states and in each one, total state and local revenue per pupil is slightly to significantly lower per pupil in higher poverty than in lower poverty districts. And in each state, the state aid is distributed in ways that guarantee the provision of at least some – sometimes significant – aid per pupil to districts that on their own are raising and spending significantly more money per pupil than their much higher need counterparts.

How do we identify the source of the Pork? This part is a bit harder, and requires much more detailed investigation of the state school finance formulas and of the “runs” of state aid programs. I hope to get a chance to provide more information on pork finding at a later point. But for those interested in exploring these issues on their own, there are two sources of information that are critical to the search – a) the documentation that explains the calculations behind the allocation of state aid and b) much more importantly, what are called the “runs” of state aid allocations to local public school districts that are used by legislators to negotiate changes to the aid formula and/or the distribution of cuts. It’s that time of year – legislative session season. And legislators are interested to see which pieces of the aid formula will bring home pork, or how cuts will affect their districts. If you’re as warped as I am about this stuff… and actually really like digging through these numbers… contact your state legislators and find out who to get in touch with in order to get an electronic spreadsheet run of the various district by district state aid allocations. And be sure to get other basic district data, including wealth measures (property wealth and income) and enrollment characteristics (Total enrollments and student needs). Play around with graphs of aid (divided by pupils – per pupil aid) with respect to wealth and need measures – and look for those aid programs in particular that appear to allocate systematically more aid to districts that are otherwise less needy. Feel free to e-mail me spreadsheets of those runs. If I get a chance, I’ll see what pork I can find!

Cheers.

And here’s to Pork!

Disparate thinking: The administration’s blind eye & racially disparate impact

Apparently the U.S. Department of Education has decided to take on public education policies that not only are intentionally racially discriminatory, but also state and local policies that happen to have a racially disparate impact on certain populations. Now, these are departmental regulations, not statutes and not a constitutional protection, so this doesn’t mean that advocates can start filing lawsuits on behalf of groups disparately affected by education policies (except where other state laws prohibiting disparate impact exist, like Illinois). But, it does mean that the U.S. Department of Education can use its biggest available threat – denial of funding – to pressure state and local education agencies to change policies that result in racially disparate impact.

Statistically, what disparate impact means is that the policy in question results in a disproportionate effect on one group of individuals versus another, where “groups” are defined by race, ethnicity or national origin. There are many occurrences in public education where racially disparate impact rears its ugly head. For example, racially disparate classification rates of children with disabilities, or racially disparate rates of disciplinary action. Identifying appropriate policy changes to reduce racially disparate impact in any of these areas, while not compromising other interests is indeed important – such as making sure that kids with legitimate special education needs still get identified and served, or making sure appropriate discipline is handed out where necessary.

The administration’s renewed interest in racially disparate impact was announced almost a year ago, and has apparently crept back into the conversation in the past few weeks.

Here’s the Education Week synopsis:

At the Feb. 11 briefing, Ricardo Soto, the deputy assistant secretary for the Education Department’s office for civil rights, elaborated on the office’s new policy, saying that the Obama administration is “using all the tools at our disposal,” including “disparate-impact theory,” to ensure that schools are fairly meting out discipline to students. Some research shows, for instance, that suspension rates for African-American males in middle school can be nearly three times as high as those for their white, male peers.

So, the interest here is specifically the racially disparate distribution of disciplinary actions. That’s all well and good. We should explore these issues and resolve them appropriately if we can. Once again, the usual target is local public school districts because when it comes to any of today’s education policy issues – in the eyes of the current administration and their closest advisers – only local public districts and local administrators can be to blame (even when it comes to funding disparities?).

But, as my readers know, this a school finance blog, and that means that eventually this topic is coming back around to school finance (okay, not always). Could it possibly be that some states actually continue to operate state school finance formulas that produce “racially disparate” effects? That is, that districts serving larger shares of minority children have systematically (statistically) less state and local revenue (the revenue under control of state policy) than districts with fewer minority children? And if so, which states might those be?

Clearly, substantial disparities in the quality of education received (funding, class sizes, teacher credentials, etc.) by minority children as a function of state policies to provide, or not, sufficient financial support to their schools is at least equally relevant to rates of discipline referrals of minority versus non minority children attending the same school or district.

In our report Is School Funding Fair?, We evaluated states in terms of whether they provided for systematically more or less state and local revenue per pupil in higher versus lower poverty districts. Now, which states did the worst on this measure? Among the big ones, the most poverty disparate states in funding were Illinois, Pennsylvania and New York! New York makes this list largely because of its Pork barrel finance policies. And believe, me, Pennsylvania and Illinois have similar pork to shred. But poverty related disparities while important, don’t fall under this racially disparate impact umbrella. The real question here is which states have the largest racial disparities in school funding?

A few years back, Robert Bifulco, now at the Maxwell School at Syracuse, did a nice quick number crunch on black-white funding disparities nationally, in the Journal of Education Finance. On average, Bob found that without any corrections for costs related to student needs, or other costs, districts with higher concentrations of black enrollments had marginally higher per pupil spending. But after correcting for costs and needs, districts with higher black enrollments had lower per pupil spending. But, these findings, while interesting and useful, look at the nation as a whole, and, if I recall, do some breakouts by region.

Just like the poverty related variation we show in the fairness report, racial disparities in funding also vary widely across states. And those disparities occur for a variety of reasons. Yes, to some extent those disparities exist because of differences in local property wealth in blacker versus whiter communities and the state’s failure to allocate sufficient aid to offset those disparities. And in many places, those disparities in property values are largely a function of carefully planned racial segregation of housing stock and distribution of other property types (Brilliant article on real estate development in the Kansas City metro: http://www.tulane.edu/~kgotham/RestrCovenants.pdf)

BUT CAN A STATE LIKE NEW YORK REALLY CLAIM THAT THERE’S JUST NOT ENOUGH AID AVAILABLE TO FIX THE RACIAL DISPARITIES WHEN IT IS DUMPING TAX RELIEF AID AND MINIMUM FOUNDATION AID INTO THE WEALTHIEST DISTRICTS IN THE COUNTRY? Redistributing the pork might not erase the disparities entirely, but it’s a start.

Other disparities actually exist by the design of the aid formulas, and various Tricks of the Trade that create racial disparities in funding – deceptively and arguably quite intentionally. Here’s an outstanding sarcastic, critical analysis of how Kansas legislators gamed their finance system to embed racially disparate effects over time: http://www.pitch.com/2005-04-14/news/funny-math/ (after reading this, consider the role of real estate development addressed in the Kansas City article above!)

Quick side-bar – This brilliant piece of school finance journalism is written by Tony Ortega, when he was in Kansas City (written from the perspective of a KC Strip Steak… it’s a Kansas City thing). Now, Tony is editor and chief of the Village Voice in NY. He Tony, how ‘bout that New York finance stuff? Subsidies for Scarsdale, while cutting Utica and Middletown, or Mt. Vernon? (even if from the perspective of a NY Strip Steak?)

There are lots of Tricks of the Trade used to reduce aid to high minority districts. A favorite choice of state legislators is to allocate aid based on average daily attendance rather than enrollment. Higher poverty, higher minority concentration districts tend to have lower attendance rates… thus reducing their aid, compared to what they would receive if the aid was allocated on the number of enrolled pupils.

There was a brief period in the late 1990s when three separate federal court challenges were filed against racially disparate state school finance systems – in Pennsylvania (Powell v. Ridge), in New York (AALDF v. State) and in Kansas (Robinson v. Kansas). These cases were cut off by a series of related decisions in the early 2000s (which basically said that an individual does not have a right to sue over racially disparate effects, because the language of “racially disparate” effects appears only in regulations and not in the Civil Rights statutes themselves).

Interestingly, around that time, the Illinois legislature countered with state legislation that prohibits policies having racially disparate effect (Illinois Civil Rights Act of 2003). And as it turns out, Illinois school districts are applying this law to challenge the Illinois school funding formula – a formula that is among the most racially disparate in the nation.

http://www.jenner.com/news/news_item.asp?id=15009924

So, what’s my point here? I’m doing a bit of rambling. Well, here’s my best shot at a synopsis.

1. Yes, it is important that the current administration explore the reasons for, and possible resolutions of racially disparate effects in such areas as discipline referrals or special education placement rates (although they appear focused on the former, not latter).

2. Yes, there exists the likelihood that local public school districts are engaged in practices that harm minority populations, ranging from the previously mentioned issues, to others such as highly tracked curricular offerings which result in significant within school and within district segregation, as well as attempts by some local boards of education to undo long running diversity plans.

3. But, I would argue, that while these issues are important, there are other at least equally important issues to be addressed – substantial racial disparities in funding and resulting programs and services – in some states far more than others. AND ILLINOIS IS ONE OF THOSE STATES!

4. And those racial disparities in funding are not entirely a result of not enough money to offset differences in local wealth, or neglect of the state aid formula (underfunding the formula) – WHICH IS BAD ENOUGH – but are largely a function of maintaining PORK in affluent communities at the expense of poor minority districts, and of Tricks of the Trade – or specific provisions in state school funding formulas that drive money to whiter districts and create or reinforce racial disparities.

READINGS

Green, P.C., Oluwole, J., Baker, B.D. (2010) Getting their hands dirty: How Alabama’s public officials may have maintained separate and unequal education. West’s Education Law Reporter 253 (2) 503‐
520

Green, P.C., Baker, B.D., Oluwole, J. (2008) Obtaining racial equal educational opportunity through school finance litigation. Stanford Journal of Civil Rights and Civil Liberties IV (2) 283‐338

Baker, B.D., Green, P.C. (2005) Tricks of the Trade: Legislative Actions in School Finance that Disadvantage Minorities in the Post‐Brown Era American Journal of Education 111 (May) 372‐413

Baker, B.D., Green, P.C. (2003) Commentary: The Application of Section 1983 to School Finance Litigation. West’s Education Law Reporter. 173 (3) 679‐696

Green, P.C., Baker, B.D. (2002) Circumventing Rodriguez: Can plaintiffs use the Equal Protection Clause to challenge school finance disparities caused by inequitable state distribution policies? Texas
Forum on Civil Liberties and Civil Rights 7 (2) 141 – 165

Where’s the Pork? Mitigating the Damage of State Aid Cuts

This is a very long and complicated post, so I’ll give you a few take home points up front…

Equity Center Radio on School Finance Pork: http://216.246.105.5/Audio_Recordings/2011-0053_ECRadio_Bruce_Baker_02-18-2011.mp3

Take home points

Crude assumptions promoted by the “new normal” pundit crowd that across the board state aid cuts are a form of shared sacrifice are misguided and dreadfully oversimplified.
State aid cuts hurt some districts and the children they serve more than others, even when those aid cuts are “flat” across districts. Districts with greater capacity will readily recover their losses (and then some) with other revenue sources. Those who can’t are out of luck.
State aid cuts as a proportion of state aid are particularly bad because they take the most from those who need the most.
Some might find it surprising that many state school finance formulas contain special provisions that allocate relatively large sums of aid to very affluent school districts – districts that could easily pay for the difference on their own and serve relatively low need student populations. One can readily identify hundreds of millions of dollars in New York State being allocated as aid to some of the nation’s wealthiest school districts.
State legislators and Governors often protect this aid – which I refer to as school finance PORK – even while slashing away disproportionately at aid for the neediest districts.
That’s just wrong!

Now for the lengthy post…

It’s budget proposal and state-of-the-state time of year right now. And Governors from both parties are laying out their state budget cuts, many refusing to consider any type of “revenue enhancements” (uh… tax increases). These include New York’s Governor Cuomo suggesting a 7% cut to state aid.

There exists a baffling degree of ignorance being spouted by pundits about school budget cuts. Stuff like – NY’s Governor cut school funding by 7%… Now, school districts need to figure out how to cut their spending by 7%! Everyone, 7% less, across the board! Shared sacrifice! It will make everyone better and more efficient in the long run (especially those high poverty districts that we know are least efficient of all)! Pundits argue – Cuts have to be made. Cuomo’s cuts are a perfect example of this reality (from a Dem. Governor). Everyone will be cut. Just suck it up and learn to deal with the New Normal.

Wrong – at so many levels it’s hard to even begin explaining reality to those pundits who’ve clearly never even taken the most basic course on public finance or public school finance and have absolutely no understanding of the interplay between “local” property tax revenues and state aid, or the process of school budget planning and adoption. More on this later.

Thankfully, most readers of my blog and many in the general public actually seem to understand this stuff better than the blowhards (bloghards and tweethards) leading the “new normal” campaign from their DC think tanks. Particularly astute are those families with children who have interest in the quality of their local public schools and live in states where local school district budget setting remains at least partially an open public budgeting process.

And there a few good education writers out there who have developed a solid grip on the interplay between state and local revenues and the resulting effects of state aid cuts. Meghan Murphy, in the Hudson Valley in New York State has done some particularly nice writing on the topic: http://www.recordonline.com/apps/pbcs.dll/article?AID=/20100426/NEWS/100429738

My reason for this post is to expand on a point made by Meghan Murphy in her writing on Hudson Valley Districts and by David Sciarra in this Education Week Article:

But David G. Sciarra, the executive director of the Education Law Center, argued that if states cut funding to school districts during this difficult financial period, the pain will be felt most by disadvantaged students. Impoverished districts have little local property-tax wealth to draw from, and so state aid is a lifeline, said Mr. Sciarra, whose Newark, N.J.-based group advocates for poor students and schools. He urged state officials to work cooperatively with districts in the years ahead to set budget priorities so that current inequities aren’t made worse.

State officials “have an obligation to look for better ways to spend money,” Mr. Sciarra said, but “they’ve got to be very careful in how they do this. Across-the-board cuts and freezes have a negative impact on schools in need.” Ideas about how to cut spending are often proposed at “30,000 feet,” he said, but officeholders need to “take a serious look at how [cuts] would play out in their state.”

Yes, that’s the point. Cuts don’t mean “cuts” or “across the board” uniform distribution, “shared sacrifice,” at least not as they are typically implemented. In general, cuts to state aid lead to increased inequity. They hurt some more than others. Some districts, in fact, have little problem overcoming cuts – rebalancing their budgets, while others, well, to put it simply – are screwed. In many states, those most screwed by aid cuts are those who’ve been most screwed all along.

The model underlying local school budgets is that local voter/citizen/parent/homeowners (not entirely overlapping) desire a certain level or quality of schooling, usually in tangible terms like class sizes or specific programs they wish to see in their schools. That is, the local voter may not be able to “guess” the per pupil expenditure of their district (nor is it particularly relevant if they can) and evaluate whether it’s enough, too much or too little, but the local voter can evaluate whether that per pupil dollar buys the programs and services – and class sizes that voter wants to see in his/her local school district and whether they are willing to add another dollar to that mix.

When the state cuts aid to local school districts the usual first local response is to figure out how to raise at least an equal sum of funding – plus additional funding to accommodate increased costs – so as to maintain the desired schooling (class sizes, programs, etc.). Few local voters seem to really want to cut back service quality. Depending on the state (or type of district within the state), district officials put together a budget requiring a specific amount of revenue – which in turn dictates the property tax rate required to raise that revenue – given the state aid allotted – and then the budget is approved by referendum – or other adoption (or back-up mediation) process.

Clearly some communities have much greater capacity than others to offset their state aid losses with additional local revenues!

For example, when New Jersey handed down state aid cuts to 2010-2011 school budgets and when- for the first time in a long time- the majority of local district budgets statewide failed to achieve approval from local voters, it was still the case that the vast majority (72%) of local budgets passed in affluent communities – in most cases raising sufficient local property tax resources to cover the state aid cuts. In another case, local residents in an affluent suburban community raised privately $420,000 to save full day kindergarten programs. Meghan Murphy’s analysis of Hudson Valley school districts shows that New York State districts also have attempted to counterbalance state aid cuts with property tax increases, but that the districts have widely varied capacity to pull this off. Parents in a Kansas district are suing in federal court requesting injunctive relief to allow them to raise their taxes for their schools (they use faulty logic and legal arguments, but their desire for better schools should be acknowledged!)

Distribution of State Sharing

There’s a bit of important background to cover here. State aid formulas drive state funding – usually from income and sales tax revenues collected to the state general fund – out to local public school districts based on a number of different factors. Typically these days, state funding formulas start with a calculation of the amount of money – state and local – that should be available at a minimum in order to provide adequate public schools. That is, each district is assigned a target amount of state and local funding. That target amount usually varies by the types of students served in a district and by other factors such as regional labor costs and remote, rural locations. In any case, each district ends up with a different estimate of total funding needs.

Next, the aid formula includes a calculation of the amount of that funding target that should be paid for with local property taxes – a local fair share, per se, or local contribution. One approach is to determine how much each district would raise if each district adopted a uniform property tax rate. For those districts that raise more than their target funding with that tax rate alone, the state would kick in nothing. For those districts that implement the local fair share tax rate and still come up short of their target funding, the state would apply aid to cover the difference.

So, for example, you might have three districts in a state, where:

The first district has a very low need student population (almost all from affluent, educated families), and has significant taxable property wealth. That district might have a target funding per pupil of $10,000, and might be able to raise all of it with an even lower tax rate than would be required. In fact, if they put up the local fair share tax rate, they might raise $20,000 per pupil. And the school finance system might allow them to do that.
A second, “middle class” district that has a modest share of children in poverty in the district, leading to an estimated target funding of $12,000 per pupil. The district adopts the local fair share tax rate and raises $8,000. The state allocates the additional $4000.
And the third district, a high need district with weak property tax base, might end up with an estimated target funding of $15,000 per pupil and after adopting the local fair share tax rate only raises $2,000 per pupil, so the state allocates $13,000.

THAT’S THE BASIC STRUCTURE OF A ‘FOUNDATION AID’ FORMULA!!!!

Now, I should be very clear here that the relationship between student needs and tax base is not really that simple. Both parts of this puzzle are critical to the formula and don’t always move in concert. There exist districts with high value tax base and high need student population and vice versa, and for many reasons.

Here’s an example of the distribution of the basic “sharing ratio” for New York State school districts, with respect to district Income/Wealth (IWI) ratio. For the lowest income/wealth districts, the state share is about 90%. For wealthier districts, that ratio – in theory – drops to 0.

Figure 1: NY State Sharing Ratio by District Income/Wealth

Types of Cuts

When it comes to state aid cuts, there’s really nothing for the state to cut from the first district above – that is – unless for some reason the state is giving other money to that district that can raise double it’s need target with the same local tax rate and no state support. But that would be silly, right? More later on that. Here are the two most common approaches to handing out cuts:

Option 1: Cut state aid proportionately across the board

This is actually usually the worst option and most regressive. Let’s take our districts above. A 5% cut to state aid for our first district is, of course, nothing. A 5% cut for our middle class district is $200 per pupil. A 5% cut to state aid for our high need district is $650 per pupil. Yes – the biggest cut comes down on the highest need district.

This distribution of cuts is problematic for two reasons. First, the biggest cut falls on the neediest kids. Second, the biggest cut falls on the district with the least capacity to offset that cut. I’m assuming here that these districts have all adopted the local fair share property tax rate. That rate only raises $2,000 per pupil in revenue for the high need district whereas the same rate raises 4X as much in the middle class district. Clearly for this reason alone, even if the cuts were in equal amounts per pupil, the middle class district would have a much easier time replacing the aid cut with local resources. Further, there a plethora of additional factors that increase the likelihood that the middle class district can offset the cut more easily than the high need, low-income district.

Option 2: Cut funding targets proportionately across the board

A better, though still problematic approach is for the state to recalculate the funding targets – – to reduce that level by just enough to result in the same state aid savings. Taking this approach leads to a constant per pupil cut in state aid across districts, for those districts receiving at least as much state aid per pupil as is being cut.

This too is of no consequence for our district that needs no foundation aid. Let’s say this approach leads to a $400 per pupil across the board reduction in target funding. If we still assume that districts are to raise the same amount of local revenue (local fair share of the fully funded target amount, as opposed to raising the local share of the lowered amount), this would result in a 10% aid cut to the middle class district ($400/$4000) and a 3.1% cut to the high need district. The per pupil target funding change would be the same. But the middle class district would certainly complain that their cut as percent of aid is much larger. But again, that district has 4X the local revenue raising capacity on a dollar per pupil basis, and in this case, they still got less than 4X the per pupil cut.

Even this approach is likely to lead to larger average budget reductions in higher need districts.

Again, there are tons of additional factors involved, and various ways that states might cut aid differently to yield different distributional effects across districts.

State Aid Formulas have Lots of Parts! Some better than others!

The state aid cut scenarios presented above assume a logical and oversimplified world of state aid and local district budgets in many ways. The cut scenarios presented above adopt one really big assumption about state aid to local schools – an assumption that may not be and usually isn’t entirely true:

That the only state aid available to cut is aid that is allocated in proportion to wealth and need across districts. That because state aid is allocated in greater proportion – if not almost in its entirety to poor and need school districts, then those districts must necessarily suffer most from the cuts! As a result, the best you can do is to spread evenly those cuts across wealthy and poor, higher and lower need districts and live with the fact that some will bounce back easier than others.

The fact is that state aid formulas may at their core be built on a seemingly logical foundation funding structure with state and local sharing as described above. But rarely these days does a state aid formula make it into law without a multitude of adjustments and other PORK added on. Yes, Pork! School finance pork and lots of it. Clearly state reps from those towns that would otherwise get nothing from the general aid formula are going to search for a way to bring home some pork.

Here are a few examples of school finance pork from the New York State school finance formula:

1) Minimum Foundation Aid: Like many state funding formulas, even though the first (and most logical) iteration of calculations for estimating the district state share of funding would end up providing 0% state aid to many districts, those formulas include a floor of funding – a minimum guarantee of state aid. That’s right, even our district above that could raise double their target funding by applying the local fair share tax rate would get something. Perhaps this is a reasonable tradeoff when the money is there. But do we really want to keep allocating that money to a district that’s a) fine on its own and b) could easily replace the money with modest increases to local taxes – and would do so.

Here, for example, is the effect of New York State’s minimum threshold factor on foundation aid. The red diamonds indicate the foundation state aid that would be received if districts got what is initially calculated to be their state share. State share hits 0 at an income/wealth index around 1.0 in the basic calculation. But, the actual calculation of state share includes a few adjustments shown in blue squares. First, between income/wealth ratios of about 1.0 to 2.0, the actual state share cuts the corner providing more gradually declining aid rather than going straight to 0. Then, above IWI of 2.0 it never hits 0, but rather levels off providing a minimum allotment of several hundred to about $1,000 per pupil to even the wealthiest districts in the state (which, by the way, are among the wealthiest in the nation!).

Figure 2: Application of Original Calculation and Adjusted Calculation for State Aid Share

Altogether, the adjustments – which also yield additional aid for New York City – add up to nearly $3.8 billion dollars. That’s right… $3.8 billion (where’s that NY Mega Millions guy when you need him?). Now, assuming that it’s hard to get the sharing ratio correct for NYC to begin with and that it is a very high need district that in fact needs this aid, it’s really just over $2.0 billion in potential excess allocation that could be redistributed. That ain’t chump change.

But let’s go really conservative here, and just look at the minimum aid being shuffled out to the richest communities. That alone is still $134 million dollars.

At the very least, if you’re going to cut state aid, cut this first! If you’re not going to cut, consider redistributing this.

2) School Tax Relief Aid: Many state aid formulas include a variety of other types of aid, some which are distributed in flat amounts across all districts regardless of need, and some which may be even allocated in inverse proportion to what most would consider needs – either local capacity related needs or educational programming and student needs. Such is the politics of school finance. For those really interested in this stuff, see the following two articles:

Baker, B.D., Green, P.C. (2005) Tricks of the Trade: Legislative Actions in School Finance that Disadvantage Minorities in the Post-Brown Era American Journal of Education 111 (May) 372-413

Baker, B.D., Duncombe, W.D. (2004) Balancing District Needs and Student Needs: The Role of Economies of Scale Adjustments and Pupil Need Weights in School Finance Formulas. Journal of Education Finance 29 (2) 97-124

New York State’s piece de resistance is a program called STAR, or School Tax Relief program. In simple terms, STAR provides state aid in disproportionate amounts to wealthy communities to support property tax relief. Here’s the distribution of STAR aid with respect to district income/wealth ratios:

Figure 3: STAR Aid per pupil and District Income/Wealth Ratio

Excluding STAR aid to NYC, the aid program in 2008-09 provided $642 million in aid to districts with an income/wealth ratio over 1.0! Even in New York State, that’s not chump change. It’s over $150 million to the wealthiest districts. Add these hundreds of millions to those above and we’re getting somewhere.

Once again, at the very least, if you’re going to cut state aid, cut this first! If you’re not going to cut, consider redistributing this.

For more information on the equity consequences of STAR (as well as simulated solutions), see: http://eus.sagepub.com/content/40/1/36.abstract

A closer look at aid to the wealthy in New York State

Here’s a closer look at minimum foundation aid and STAR aid received by several of the state’s wealthier communities (at least statistically wealthier). Note that our recent report on school funding fairness (www.schoolfundingfairness.org) identified New York State (along with Illinois and Pennsylvania) as having one of the most regressively financed systems in the nation. On average, low poverty districts have greater state and local revenue than high poverty ones, yet the state is still allocating significant sums of aid to low poverty districts!

Table 1: Aid to the Wealthy in New York

This table shows that many of these wealthier communities are picking up millions in STAR aid and upwards to a thousand dollars per pupil in basic foundation aid. Yes, the state is subsidizing the spending – quite significantly – of some of the wealthiest districts in the nation, while maintaining a regressive system as a whole. And now, while cutting aid disproportionately from poor districts.

The distribution of proposed aid cuts

Here is the distribution of the Governor’s proposed cuts in foundation aid to NY State school districts, on a per pupil basis, with respect to Income/Wealth Ratios of school districts:

Figure 4: Governor’s proposed cuts to foundation aid and income/wealth ratios

The income/wealth ratio is along the horizontal axis. The per pupil cuts in aid are on the vertical axis. Here, I’ve represented district size by the size of the “bubble” in the graph. New York City is the bowling ball here! And NYC gets a larger per pupil cut than many much wealthier districts – wealthy districts that actually receive aid! Excuse me, PORK!

Yes, districts with very high income/wealth ratios will experience little cut at all. $100 per pupil or so will look like a big cut of their $1000 per pupil in foundation aid, but some low wealth districts will actually see their foundation aid cut by $1000 per pupil.

And with respect to the Pupil/Need Index:

Figure 5: Governor’s proposed cuts to foundation aid and pupil need index

In this figure, as student needs increase, per pupil cuts in aid increase. Yep, that’s right, districts that NY State itself identifies as having higher needs receive larger per pupil cuts in aid, even while the PORK… yes PORK is retained in the system! All the while, the state continues to allocate minimum foundation aid and STAR funding to wealthy districts.

Summing it up

Yes, it may suck for legislators or the Governor to tell Scarsdale, Pocantico Hills or Locust Valley that there’s just not enough aid to continue to subsidize their “tax relief” (PORK) or to subsidize their very small class sizes and rich array of elective courses and special programs with add-ons such as minimum foundation aid. But aid programs for these districts are merely the bells and whistles – or PORK – of state school finance policies, the political tradeoffs which are often needed to get a formula passed. It’s not a politically easy task, but it’s time to collect that pork and redistribute it to where it’s actually needed.

Further, it is highly unlikely that these districts will actually go without those bells and whistles – that they will forgo their furs and Ferraris. Yes, we all know that school finance formulas are a complicated mix of political tradeoffs. The goal of this post is to make that painfully clear. Let’s call it what it is – school finance Pork. And let’s take that pork and make better use of it. And let’s make it absolutely clear that protecting the pork while slashing basic needs is entirely unacceptable.

There may not be enough pork in the system to either cover all of the proposed cuts, or to be redistributed to fully resolve the funding deficits of higher need districts. But in New York State and many others, there’s quite a bit – quite a bit of aid that could be either used to make the formula fairer to begin with or to buffer the neediest districts and children they serve from suffering the most harmful and real funding cuts.

Note: On Cuts and Caps

Now, some of what I discuss herein is complicated by the current bipartisan political preference to show affluent communities that we’re not going to push these costs off onto them in the form of property tax increases – or backhanded property tax increases through state aid cuts. Instead, we’ll tell them they simply can’t raise their property taxes to cover the difference. That’ll learn ’em!

Capping property taxes while cutting state aid, is simply lying about the true cost of the programs and services desired by local citizens, who invariably have supported tax increases for their local schools and have reverted to major private giving when tax increases were not feasible. There are legitimate reasons to control local spending variation – a topic for another day – and states need to have such policy tools available. But slash-and-cap policies are generally shortsighted and ill-conceived.

Cutting and limiting property taxes merely shifts the burden in yet a different direction – increased fees, private fundraising, volunteering etc. All of which increase inequities in access to quality schooling.

Yeah, I know… all of the pundity types are saying that voters are fed up… they totally underestimate what’s being spent on their schools and they’re totally fed up with paying higher taxes. They don’t want any more of this school spending – bloated bureaucracy, etc. I might buy that if the local voter behavior in affluent communities with preferences for high quality schooling actually supported that argument. But it doesn’t!

If it doesn’t work, don’t do it! CAP’s ROI

The Center for American Progress released its new Return on Investment (ROI) Index for K-12 public school districts of greater than 250 students this week. I should note in advance that I had the opportunity to provide advice on this project early on, and occasionally thereafter and I do believe that at least some involved had and still have the best intentions in coming up with a useful way to represent the information at hand. I’ll get back to the validity of the information at hand in a moment.

First, I need to point out that the policy implications and proposals, or even general findings presented in the report cannot be supported by the analysis (however well or poorly done). The suggestion that billions of dollars might be saved nationally (little more than a back-of-the-napkin extrapolation) at no loss to performance outcomes, based on the models estimated is a huge, unwarranted stretch and quite simply arrogant, ignorant and irresponsible.

The method used provides no reasonable basis for the claim that all low “rate of return” districts could simply replicate the behaviors of high “rate of return” districts and achieve the same or better outcomes at lower cost. The limitations of these methods, when applied in their best and most rigorous complete possible form, no less in this woefully insufficient and incomplete form simply do not allow for such extrapolation.

Further, given the crudeness of the models and adjustments used in the analysis, it is inappropriate to make too much, if anything of supposed differences in the characteristics of districts with good versus bad rate of return indices. You can do some fun hunting and pecking through the maps, but that’s about it!

For example, one major finding is that districts with good ROI’s spend less on administration. However, much more rigorous studies using better data and more appropriate models exploring precisely the same question have found the opposite. The CAP ROI methods are insufficient to draw any conclusion in this regard.

There is little basis in this analysis that states need to provide fairer funding – except that funding does appear to vary within states. But how funding varies is not explored. I like the idea of improving funding fairness, but a better basis for that argument can be found here: www.schoolfundingfairness.org.

And there is no basis for suggesting that fairer funding would be accomplished by student-based funding. Evidence to the contrary might be found here: http://epaa.asu.edu/ojs/article/view/5. This is report writing 101. Be sure that your policy implications follow logically from your findings, and that your findings are actually justified by your analysis.

It was also concluded that on average, higher poverty districts are simply less efficient. When you estimate a model of this type, where you are trying to account for various factors, such as cost factors, outside the control of local school districts – but you aren’t really sure you’ve accomplished the task – when you see a result like this it is a rather basic step to ask yourself – Did I really control sufficiently for costs related to child poverty? Is this finding real? Or is it an indication of bias in my model? A failure to capture real, important differences in the characteristics of these districts?

That single bias – failure to fully account for poverty related costs – is pervasive throughout the entire CAP ROI analysis. There is a strong relationship between poverty and supposed inefficiency in most states in the analysis. That bias exists in states where the state has provided additional resources to higher poverty districts, making them higher spending on average, and that bias exists even in states where spending per pupil is systematically lower in higher poverty districts. And every map and scatterplot in the analysis must be viewed carefully with an understanding of the pervasive, uncontrolled bias against higher poverty districts. A bias that results largely from failure to fully account for cost variation.

Okay, now that I’ve said that rather bluntly, let’s walk through the three different ROI approaches, and what they are missing.

Basic ROI

In any of the ROI’s you’ve got two sides to the analysis. You’ve got the student outcome measures (which I’ll spend less time on), and you’ve got the per pupil spending measures. Within the per pupil spending measures, you’ve got cost adjustments for the “cost” of meeting student population needs and cost adjustments for addressing regional differences in competitive wages for school personnel. The Basic ROI uses an approach similar to that used by Education Week in Quality Counts as a basis for calculating “cost adjusted spending per pupil.”

Weighted Pupil Count = Enrollment + .4*Free Lunch Count + .4*ELL Count + 1.1*IEP Count

After using the weighted pupil count to generate a student need adjustment, CAP uses the NCES Comparable Wage Index to adjust for regional variation in wages. So, they try to adjust for student needs, using a series of arbitrary weights, and for regional wage variation.

The central problem with this approach is that it relies on setting rather arbitrary weights to account for the cost differences associated with poverty, ELL and special education. And in this case, CAP, like Ed Week, shot low – claiming those low weights to be grounded in research literature, but that claim is a stretch at best and closer to a complete misrepresentation. More below.

Adjusted ROI

For the adjusted ROI, CAP uses a regression equation which compares the actual spending of each district to the predicted spending of each district, given student population characteristics. Here’s their equation:

ln(CWI adjusted ppe)= β0 + β1% free lunch + β2 % ELL+ β3 % Special Ed + ε

Now, this method is a reasonable one for comparing how much districts spend, but has little or nothing to do with adjusting for the costs of achieving comparable educational outcomes – a true definition of cost. That is, one can use a spending regression model to determine if a state, on average spends more on high poverty than on low poverty districts. But this is a spending differential not a cost factor. It’s useful, and has meaning, but not the right meaning for this context. One would need to determine how much more or less needs to be spent in order to achieve comparable outcomes.

So, for example, using this approach it might be determined that within a state, higher poverty districts spend less on average than lower poverty districts. This negative or regressive poverty effect would become the cost adjustment. That is, it would be assumed that higher poverty districts have lower costs than lower poverty ones. NO. They have lower spending, but they still most likely have higher costs of achieving constant educational outcomes. Including outcomes and holding outcomes constant is the key – AND MISSING – step toward using this approach to adjust for costs.

Further, the overly simplistic equation above completely ignores significant factors that do affect cost differences and/or spending differences across districts, such as economies of scale and population sparsity as well as more fine grained differences in teacher wages needed to recruit or retain comparable teachers across districts of differing characteristics within the same labor market.

Predicted Efficiency

Finally, there’s the predicted efficiency regression equation, which attempts to generate a predicted achievement level based on a) cost adjusted per pupil spending, b) free lunch, ELL and special education shares. This one, like the others doesn’t attempt to adjust for economies of scale or sparsity and suffers from numerous potential problems with figuring out how and why each district’s actual performance differs from its predicted performance.

achievement = β0 + β1 ln(CWI adjusted ppe) + β2 % free lunch + β3 % ELL + β4 %Special Ed + ε

In this (dreadfully over-) simplified production function approach, any individual district’s actual outcomes could be much lower than predicted or much higher than predicted for any number of reasons. It would appear from scanning through the findings that this particular indicator is most biased with respect to poverty.

Summary of what’s missing or mis-specified

The table below summarizes the three ROI indices – or at least the “adjusted expenditure” side of those indices – with respect to what we know are the major cost factors that must be accounted for in any reasonable analysis of education spending data in relation to student outcomes. Here, the basic conception of cost, and cost difference is “what are the differences in cost toward achieving comparable outcome objectives?” Cost cannot be estimated without an outcome objective.

First, I would argue that the selected weights in the Basic ROI are simply too low, especially in certain parts of the country.

Second, none of the models address economies of scale. CAP notes this, but in a section of the report most will never read. Instead, we’ll all see the pretty maps that tell us that all of the rural districts in the upper Hudson Valley in NY State or in north Central Pennsylvania are really, really inefficient.

Third, recall that the “adjusted ROI” model really doesn’t control for cost at all, but rather for underlying spending variation, without respect for outcomes.

Table 1

Regarding pupil need weights in particular, there exists at least some literature – the most rigorous and direct literature on the question – which suggests the need for much higher weights than those used by CAP. For example, Duncombe and Yinger note that in two versions of their models:

Overall, this poverty weight ranges from 1.22 to 1.67 (x census poverty rate), the LEP weight ranges from 1.01 to 1.42, and the special education weight varies from 2.05 to 2.64.

Across several models produced in this particular paper, one might come to a rounded weight on Census poverty of about 1.5 or weight on subsidized lunch rates of about 1.0 (100% above average cost, or 2x average, more than double the CAP weight), a weight on limited English proficient students around 1.0 and on special education students over 2.0 (slightly less than double the CAP weight).

Other work by me, along with Lori Taylor and Arnold Vedlitz, done for the National Academies of Science, reviewing numerous studies also comes to higher average weights – using credible methods – for children in poverty.

While one can quibble over the selection of “cost” weights from literature, the bigger deal for me remains that the findings of the various ROIs reflect such a strong bias that any reasonable researcher would be obligated to explore further, and perhaps test out alternative research based weights as a way to reduce the bias. It’s a never ending battle, and when you’ve improved the distribution in one state, you’ve likely messed it up in another (because different patterns of poverty and distributions of ELL children lead to different appropriate weights in different settings – even within a state). But this happens – if it turns out to simply be unreasonable to identify a global method for estimating ROIs across school districts and across states, THEN STOP!!!!! DON’T DO IT!!!!! IT JUST DOESN’T WORK!!!!

Here is an example of how much a corrected cost adjustment might matter, when compared with the Basic ROI. The scatterplot below includes one set of dots (red triangles) which represent adjusted operating expenditures of Illinois school districts using the Basic ROI weights. The other set of dots (blue circles) uses a cost index derived from a more thorough statistical model of the costs of achieving statewide average outcomes for Illinois school districts. For the highest poverty districts, the adjusted spending figures drop by $4,000 to $5,000 per pupil when the more thorough cost adjustment method is used. This is substantial, and important, since the ROI is much more likely to identify these districts as inefficient and might be used by state policy makers to argue that cuts to these districts are appropriate (when they clearly are not).

Figure 1

How you specify models to identify efficient or inefficient districts matters, a lot!

Here’s one example of how this type of analysis can produce deceiving results, simply based on the “shape” of the line fit to the scatter of districts. Below is a scatterplot of the cost adjusted spending per pupil for Illinois school districts (unified K-12 districts) in 2008, and the proficiency rates (natural log) for those districts. In this case, I’m actually using much more fully cost adjusted spending levels, accounting for regional and more local wage variation, accounting for desired outcome levels, for poverty, language proficiency, racial composition and economies of scale. As a result, the graph actually shows a reasonable relationship between cost adjusted operating expenditures per pupil and actual outcomes. Spending – when appropriately adjusted – is related to outcomes.

Figure 2

Even then, it’s a bit hard to figure out what shape “best fit” line/curve should go through this scatter. If I throw a straight line in there, and compare each district against the straight line, those districts below the line at the left hand side of the picture are identified as really inefficient – getting much lower outcome than the trendline predicts. But, if I were to fit a curve instead (I’ve simply drawn this one, for illustrative purposes), I might find that some districts previously identified as below the line are now above the line. Are they inefficient, or efficient? Who really knows, in this type of anlaysis!

My biggest problem with the CAP production function analysis is that they came to a result that is so strongly biased on the basis of poverty and instead of questioning whether the model was simply biased – missing important factors related to poverty – they accepted as truth – as a major finding that higher poverty districts are less efficient. It is indeed possible that this is true, but the CAP analysis does not provide any compelling evidence to this effect.

Research literature on this stuff

Note that there is a relatively large literature on this stuff… on whether or not we can, with any degree of precision, classify the relative efficiency of schools or districts. There are believers and there are skeptics, but even among the believers and the skeptics, all are applying much more rigorous methods and refined models and more fully accounting for various cost factors than the present CAP analysis. Here are some worthwhile readings:

Robert Bifulco & William Duncombe (2000) Evaluating School Performance: Are we ready for prime time? In William Fowler (Ed) Developments in School Finance, 1999 – 2000. Washington, DC: National Center for Education Statistics, Office of Educational Research and Improvement.

Robert Bifulco and Stewart Bretschneider (2001) Estimating School Efficiency: A comparison of methods using simulate data. Economics of Education Review 20

Ruggiero, J. (2007) A comparison of DEA and Stochastic Frontier Model using panel data. International Transactions in Operational Research 14 (2007) 259-266

What, if anything, can we learn from those pretty maps and scatters?

Now, moving beyond all of my geeky technical quibbling is there anything we actually can learn from the cool maps and scatters that CAP presents to us. First, and most important, any exploration of the data has to be undertaken with the understanding that all 3 ROI’s suffer from a severe bias toward labeling high poverty urban districts as inefficient and affluent suburban districts as highly efficient (especially in Kansas!). But, with that in mind, one can find some interesting contrasts.

First, I think it would be useful for CAP to reframe and re-label their color schemes. Here’s my perspective on their scatters and color coding. The assumption with the ROI is that there exists an expected relationship between adjusted spending and student outcomes. That’s the diagonal line. Districts in the lower left and upper right are essentially where they are supposed to be. There is nothing particularly inefficient about being in the lower left, or upper right. The use of orange to represent the lower left makes it seem like the lower left is like the lower right. The lower left hand districts in the scatterplot, in theory, are those that got screwed on funding and have low outcomes. Arguably, the lower left hand quadrant of the scatterplots is where one should go looking for school districts wishing to sue their state over inequitable and inadequate funding. These districts aren’t to blame. They are getting what’s expected of them. They are getting slammed on funding and their kids are suffering the consequences – that is, if there really is any precision (which is a really, really suspect assumption) to these models.

Figure 3

Historically, Pennsylvania has operated one of the least equitable, most regressive state school finance formulas in the nation (www.schoolfundingfairness.org). Philadelphia has been one of the least well funded large poor urban core districts in the nation. Strangely, Pittsburgh has made out much better financially. Here’s what happens when we identify the locations of a few Pennsylvania school districts in the CAP ROI interactive tool. I’ve recreated the locations of 4 districts. The location of Philadelphia actually makes some sense on the basic ROI. Philly is royally screwed. Low funding and low outcomes. The implication of the orange shading seems problematic. But if we ponder the meaning of the lower left quadrant it all makes sense. Now, I’m not sure Pittsburgh is really overfunded and/or inefficient, as implied by being in the lower right quadrant – but at least relative to Philadelphia, it does make sense that Pittsburgh falls to the right of Philadelphia on the scatterplot. Lower Merion, an affluent high spending suburb of Philly seems to be in the right place too. I’m not sure, however, what to make of any of the districts, including affluent suburban Central Bucks, which fall in the upper left – the Superstars.

Figure 4

Because the various ROIs generally under-compensate for poverty related costs, if a district falls in that lower left hand quadrant, we can be pretty sure that the district is relatively underfunded as well as low performing. That is, the district shows up as underfunded even when we don’t fully adjust for costs. This is especially true for those districts that fall furthest into the lower left hand corner. The basic ROI is most useful in this regard, because you know what you’re getting (specific underlying weights). I’ve opened up the comments section so you all can help me identify those notable lower left quadrant districts!

Finally

This type of analysis is an impossible task, especially across all states and dealing with vastly different student outcome data as well as widely varied cost structures. Only precise state by state analysis can yield more useful information of this type. A really important lesson one has to learn when working with data of this type is to realize when the original idea just doesn’t work. I’ve been there a lot myself, even trying this very activity on more than one occasion. There comes a point where you have to drop it and move on. Sometimes you just can’t make it do what you want it to. And sometimes what you want it to do is wrong to begin with. Releasing bad information can be very damaging, especially information of this type in the current political context.

But even more disconcerting, releasing bad data, acknowledging many of the relevant caveats, but then drawing bold and unsubstantiated conclusions that fuel the fire… that endorse slashing funds to high need districts and the children they serve – on a deeply flawed and biased empirical basis – is downright irresponsible.

References

Andrews, M., Duncombe, W., Yinger, J. (2002). Revisiting economies of size in American education: Are we any closer to consensus? Economics of Education Review, 21, 245-262.

Baker, B.D. (2005) The Emerging Shape of Educational Adequacy: From Theoretical Assumptions to Empirical Evidence. Journal of Education Finance 30 (3) 277-305

Baker, B.D., Taylor, L.L., Vedlitz, A. (2008) Adequacy Estimates and the Implications of Common Standards for the Cost of Instruction. National Research Council.

Duncombe, W. and Yinger, J.M. (2008) Measurement of Cost Differentials In H.F. Ladd & E. Fiske (eds) pp. 203-221. Handbook of Research in Education Finance and Policy. New York: Routledge.

Duncombe, W., Yinger, J. (2005) How Much more Does a Disadvantaged Student Cost? Economics of Education Review 24 (5) 513-532

Taylor, L. L., Glander, M. (2006). Documentation for the NCES Comparable Wage Index Data File (EFSC 2006-865). U.S. Department of Education. Washington, DC: National Center for Education Statistics.

Unspinning Data on New Jersey Charter Schools

Today’s (okay…yesterday… I got caught up in a few other things) New Jersey headlines once again touted the supposed successes of New Jersey Charter Schools:

http://www.nj.com/news/index.ssf/2011/01/gov_christie_releases_study_sh.html

The Star Ledger reporters, among others, were essentially reiterating the information provided them by the New Jersey Department of Education. Here’s their story.

http://www.state.nj.us/education/news/2011/0118chart.htm

And here’s a choice quote from the press release:

“These charter schools are living proof that a firm dedication to students and a commitment to best education practices will result in high student achievement in some of New Jersey’s lowest-income areas,” said Carlos Perez, chief executive officer of the New Jersey Charter School Association. He pointed to NJASK data for third grade Language Arts, where more than half the charters outperformed the schools in their home districts, and of those, more than 75 percent were located in former Abbott districts.

No spin there. Right? Just a balanced summary of achievement data, with thoughtful interpretation of what they might actually mean. Not really.

There are many, many reasons why the comparisons released yesterday are deeply problematic, and well, quite honestly, pretty darn meaningless. I could not have said it better than Matt DiCarlo of Shanker Blog did here:

“Unfortunately, however, the analysis could barely pass muster if submitted by a student in one of the state’s high school math classes (charter or regular public).”

Here are some guidelines I have posted in the past, regarding appropriate ways to compare New Jersey Charter Schools to their host districts on various measures including outcome measures:

When comparing across schools within poor urban setting, compare on basis of free lunch, not free or reduced, so as to pick up variation across schools. Reduced lunch income threshold too high to pick up variation.
When comparing free lunch rates across schools either a) compare against individual schools and nearest schools, OR compare against district averages by GRADE LEVEL. Subsidized lunch rates decline in higher grade levels (for many reasons, to be discussed later). Most charter schools serve elementary and/or middle grades. As such they should be compared to traditional public schools of the same grade level. High school students bring district averages down.
When comparing test score outcomes using NJ report card data, be sure to compare General Test Takers, not Total Test Takes. Total Test Takers include scores/pass rates for children with disabilities. But, as we have seen time and time again, in charts above, Charters tend not to serve these students. Therefore, it is best to exclude scores of these students from both the Charter Schools and Traditional Public Schools.

Today’s (okay, yesterday – publication lag) primary violation involves #3 above, but also relates to the first two basic rules. Let’s do a quick walk through, using the 2009 data, because the 2010 school level school reports data are not yet posted on the NJDOE web site. The bottom line is that it is relatively meaningless to simply compare raw scores or proficiency rates of charter schools to host district schools – as done by NJDOE and the Star Ledger. That is, it is meaningless unless they actually serve similar student populations, which they do not.

Below, I walk through a few quick examples of student population differences in Newark, home to the state’s high-flying charter schools (North Star Academy and Robert Treat Academy). Next, I construct a statistical model of school performance including New Jersey Charter schools and traditional public schools in their host district, controlling for student demographics and location. I first used this same model here: Searching for Superguy in New Jersey. I use that model to show adjusted performance comparisons on a few of the tests, and then I use a variation of that model to test the proficiency rate difference – on average statewide – between charter schools and schools in the host district. Finally, I address one additional factor which I am unable to fully control for in the model – the fact that some New Jersey Charter Schools – high performing ones – seem to have unusually high rates of cohort attrition between grade 6 and 8, concurrent with rising test scores. I raise this point because pushing out of students is not an option available to traditional public schools. In fact, it is the traditional public schools that must take back those students pushed out.

Demographic Examples from Newark

Here are a few slides from previous posts on the demography of Newark Charter Schools in particular, compared to other Newark Public Schools. Here are the shares of kids who qualify for free lunch by school in Newark (city boundaries). Clearly, most of the charters fall toward the left hand side of the graph with far fewer of the lowest low-income children.

The shares of English Language Learners look similar if not more dramatic. Many NPS schools have very high rates of English Language Learners while few charters have even a modest share.

Finally, here’s a 4 year run of the most recent available special education classification rate data (More recent years of data have a dead link on the classification rates). This graph compares Essex County charter schools with Essex County public school districts. Charter Schools have invariably low special education rates, but for those focused on children with disabilities.

One cannot reasonably ignore these differences when comparing performance outcomes of kids across schools. It’s just silly and not particularly useful.

The Outcomes Corrected for the Demographics

So then, what happens if we actually use some statistical adjustments to evaluate whether the charter schools outperform (on average proficiency rate) other schools in the same city on the same test. Well, I’ve done this for charter data from 2009 and previous years and will do it again for the 2010 data when available. I use variables available in the Fall Enrollment Files and from the School Report Card and information on school location from the NCES Common Core of Data in order to create a model of the expected scores for each charter school and each other school in the same city. In the model, I use only the performance of GENERAL TEST TAKERS, so as to exclude those scores of special education students (who, for the most part don’t attend the charter schools). The model:

Outcome = f(Poverty, Race, Homelessness, City, Tested Grade, Subject)

Is use the model to create a predicted performance level (proficiency rate) for each school, considering which grade level test we are looking at, in which subject, the race/ethnicity of the students (where Hispanic concentration is highly correlated with available ELL data, and Hispanic concentration data are more consistently reported), the share of students qualifying for free lunch, the percent identified as homeless and the city of location for the school. That is, each charter school is effectively compared against only other schools in the same geographic context (city).

This is a CRUDE model, which can’t really account for other factors, such as the possibility that some charter schools actually shed, or push out, lower performing students over time. More on that below. So, for each school, I get a predicted performance level – what that school is expected to achieve given the children it serves and the location. I can then compare the actual performance to the predicted performance to determine whether the school beats expectations or falls below expectations.

The next two graphs provide a visual representation of schools beating the odds and schools under-performing with respect to expectations. Charters are identified in red and named. Blue circles are traditional public schools in the same district. Note that there are about the same number of charters beating expectations as there are falling short. The same is true for non-charters. On average, both groups appear to be about average.

8th Grade Math performance looks much like 4th grade. Charters are evenly split between “good” and “bad,” as are the traditional public schools in their host districts.

The Overall Charter Difference (Or Not?)

Now, the above graphs don’t directly test whether the average charter performance is better or worse than the average non-charter performance on the same test, same grade and in the same location. But, conducting that test (for these purposes) is as simple as adding into the statistical model an indicator of whether a school is a charter school. Doing so creates a simple (oversimplified, in fact) comparison of the average performance of charters to the average performance of non-charters in the same city (on the same test, in the same grade level), while “correcting” statistically for differences in the student population. I SHOULD POINT OUT THAT ONE CAN NEVER REALLY FULLY CORRECT FOR THOSE DIFFERENCES!

Using this oversimplified method, the analysis (statistical output) below shows that the charter average proficiency rate is about 3% higher than the non-charter average – BUT THAT DIFFERENCE IS NOT STATISTICALLY SIGNIFICANT. That is, there really isn’t any difference. THAT IS, THERE REALLY ISN’T ANY DIFFERENCE.

Some Other Intervening Factors: Cohort Attrition, or Pushing Out

As I mentioned above, even the “tricky statistics” I used cannot sort out such things as a school that systematically dumps, or pushes out lower performing students, where those lower performing students end up back in the host district. Such an effect would simultaneously boost the charter performance and depress the host district performance (if enough kids were pushed back). I’ve written on this topic previously. So, I’ll reuse some of the older stuff – which isn’t really that old (last Fall).

In this figure, we can see that for the 2009 8th graders, North Star began with 122 5th graders and ended with 101 in 8th. The subsequent cohort also began with 122, and ended with 104. These are sizable attrition rates. Robert Treat, on the other hand, maintains cohorts of about 50 students – non-representative cohorts indeed – but without the same degree of attrition as North Star. Now, a school could maintain cohort size even with attrition if that school were to fill vacant slots with newly lotteried-in students. This, however, is risky to the performance status of the school, if performance status is the main selling point.

Here, I take two 8th grade cohorts and trace them backwards. I focus on General Test Takers only, and use the ASK Math assessment data in this case. Quick note about those data – Scores across all schools tend to drop in 7th grade due to cut-score placement (not because kids get dumber in 7th grade and wise up again in 8th). The top section of the table looks at the failure rates and number of test takers for the 6th grade in 2005-06, 7th in 2006-07 and 8th in 2007-08. Over this time period, North Star drops 38% of its general test takers. And, cuts the already low failure rate from nearly 12% to 0%. Greater Newark also drops over 30% of test takers in the cohort, and reaps significant reductions in failures (partially proficient) in the process.

The bottom half of the table shows the next cohort in sequence. For this cohort, North Star sheds 21% of test takers between grade 6 and 8, and cuts failure rates nearly in half – starting low to begin with (starting low in the previous grade level, 5th grade, the entry year for the school). Gray and Greater Newark also shed significant numbers of students and Greater Newark in particular sees significant reductions in share of non(uh… partially)proficient students.

My point here is not that these are bad schools, or that they are necessarily engaging in any particular immoral or unethical activity. But rather, that a significant portion of the apparent success of schools like North Star is a) attributable to the demographically different population they serve to begin with and b) attributable to the patterns of student attrition that occur within cohorts over time.

Understanding Differing Perspectives

Some will say, why should I care if charters are producing higher outcomes with similar kids? What matters to me is that they are producing higher outcomes! Anyone who produces higher outcomes in Newark or Trenton should be applauded, no matter how they do it. It’s one more high performing school where there wasn’t one previously.

It is important to understand that comparisons of student outcomes that ignore differences in student populations reward – in the public eye – those schools that manage to find a way to serve more advantaged populations, either by achieving non-representative initial lottery pool or by selective attrition. As a result, there is a disincentive for charter operators to actually make greater effort to serve higher need populations – the ones who really need it! And there are many out there who see this as their real mission. Those charter operators who do try to serve more ELL children, more children in severe multi-generational poverty, and children with disabilities often find themselves answering tough questions from their boards of directors and the media regarding why they can’t produce the same test scores as the high-flying charter on the other side of town. These are not good incentives from a public policy perspective. They are good for the few, not the whole.

Further, one’s perspective on this point varies whether one is a parent looking for options for his/her own child, or a policymaker looking for “scalable” policy options for improving educational opportunities for children statewide. From a parent (or child) perspective, one is relatively unconcerned whether the positive school effect is function of selectivity of peer group and attrition, so long as there is a positive effect. But, from a public policy perspective, the “charter model” is only useful if the majority of positive effects are not due to peer group selectivity and attrition, but rather to the efficacy and transferability of the educational models, programs and strategies. Given the uncommon student populations served by many Newark charters and even more uncommon attrition patterns among some… not to mention the grossly insufficient data… we simply have no way of knowing whether these schools can provide insights for scalable reforms.

As they presently operate, however, many of the standout schools do not represent scalable reforms. And on average, New Jersey charters are still… just… average.

Understanding Education Costs versus “Inflation”

We often see pundits arguing that education spending has doubled over a 30 year period, when adjusted for inflation, and we’ve gotten nothing for it. We’ve got modest growth in NAEP scores and huge growth in spending. And those international comparisons… wow!

The assertion is therefore that our public education system is less cost-effective now than it was 30 years ago. But this assumption is based on layers of flawed reasoning, on both sides of the equation.

Here’s a bit of School Finance 101 on this topic:

First, what are the two sides of the equation, or at least the two parts of the fraction? The numerator here is education spending and how we measure it now compared to previously. The major flaw in the usual reasoning is that we are making our comparison of the education dollar now to then by simply adjusting the value of that dollar for the average changes in the prices of goods purchased by a typical consumer (food, fuel, etc.), or the Consumer Price Index.

Unfortunately, the consumer price index is relatively unhelpful (okay, useless) for comparing current education spending to past education spending, unless we are considering how many loaves of bread or gallons of gas can be purchased with the education dollar.

If we wanted to maintain constant quality education over time, the main thing we’d have to do is maintain a constant quality workforce in schools – mainly a teacher workforce, but also administrators, etc. At the very least, if quality lagged behind we’d have to be able to offset the quality losses with additional workers, but the trade-offs are hard to estimate.

The quality of the teacher workforce is influenced much more by the competitiveness of the wages for teachers, compared to other professions, than to changes in the price of a loaf of bread or gallon of gas. If we want to get good teachers, teaching must be perceived as a desirable profession with a competitive wage. That is, to maintain teacher quality we must maintain the competitiveness of teacher wages (which we have not over time) and to improve teacher quality, we must make teacher wages (or working conditions) more competitive. On average, non-teacher wage growth has far outpaced the CPI over time and on average, teacher wages have lagged behind non-teacher wages, even in New Jersey!

Now to the denominator or the outcomes of our education system. First of all, if we allow for a decline in the quality of the key input – teachers – we can expect a decline in the outcomes however we choose to measure them. But, it is also important to understand that if we wish to achieve either higher outcomes, or to achieve a broader array of outcomes, or to achieve higher outcomes in key areas without sacrificing the broader array of outcomes, costs will rise. In really simple terms, the cost of doing more is more, not less. And yes, a substantial body of rigorous peer-reviewed empirical literature supports this contention (a few examples below).

So, as we ask our schools to accomplish more we can expect the costs of those accomplishments to be greater. If we expect our children to compete in a 21^st century economy, develop technology skills and still have access to physical education and arts, it will likely cost more, not less, than achieving the skills of 1970. But, we must also make sure we are adequately measuring the full range of outcomes we expect schools to accomplish. If we are expecting schools to produce engaged civic participants, we may or may not see the measured effects in elementary reading and math test scores.

An additional factor that affects the costs of achieving educational outcomes is the student inputs – or who is showing up at the schoolhouse door (or logging in to the virtual school). A substantial body of research (see chapter by Duncombe and Yinger, here) explains how child poverty, limited English proficiency, unplanned mobility and even school racial composition may influence the costs of achieving any given level of student outcomes. Differences in the ways children are sorted across districts and schools create large differences in the costs of achieving comparable outcomes and so too do changes in the overall demography of the student population over time. Escalating poverty, and mobility induced by housing disruptions, increased numbers of children not speaking English proficiently all lead to increases of the cost of achieving even the same level of outcomes achieved in prior years. This is not an excuse. It’s reality. It costs more to achieve the same outcomes with some students than with others.

In short, the “cost” of education rises as a function of at least 3 major factors:

Changes in the incoming student populations over time
Changes in the desired outcomes for those students, including more rigorous core content area goals or increased breadth of outcome goals
Changes in the competitive wage of the desired quality of school personnel

And the interaction of all three of these! For example, changing student populations making teaching more difficult (a working condition), meaning that a higher wage might be required to simply offset this change. Increasing the complexity of outcome goals might require a more skilled teaching workforce, requiring higher wages.

The combination of these forces often leads to an increase in education spending that far outpaces the consumer price index, and it should. Cost rise as we ask more of our schools, as we ask them to produce a citizenry that can compete in the future rather than the past. Costs rise as the student population inputs to our public schooling system change over time. Increased poverty, language barriers and other factors make even the current outcomes more costly to achieve. And costs of maintaining the quality of the teacher workforce change as competitive wages in other occupations and industries change, which they have.

Typically, state school finance systems have not kept up with the true increased costs of maintaining teacher quality, increased outcome demands or changing student demography. Nor have states sufficiently targeted resources to districts facing the highest costs of achieving desired outcomes. See www.schoolfundingfairness.org. And many states, with significantly changing demography including Arizona, California and Colorado have merely maintained or even cut current spending levels for decades (despite what would be increased costs of even maintaining current outcome levels).

Evaluating education spending solely on the basis of changes in the price of a loaf of bread and/or gallon of gasoline is, well, silly.

Notably, we may identify new “efficiencies” that allow us to produce comparable outcomes, with comparable kids at lower cost. We may find some of those efficiencies through existing variation across schools and districts, or through new experimentation. But it is downright foolish to pretend that those efficiencies are simply out there (even if we can’t see them, or don’t know them) and we can simply squeeze the current system into achieving comparable or better outcomes at lower cost.

Readings

Baker, B.D., Taylor, L., Vedlitz, A. (2008) Adequacy Estimates and the Implications of Common Standards for the Cost of Instruction. National Research Council. http://www7.nationalacademies.org/CFE/Taylor%20Paper.pdf

Duncombe, W., Lukemeyer, A., Yinger, J. (2006) The No Child Left Behind Act: Have Federal Funds been Left Behind? http://cpr.maxwell.syr.edu/efap/Publications/costing_out.pdf

This second one is a really fun article showing the vast differences in the costs of achieving NCLB proficiency targets in two neighboring states which happen to have very different testing standards. In really simple terms, Missouri has a hard test with low proficiency rates and Kansas and easy test with high proficiency rates. The authors show the cost implications of achieving the lower, versus higher tested achievement standards.

Thinking through cost-benefit analysis and layoff policies

If you’re running a school district or a private school and you are deciding on what to keep in your budget and what to discard, you are making trade-offs. You are making trade-offs as to whether you want to spend money on X or on Y, or perhaps a more complicated mix of many options. How you come to your decision depends on a number of factors:

The cost – the total costs of the various ingredients that go into providing X and providing Y. That is, how many people, at what salary and benefits, how much space at what overhead cost (per time used) and how much stuff (materials, supplies and equipment) and at what market prices?
The benefits – the potential dollar return to doing X versus doing Y. For example, how much dollar savings might be generated in operating cost savings from reorganizing our staffing and use of space, if we spend up front (capital expenses) to reorganize and consolidate our elementary schools where they have become significantly imbalanced over time?
The effects – the relative effectiveness of doing X versus doing Y. For example, in the simplest case, if we are choosing between two reading programs, what are the reading achievement gains, or effects, from each program? Or, more pertinent to the current conversation (but far more complex to estimates), what are the relative effects of reducing class size by 2 students when compared to keeping a “high quality” teacher.
The utility – The utility of each option refers to the extent that the option in question addresses a preferred outcome goal. Utility is about preferences, or tastes. For example, in the current accountability context, one might be pressured to place greater “utility” on improving math or reading outcomes in grades 3 through 8. If the costs of a preferred program are comparable to the costs of a less preferred program… well… the preferred program wins. There are many ways to determine what’s “preferred,” and more often than not, public input plays a key role especially in smaller, more affluent suburban school districts. As noted above, federal and state policy have played a significant role in defining utility in the past decade (and arguably, distorting resource allocation to a point of significant imbalance in resource-constrained districts)

This basic cost analysis framework laid out by Henry Levin back in 1983 and revisited by Levin and McEwan since should provide the basis for important trade-off decisions in school budgeting and should provide the conceptual basis for arguments like those made by Petrilli and Roza in their recent policy brief. But such a framework is noticeably absent and likely so because most of the proposals made by Petrilli and Roza:

are not sufficiently precise to apply such a framework largely because little is known about the likely outcomes (which may in fact be quite harmful); and
because they have failed entirely to consider in detail the related costs of proposed options, especially up-front costs of many of the options (like school reorganization or developing teacher evaluation systems). Note that the full length book (from which the brief comes) is no more thoughtful or rigorous.

Back of the Napkin Application to Layoff Options

Allow me to provide a back-of-the-napkin example of some of the pieces that might go into determining the savings and/or benefits from the BIG suggestion made by Pettrilli and Roza – which is to use quality based layoffs in place of seniority based layoffs when cutting budgets. This one would seem to be a no-brainer. Clearly, if we layoff based on quality, we’ll have better teachers left (greater effectiveness) and we’ll have saved a ton money or a ton of teachers. That is, if we are determined to layoff X teachers, it will save more money to lay off more senior, more expensive teachers than to lay off novice teachers. However, that’s not the likely what-if scenario. More likely is that we are faced with cutting X% of our staffing budget, so the difference will be in the number of teachers we need to lay off in order to achieve that X%, and the benefit difference might be measured in terms of the change in average class size resulting from laying off teachers by “quality” measures and laying off teachers by seniority.

Let’s lay out some of the pieces of this cost benefit analysis to show its complexity.

First of all, let’s consider how to evaluate the distribution of the different layoff policies.

Option 1 – Layoffs based on seniority

This one is relatively easy and involves starting from the bottom in terms of experience and laying off as many junior teachers as necessary to achieve 5% savings to our staffing budget.

Option 2 – Layoffs based on quality

Here’s the tricky part. Budget cuts and layoffs are here and now. Most districts do not have in place rigorous teacher evaluation systems that would allow them to make high stakes decisions based on teacher quality metrics. AND, existing teacher quality metrics where they do exist (NY, DC, LA) are very problematic. So, on the one hand, if districts rush to immediately implement “quality” based layoffs, districts will likely revert to relying heavily on some form of student test score driven teacher effectiveness rating, modeled crudely (like the LA Times model). Recall that even in better models of this type, we are looking at a 35% chance of identifying an average teacher as “bad” and 20% chance of identifying a good teacher as “bad.”

In general, the good and bad value-added ratings fall somewhat randomly across the experience distribution. So, for simplicity in this example, I will assume that quality based firings are essentially random. That is, they would result in dismissals randomly distributed across the experience range. Arguably, value-added based layoffs are little more than random, given that a) there is huge year to year error even when comparing on the same test and b) there are huge differences when rating teachers using one test, versus using another.

Testing this out with Newark Public Schools – Elementary Classroom Teachers 2009-10

At the very least, one would think that randomly firing our way to a 5% personnel budget cut would create a huge difference when compared to firing our way to a 5% personnel budget cut by eliminating the newest and cheapest teachers. I’m going to run these numbers using salaries only, for illustrative purposes (one can make many fun arguments about how to parse out fixed vs. variable benefits costs, or deferred benefits vs. short run cost differences for pensions and deferred sick pay, etc.).

We start with just over 1,000 elementary classroom teachers in Newark Public Schools, and assume an average class size of 25 for simplicity. The number of teachers is real (at least according to state data) but the class sizes are artificially simplified. We are also assuming all students and classroom space to be interchangeable. A 5% cut is about $3.7 million. Let’s assume we’ve already done our best to cut elsewhere in the district budget, perhaps more than 5% across other areas, but we are left with the painful reality of cutting 5% from core classroom teachers in grades K-8. In any case, we’re hoping for some dramatic saving here – or at least benefits revealed in terms of keeping class sizes in check.

Figure 1: Staffing Cut Scenarios for Newark Public Schools using 2009-10 Data

If we layoff only the least experienced teachers to achieve the 5% cut, we layoff only teachers with 3 or fewer years of experience when using the Newark data. The average experience of those laid off is 1.8 years. And we end up laying off 72 teachers (a sucky reality no matter how you cut it).

If we use a random number generator to determine layoffs (really, a small difference from using Value-added modeling), we end up laying off only 54 teachers instead of 72. We save 18 teachers, or 1.7% of our elementary classroom teacher workforce.

What’s the class size effect of saving these 18 teachers? Well, under the seniority based layoff policy, class size rises from 25 to 26.86. Under the random layoff policy, class size rises from 25 to 26.37. That is, class size is affected by about half a student per class. This may be important, but it still seems like a relatively small effect for a BIG policy change. This option necessarily assumes no downside to the random loss of experienced teachers. Of course, the argument is that more of those classes now have a good teacher in front of them. But again, doing this here and now with the type of information available means relying not even on the “best” of teacher effectiveness models, but relying on expedited, particularly sloppy, not thoroughly vetted models. I would have continued concerns even with richer models, like those explored in the recent Gates/Kane report, which still prove insufficient.

Perhaps most importantly, how does this new policy affect the future teacher workforce in Newark – the desirability for up-and-coming teachers to pursue a teaching career in Newark, where their career might be cut off at any point, by random statistical error? And how does that tradeoff balance with a net difference of about half a student per classroom?

What about other costs?

Petrilli and Roza, among others, ignore entirely any potential downside to the teacher workforce – those who might choose to enter that workforce if school districts or states al-of-the-sudden decide to rely heavily on error prone and biased measures of teacher effectiveness to implement layoff policies. This downside might be counterbalanced by increased salaries, on average and especially on the front end. That is, to achieve equal incoming teacher quality over time, given the new uncertainty, might require higher front end salaries. This cost is ignored entirely (or simply assumed to come from somewhere else, like cutting benefits… simply negating step increments, or supplements for master’s degrees, each of which have other unmeasured consequences).

I have assumed above that districts would rely heavily on available student testing data, creating error-prone, largely random layoffs, while ignoring the cost of applying the evaluation system to achieve the layoffs. Arguably, even contracting an outside statistician to run the models and identifying the teachers to be laid off would cost another $50,000 to $75,000, leading to reduction of at least one more teacher position under the “quality based” layoff model.

And then there are the legal costs of fighting the due process claims that the dismissals were arbitrary and the potential legal claims over racially disparate firings. Forthcoming law review article to be posted soon.

Alternatively, developing a more rigorous teacher evaluation system that might more legitimately guide layoff policies requires significant up-front costs, ignored entirely in the current overly simplistic, misguided rhetoric.

How can we implement quality based layoffs when we’re supposed to be laying off teachers NOT teaching math and reading in elementary grades?

Here’s another issue that Petrilli, Roza and others seem to totally ignore. They argue that we must a) dismiss teachers based on quality and b) must make sure we don’t compromise class sizes in core instructional areas, like reading and math in the elementary grades.

Let’s ponder this for a moment. The only teachers to whom we can readily assign (albeit deeply flawed) effectiveness ratings are those teaching math and reading between grades 3 and 8. So, the only teachers who we could conceivably layoff based on preferred “reformy” quality metrics are teachers who are directly responsible for teaching math and reading between grades 3 and 8.

That is, in order to implement quality based layoffs, as reformers suggest, we must be laying off math and reading teachers between grades 3 and 8, except that we are supposed to be laying off other teachers, not those teachers. WOW… didn’t think that one through very well… did they?

Am I saying seniority layoffs are great?

No. Clearly seniority layoffs are imperfect and arguably there is no perfect answer to layoff policies. Layoffs suck and sometimes that sucky option has to be implemented. Sometimes that that sucky option has to be implemented with a blunt and convenient instrument and one that is easily defined, such as years of service. It is foolish to argue that teaching is the only profession where those who’ve been around for a while – those who’ve done their time – have greater protection when the axe comes down. Might I suggest that paying one’s dues even plays a significant role in many private sector jobs? Really? And it is equally foolish to argue that every other profession EXCEPT TEACHING necessarily makes precise quality decisions regarding employees when that axe comes down.

The tradeoff being made in this case is a tradeoff NOT between “keeping quality teachers” versus “keeping old, dead wood” as Petrilli, Roza and others would argue, but rather the tradeoff between laying off teachers on the unfortunately crude basis of seniority only, versus laying off teachers on a marginally-better-than-random, roll-of-the-dice basis. I would argue the latter may actually be more problematic for the future quality of the teaching workforce! Yes, pundits seem to think that destabilizing the teaching workforce can only make it better. How could it possibly get worse, they argue? Substantially increasing the uncertainty of career earnings for teachers can certainly make it worse.

Bad Teachers Hurt Kids, but Salary Cuts Have no Down Side?

The assumption constantly thrown around in these policy briefs is that putting a bad teacher in front of the kids is the worst possible thing you could do. We have to fire those teachers. They are bad for kids. They hurt kids.

But, the same pundits argue that we should cut pay for the teachers in any number of ways (including paying for benefits) and subject teachers to layoff policies that are little more than random. Since so many teachers are bad teachers – and simply bad people – these policies are, of course, not offensive. Right? Kids good. Teachers bad. Treat kids well. Take it out on teachers. No harm to kids. Easy!

I’m having a hard time swallowing that. That’s just not a reasonable way to treat a workforce (if you want a good workforce), no less a reasonable way to treat a workforce charged with educating children. In fact, it’s bad for the kids, and just plain ignorant to assert that one can treat the teachers badly, lower their pay, morale and ultimately the quality of the teacher workforce and expect there to be no downside for the kids.

Petrilli and Roza make the assumption that there is big savings to be found from cutting teacher salaries directly and also indirectly by passing along benefits costs to teachers. That’s a salary cut! Or at least a cut to the total compensation package and it’s a package deal! This argument seems to be coupled with an assumption that there is absolutely no loss of benefit or effectiveness from pursuing this cost-cutting approach (because we’ll be firing all of the sucky teachers anyway). That is, teacher quality will remain constant even if teacher salaries are cut substantially. A substantial body of research questions that assumption:

Murnane and Olson (1989) find that salaries affect the decision to enter teaching and the duration of the teaching career;

Figlio (1997, 2002) and Ferguson (1991) find that higher salaries are associated with better qualified teachers;

Figlio and Reuben (2001) “find that tax limits systematically reduce the average quality of education majors, as well as new public school teachers in states that have passed these limits;”

Ondrich, Pas and Yinger (2008) “find that teachers in districts with higher salaries relative to non-teaching salaries in the same county are less likely to leave teaching and that a teacher is less likely to change districts when he or she teaches in a district near the top of the teacher salary distribution in that county.”

To mention a few.

That is, in the aggregate, higher salaries (and better working conditions) can attract a stronger teacher workforce, and at a local level, having more competitive teaching salaries compared either to non-teaching jobs in the same labor market or compared to teaching jobs in other districts in the same labor market can help attract and especially retain teachers.

Allegretto, Corcoran and Mishel, among others, have shown that teacher wages have lagged over time – fallen behind non-teaching professions. AND, they have shown that the benefits differences are smaller than many others argue and certainly do not make up the difference in the wage deficit over time. I have shown previously on my blog that teacher wages in New Jersey have similarly lagged behind!

So, let’s assume we believe that teacher quality necessarily trumps reduced class size, for the same dollar spent. Sadly, this has been a really difficult trade-off to untangle in empirical research and while reformers boldly assume this, the evidence is not clear. But let’s accept that assumption. But let’s also accept the evidence that overall wages and local wage advantages lead to a stronger teacher workforce.

If that’s the case, then the appropriate decision to make at the district level would be to lay off teachers and marginally increase class sizes, while making sure to keep salaries competitive. After all, the aggregate data seem to suggest that over the past few decades we’ve increased the number of personnel more than we’ve increased the salaries of those personnel. That is, cut numbers of staff before cutting or freezing salaries. In fact, one might even choose to cut more staff and pay even higher salaries to gain competitive advantage in tough economic times. Some have suggested as much. I’m not sold on that either, especially when we start talking about increasing class sizes to 30, 35 or even 50. Note that class size may also affect the competitive wage that must be paid to a teacher in order to recruit and retain teachers of constant quality. Nonetheless, it is important to understand the role of teacher compensation in ensuring the overall quality of the teacher workforce and it is absurd to assume no negative consequences of slashing teacher pay across-the-board.

Take home point!

In summary, we should be providing thoughtful decision frameworks for local public school administrators to make cost-effective decisions regarding resource allocation rather than spewing laundry lists of reformy strategies for which no thoughtful cost-effectiveness analysis has ever been conducted.

Further, now is not the time to act in panic and haste to adopt these unfounded strategies without appropriate consideration of the up-front costs of making truly effective reforms.

A few references

Richard J. Murnane and Randall Olsen (1989) The effects of salaries and opportunity costs on length of state in teaching. Evidence from Michigan. Review of Economics and Statistics 71 (2) 347-352

David N. Figlio (1997) Teacher Salaries and Teacher Quality. Economics Letters 55 267-271.

David N. Figlio (2002) Can Public Schools Buy Better-Qualified Teachers?” Industrial and Labor Relations Review 55, 686-699.

Figlio (1997, 2002) and Ferguson (1991) find that higher salaries are associated with better qualified teachers

Ronald Ferguson (1991) Paying for Public Education: New Evidence on How and Why Money Matters. Harvard Journal on Legislation. 28 (2) 465-498.

Figlio, D.N., Reuben, K. (2001) Tax limits and the qualifications of new teachers Journal of Public Economics 80 (1) 49-71

Ondrich, J., Pas, E., Yinger, J. (2008) The Determinants of Teacher Attrition in Upstate New York. Public Finance Review 36 (1) 112-144

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this: