Category: Center for American Progress Reports

Inexcusable Inequalities! This is NOT the post funding equity era!

I’ve heard it over and over again from reformy pundits. Funding equity? Been there done that. It doesn’t make a damn bit of difference. It’s all about teacher quality! (which of course has little or nothing to do with funding equity?).  The bottom line is that equitable and adequate financing of schools is a NECESSARY UNDERLYING CONDITION FOR EVERYTHING ELSE!

I’m sick of hearing, from pundits who’ve never run a number themselves and have merely passed along copies of the meaningless NCES Table showing national average spending in high poverty districts slightly greater than that for lower poverty ones. 

I’m sick of the various iterations of the “we’ve tripled spending and gotten nothing for it” argument and accompanying bogus graphs.  And further, the implication put forward by pundits that these graphs and table taken together mean that we’ve put our effort into the finance side for kids in low-income schools, but it’s their damn lazy overpaid teachers who just aren’t cutting it.

I’m intrigued by those pundits who would point out that perhaps outcomes of low-income children have improved over the past few decades and that the improvement is entirely attributable to increased accountability measures (when the same pundits have argued previously that the massive increases in funding led to no improvement. Perhaps there has been improvement, and perhaps there has been some increase in funding on average… and perhaps that’s the connection? More insights on achievement gap closure and shifting resources here!).

I’m also sick of those who would so absurdly argue that districts serving low-income and minority children really have more than enough money to deliver good programs, but they’ve squandered it all on useless stuff like cheerleading and ceramics.

Anyway, the goal of this post is  to point out some of the inexcusable inequalities that persist in K-12 education, inequalities that have real consequences for kids. Let’s take a look, for example, at two states that have persistently large achievement gaps between low-income and non-low income students – Illinois and Connecticut. These two states have somewhat different patterns of overall funding disparity, but suffice it to say, both states have their winners and losers, and the differences between them are ugly and unacceptable.

Let’s start with Connecticut. Below is a graph of Connecticut school district “need and cost adjusted current spending per pupil” and standardized test outcomes on the Connecticut Mastery Test (CMT). Expenditures are adjusted for differences on labor market competitive wages and for shares of children qualifying for free or reduced price lunch and for children with limited English language proficiency (based on estimates reported here). I’ve used essentially the same methods I discussed in this previous post.

What we see here is that resources – after adjustment for needs and costs – vary widely. Heck, they vary quite substantially even without these adjustments! What we also see is that we’ve got some really high flyers, like Weston, New Canaan and Westport, and we’ve got some that, well, are a bit behind in both equitable resources and outcomes (Bridgeport and New Britain in particular). To be blunt, THIS MATTERS!  Yeah.. okay, reformy pundits are saying, but they really have enough anyway. Why put anything else into those rat-holes.

Let’s break it down a bit further. Here are the characteristics of a few of the most advantaged and most disadvantaged districts in the above figure.

But of course, all we need to do is reshuffle the deck chairs  in Bridgeport and New Britain – fire their bottom 5% – heck let’s go for 20% teachers – pay the new ones based on test scores… and all will be fixed! Those deficits in average salaries might be a bit problematic. And even the nominal (no adjustments) spending figures fall well short of their advantaged neighbors. But bring on those reformy fixes, and throw in some funding cuts while you’re at it!

I’m sure… absolutely sure that the only reason those salaries are low is because they’ve wasted too much money on administrators and reducing class size… which we all know doesn’t accomplish anything???? But wait, here are the elementary class sizes?

Well, there goes that ridiculous reformy assumption. Class sizes are actually larger in these higher need districts! and Salaries lower. Damn cheerleading costs! Killing us! Perhaps it’s even going into  junk like band and art which are obviously a waste of time and money on these kids!

Well, here are the staffing structures of the schools, with staffing positions reported per 100 pupils.

Hmmm… disadvantaged districts have far fewer total positions per child, and if we click and blow up the graph, we can see some striking discrepancies! Those high need districts have far more special education and bilingual education teachers (squeezing out other options, from their smaller pot!). Those high need districts have only about half the access to teachers in physical education assignments or art, much less access to Band (little or none to Orchestra), and significantly less access to math teachers!

But, okay… this Connecticut thing is a freakin’ anomaly, right?  These kind of disparities – savage inequalities – are surely a thing of the past. This is, after all, THE POST-FUNDING EQUITY ERA? Been there and done that!

Let’s do the same walk through for a few Illinois districts. First, here are the graphs of need and cost adjusted (based on a cost model used in my previous post and related working paper) operating expenditures and outcomes –

For unified K-12 districts


For High School districts

Here are the basic stats on these districts

In this case, imagine trying to recruit and retain teachers of comparable quality in JS Morton to those in New Trier at $20k less on average, or in Aurora East compared to Barrington, at nearly $20k less. Ahh…you say… Baker… you’re making way too much of the funding issue. First, we know their wasting it all on small class size and cheerleading. Second, Baker… you’re missing the point that if we fire the bad teachers and pay the good teachers based on student test scores, those New Trier teachers will be banging down the door to get into J S Morton! That’s real reform dammit! And we know it works (even though we don’t have an ounce of freakin’ evidence to that effect!).

Clearly, if schools in Aurora East and JS Morton are slated for closure under NCLB (I’ve not checked this actually), it’s not because of poverty. It’s not for lack of resources… Clearly it’s their lazy, overpaid teachers who refuse to pull all-nighters with their kids to beat those odds????? To get those kids into calculus and trig classes presently filled with empty seats (and their own overpaid under-worked teachers!)

So, here’s what the staffing ratios look like.

First, those advantaged districts just have a lot more teacher assignments (position assignments) than the disadvantaged ones. And they especially have far more assignments in advanced math, advanced science, Library/Media, Art and music. There’s not a whole lot of squandering on extras going on in JS Morton and Aurora East. Like CT though, the disadvantaged districts do have bilingual education and special education teachers!  The staffing disparities are baffling – Savage in fact!

In fact, I must be making this stuff up right. After all, THIS IS THE POST-FUNDING DISPARITY ERA? This kind of stuff is just pulled from the chapters of an old Kozol book!  Teachers matter. Not funding. We all know that (except perhaps the various researchers who’ve actually explored the relationship between school funding reforms and student outcomes, only to find that it does matter).

Clearly, this matters. These funding disparities are substantial. And while these examples are selected from the extremes of the distributions, these districts have plenty of company at the extremes, and these districts fall along a clearly patterned continuum. And, with enough data and enough space, I could keep going and going here. CT and IL are not unique – though IL is clearly among the worst in the nation. New York anyone?

Utica is quite possibly one of the most financially screwed local public school districts in the nation (Poughkeepsie isn’t far behind)!

Arguably, there are entire states – like Tennessee and Arizona that are approaching (if they’ve not already surpassed) the conditions of districts like Utica, JS Morton, Bridgeport or New Britain.

Until we take these disparities seriously and stop counting on miracles and superman to give us a free ride, we’re not likely to make real progress on the “Scarsdale-Harlem” achievement gap.

Treating teachers like crap, cutting state funding, basing teacher salaries on student test scores will do nothing to correct these disparities, and will likely only make them worse. Nor can we expect to close the gap by simply replacing the current underfunded schools with comparably underfunded schools under new management (or simply paying parents of kids in these districts a discount rate to just go somewhere else, and never follow up on the kids). This reformy goo is a dangerous distraction from the real issues!

THIS IS NOT THE POST FUNDING EQUITY ERA.

FUNDING MATTERS.

GOOD EDUCATION IS EXPENSIVE & WORTH IT!

EQUITABLE AND ADEQUATE FUNDING IS A NECESSARY UNDERLYING CONDITION FOR THE FUTURE SUCCESS OF AMERICAN PUBLIC EDUCATION.


Resource Deprivation in High Need Districts? (& CAP’s goofy ROI)

This post provides a follow-up on two seemingly unrelated topics, both of which can be traced back to the Center for American Progress.

First, there was that wonderful little Return on Investment indicator series that CAP did a while back.

Second, there’s the frequent, anecdotal argument that creeps into CAP/Ed Trust and AEI conversations that high need districts all have enough resources anyway and just have to stop wasting them on things like Cheerleading and Ceramics.

In this post, I provide an abbreviated version of some of the findings one of my recent conference papers.

The goal of the research study was to first identify those districts which fell into various regions or quadrants, applying a framework similar to that used by CAP in their ROI and second, explore the differences in personnel allocation in each group of districts looking for insights into what makes them tick (or not). It’s not a very good framework to begin with, but at least provides a common starting point:

The idea is that districts may fall into four groups. Some are high spending high performers and some are low spending low performers. Others are high spending low performers and still others are low spending high performers. What would be interesting from a policy perspective is whether we really could identify those in Q1 above and those in Q3 above and determine what makes them tick (Q1), or not tick (Q3).

As I discussed in a previous post, CAP took a particularly egregiously flawed approach to correcting/adjusting for various factors and laying out districts across these four quadrants. Here’s a snapshot of their Illinois findings:

The CAP IL snapshot shows plenty of districts in those green and red quadrants. Of course, the CAP snapshot a) fails to full correct for poverty related costs or ELL related costs and b) doesn’t correct at all for economies of scale or population density. If one were to believe the CAP findings, one would assume that there are similar proportions of districts that are in each group – both the expected groups (upper right and lower left) and the less likely groups (upper left and lower right). Of course, CAP also blew it in their interpretation of what’s going on in the lower left. They seemed to chastise these low spending low performing districts for their low performance, rather than acknowledge that these are actually the districts that have been screwed on funding, and are producing exactly what is expected of them in terms of outcomes.

Of course, if one more fully corrects for differences in costs across IL school districts, the actual distribution by quadrant comes out more like this (see conference paper for details on cost adjustment model):

The reality is that there aren’t a whole lot of districts – at least in the Chicago metro area that fall in the upper left and lower right quadrants. In fact, districts are largely where they are expected to be – Some have plenty of resources and do quite well, and others have limited resources and are doing poorly. Now, there is plenty of variance in the lower left and upper right which could be explored for interesting patterns.

Note that Illinois (along with PA and NY) is among the most regressively funded and racially disparately funded systems in the country!

How do resource constraints relate to curricular offerings?

Much of the  conversation of the past few days/weeks by pundits on twitter and in blogs has been on the question of what’s good for the “rich” and what’s good for the “poor.” Let me reframe that issue in this post in terms of what kids have access to in districts in the upper right quadrant of the above figure versus what kids have access to in the lower left quadrant.  Of course, the anecdotal assumption laid out above is that there are actually a whole bunch of districts in the lower right that have elaborate cheerleading and ceramics programs. Say it ain’t so! Okay… it ain’t!

What is so is that students attending districts in the lower left hand quadrant tend to have much less access to advanced curricular opportunities and boutique electives courses than children attending districts in the upper right hand quadrant. Here are a few figures, based on individual staffing assignment data:


Children attending districts in the upper right hand quadrant are nearly 3 times as likely to have access to a teacher assigned primarily to advanced math courses, nearly twice as likely to have access to a teacher primarily assigned to advanced literature or advanced science, and significant more likely to have access to a teacher assigned primarily to advanced social sciences or even seemingly more basic offerings like Algebra and Geometry. Moving deeper into the extremes of the upper right and lower left quadrants magnifies these disparities.  Further, while these distributions are expressed as a percent of total staffing, high spending high outcome districts tend to have significantly more staff per pupil.

Students in the lower left hand quadrant do have more of some stuff. They have a greater density (as a share of total staffing, but NOT on a per pupil basis) of elementary classroom teachers, and teachers in bilingual, alternative and at risk education. They also seem to have marginally more school site administrators. They have only comparable shares of staff allocated to basic level courses.

Implications?

Analyses in the full paper provided little evidence in Illinois or Missouri that high need and low performing districts were squandering their resources on things like cheerleading or ceramics, or, for that matter that there were large numbers of high need low performing districts that really had enough resources to begin with but weren’t using them productively. The classic emergent profile of a high need low performing district in Missouri and Illinois was of a district with highly constrained resources after adjustment for costs, and a district that had largely forgone assigning teachers to advanced content areas and elective courses for which they perhaps expected few students to enroll. Lack of a rich curriculum in high need settings is a significant policy concern and is a concern that cannot likely be remedied by reshuffling deck chairs.  These districts in fact need more total resources than high spending high outcome districts because they must be able to offer both the basic course work to prepare students to gain access to higher level courses, and to offer the higher level courses. Under present circumstances in many states, those resources just aren’t there, and it is very counterproductive to pretend either that they are or that it’s the districts’ fault they aren’t!

More expensive than what? A quick comment on CAP’s CSR report

The Center for American Progress today release a report on class size reduction authored by Matthew Chingos, who has conducted a handful of recent interesting studies on the topic.

http://www.americanprogress.org/issues/2011/04/pdf/class_size.pdf

This report reads more or less like a manifesto against class size reduction as a strategy for improving school quality and student outcomes. I’ll admit that I’m also probably not the biggest advocate for class size reduction as a single, core strategy for education reform, and that I do favor some balanced emphasis on teacher quality issues. I’m also not the naysayer that I once was regarding class size reduction and its relative costs.  There still exists too little decisive information regarding the cost-benefit tradeoffs between the two – teacher quantity and teacher quality.

I only had a chance to view this report briefly, and one specific section caught my eye – the section titled: CSR, The Most Expensive School Reform.

I found this interesting, because it included a bunch of back of the napkin estimates of the potential costs of CSR (based on reasonable assumptions), BUT PROVIDED NOT ONE SINGLE COMPARISON OF THE COST AND BENEFITS OF CSR TO ANY OTHER ALTERNATIVE.

You see – You can’t say something is the most expensive without actually comparing it to, uh, something else. That’s how cost comparisons work. Cost benefit analysis works this way too. You compare the costs of option A, and outcomes achieved under option A, to the costs of option B, and outcomes achieved under option B.

Implicit in this section of the report is that reducing class size for any given improvement in student outcomes is necessarily more expensive than improving student outcomes by the same amount by improving teacher quality.  In fact, explicit in the title of this section of the report is that pretty much any alternative that might get the same outcome is cheaper than CSR. That’s one freakin’ amazing stretch!

Here are a few quotes provided by Matt Chingos on this point:

A school that pays teachers $50,000 per year (roughly the national average) would save $833 per student in teacher salary costs alone by increasing class size from 15 to 20.30 The true savings, including facilities costs and teacher benefits, would be significantly larger. These resources could be used for other purposes. If all of the savings were used to raise teacher salaries, for example, the average teacher salary in this example would increase by $17,000 to $67,000.

And:

The emerging consensus that teacher effectiveness is the single most important in-school determinant of student achievement suggests that teacher recruitment, retention, and compensation policies ought to rank high on the list.

Chingos goes on to address the various teacher effect and effectiveness based layoff simulations by authors including Eric Hanushek and how those simulations project larger gains than would be achieved by class size reduction. Chingos does acknowledge in the next paragraph that:

Teachers would need to be paid more to compensate them for the loss of job security. Providing bonuses to teachers in high-need subjects and schools would also consume resources. If these policies are more cost-effective than reducing class size, then increasing class size in order to pursue them would increase student achievement.

However, it would seem by the title and the rest of the content of this section that Chingos has jumped to a conclusion on this point. No actual cost comparison is made between improving student outcomes by improving teacher effectiveness versus improving student outcomes by class size reduction.

The relevant research question based on the hypothetical here is:

…on a given labor market with a given supply of teacher quantities and qualities, does the teacher that will teach for a salary of $67,000 with a class of 20 children get a better result than the teacher that will teach for a salary of $50,000 with a class of 15?

I’m not sure we know the answer to that, in part because the teacher labor market research also suggests that while there is sensitivity of teacher labor markets to salaries, it may take quite substantial salary increases to achieve comparable gains to class size reduction. Further, given class size and total student load as a working condition, the same teacher might teach a class of 15 for marginally lower salary than to teach a class of 20 (which could be the difference between a total load, at 6 sections per day, of 90 vs. 120 students, which is a pretty big difference).

I’ve been waiting for years for good answers to this tradeoff, and hoping for data that will provide better opportunities to address this question. Unfortunately, the wait continues.

School Funding Equity Smokescreens: A note to the Equity Commission

In this blog post, I summarize a number of issues I’ve addressed in the past. In my previous post, I discussed general reformy myths about school spending. In this post, I address smokescreens commonly occurring in DC beltway rhetoric about school funding equity and adequacy. School funding is largely a state and local issue, where even that “local” component is governed under state policies. So I guess that makes it a state issue, really. Occasionally, the federal government will dabble in the debate over how or whether to intervene more extensively in state and local public school finance.  Now is one of those times where the federal government is again at least paying lip services to the question of equity – with some implication that they may even be talking about school funding equity. The federal government has created an equity commission!

One of my fears is that this current discussion of funding equity will be typical of recent beltway discussions of school funding, and be trapped in the constant fog of School Funding Smokescreens and insulated entirely from more legitimate representations and analyses of the critical issues that should be addressed.

So, for you – the equity commission – here’s a quick run down on School Funding Smokescreens:

1. On average, nationally, we now put more funding into higher poverty school districts than lower poverty ones (to no avail)

This argument seems to be popping up more and more of late, and often with the table below attached. This table is from the National Center for Education Statistics and shows the average current operating expenditure per pupil of school districts nationally over time. The table would appear to show that in 1994-95, low poverty school districts had between $300 and $400 less in per pupil spending than higher poverty ones. By 2006-07, the highest poverty quintile of school districts had about $100 per pupil more than the lowest poverty quintile. That’s it. We’re done. Equity problems fixed. No more savage inequalities. And after all of this fixing of school finance equity, we really got nothing for it. Achievement gaps are still unacceptably large and NAEP scores stagnant? Right? All of this after dumping a whole extra $100 per pupil into high poverty districts. I guess we should be rethinking this crazy strategy of systematically pouring so much into high poverty districts.

Table 1

NCES Oversimplification of Funding Differences by Poverty


Well, to begin with, a $100 difference really wouldn’t be that much anyway, given that the costs of actually meeting the needs of children from economically disadvantaged backgrounds are much greater than this. Setting that (really important) question aside, this table provides a less than compelling argument that we as a nation have accomplished improved funding equity for kids in high poverty districts.

Here’s the underlying scatter of school districts that lead to the neatly packed aggregations above. In the graph below, districts are plotted by current expenditures per pupil with respect to census poverty rates, using 2007-08 data. Clearly there is substantial variation in current spending. In fact, the underlying relationship isn’t even a relationship at all. It’s all over the place. And yes, if you fit a trend line – if you take out a huge magnifying glass – you can see that the trendline is ever so slightly higher in the higher poverty schools than in the lower poverty ones (perhaps about $100?). It’s not systematic. It’s not statistically significant. It’s pretty darn meaningless.

Figure 1

Pattern of school districts underlying Table 1


In our recent report Is school funding fair? we conducted a far more rigorous analysis of state and local revenue per pupil with respect to poverty, for each state. What we showed was that there exists huge variation across states both in the overall level of resources available to local public school districts and in the differences in state and local revenue in higher and lower poverty districts.  In that report, we showed that 9 states have statistically significantly lower state and local revenue per pupil in higher poverty districts (after controlling for economies of scale and competitive wage variation). Overall, half of states had lower funding per pupil in higher poverty districts (with many of those approximately the same).

Among the worst states were New York, Illinois and Pennsylvania. Let’s pull Illinois forward in Figure 1 – and also look at state and local revenues (excluding federal support, to focus on state policies) in place of current expenditures.

Figure 2

State and local revenues with respect to poverty, with Illinois highlighted


Now, when we exclude federal revenues the overall line tips slightly downward. The federal effect is slight, but there. More strikingly, when we pull Illinois forward in the picture, we see that funding by poverty across Illinois districts is highly regressive, and is systematic and statistically significant. Funding inequities across Illinois districts are far from being resolved. AND ILLINOIS IS NOT ALONE. I could go on and on with this.

2. The remaining (Because of #1), most egregious disparities in funding and teacher quality occur across schools within districts (because of politically motivated and corrupt local administrators) and these disparities are what cause the persistent racial achievement gaps (the reason those gaps haven’t improved since we’ve fixed between district inequity)

To many, this argument seems absurd (and is) on its face. Who really says that? Does anyone? Am I just makin’ this stuff up? No. And in fact, because this argument has become so pervasive of late, I even had to take the time to write a fairly extensive research article on the topic. See: http://epaa.asu.edu/ojs/article/view/718

I have written about this topic on my blog on several occasions and much of my writing on this topic can be found by reading my critiques of reports from the Center for American Progress and from the Education Trust. Here are some choice quotes where CAP and Ed Trust frame this argument – or blow this smoke!

Center for American Progress

State funding formulas tend to exert an equalizing effect on per pupil revenues between districts, on average, and not by accident. These formulas were sculpted by two generations of litigation and legislation seeking equitable or adequate funding for property-poor school districts.

Scandalous inequity in the distribution of resources within school districts has plagued U.S. education for more than a hundred years.

empirical literature documenting the extent of within-district inequity is astonishingly thin. [my reply: well, not if you actually read the research on the topic]

Center for American Progress

The outcome of such practices is predictable: A further widening of the dangerous achievement gap that has become endemic in American schools today.

Education Trust

Many states have made progress in closing the funding gaps between affluent school districts and those serving the highest concentrations of low-income children. But a hidden funding gap between high-poverty and low-poverty schools persists between schools within the same district.

These gaps occur partly because teachers in wealthier schools tend to earn more than their peers in high-poverty schools and because of pressure to “equalize” other resources across schools.

All of these claims that within district inequities are the major source of persistent inequity and that our failure to close within district funding and teacher quality gaps (having already fixed between district ones) are the reason for persistent black-white and poor-non-poor achievement gaps might be reasonable if poor children and non-poor children and black children and white children actually lived in the same school districts. BUT, IN GENERAL,* THEY DO NOT! As a result this argument is patently absurd, ridiculous, irresponsible and ignorant. It’s one massive distraction. A smokescreen of monumental proportion!

Here’s a quick visual of the reality that any informed analyst (or anyone who simply lives in the real world) understands. Below are two maps of the Chicago metropolitan area. On the left is a map which shows school districts and the level of state and local revenue per pupil in each of those districts. We know from above that Illinois maintains a very regressive state school finance formula. That is, higher poverty districts have less funding than lower poverty ones. Note that the diagonal shading indicates the location of districts that have majority minority (black and Hispanic) enrollment. As it turns out, most of those districts are in the orange – lower funding levels.

Now, CAP and Ed Trust would have you believe otherwise to begin with (that poor minority districts already have enough money), but would then go further to say that the real problem is that these Illinois districts are putting money into their white, rich schools at the expense of their poor black and Hispanic ones. How is that even possible?

Okay, so let’s look at the right hand panel, in which I have indicated the locations of individual schools, with majority black schools in red and majority Hispanic schools in purple. Majority white schools are in white. NOTE THAT THE WHITE DOTS TEND TO BE DISTRICTS ENTIRELY SEPARATE FROM THE PURPLE OR RED ONES. AND ONLY CHICAGO PUBLIC SCHOOLS HAS MUCH OF A MIX OF PURPLE AND RED. Only a handful of districts have both white and majority minority schools. Also, only a handful of districts have both low(er) poverty and high poverty schools. Districts are highly segregated.

Figure 3

State and Local Revenue and the Location of Majority Minority Schools in Illinois


FEW IF ANY SCHOOL DISTRICTS IN THIS MAP HAVE THE OPPORTUNITY TO REDISTRIBUTE RESOURCES ACROSS THEIR “RICH” AND “POOR” OR “BLACK” AND “WHITE” SCHOOLS – BECAUSE THEY DON’T HAVE BOTH!!!!!  Yes, Chicago Public Schools and a few other districts can re-allocate between poor black and poor Hispanic schools. But such re-allocation accomplishes little toward improving educational equity in the Chicago metro area or State of Illinois.

Now… to those at Ed Trust and CAP – if you really don’t mean this, I dare you to actually say it. Say that between district differences in demographics and funding are THE BIG ISSUE. At least as big if not much bigger than within district differences. Say it. Acknowledge it. I challenge you. Release another hastily crafted report and press release – but this time – having conclusions that are at least reasonably grounded in reality.  The data are unambiguous in this regard. Yes, within district disparities exist and it is important to address them. I will certainly admit that, and I’ve never said otherwise.  But solving within district resource variation alone will accomplish very little.

*Clarification – In states with county-wide districts, and large diverse populations, like Florida, one is more likely to see between school  within district segregation to be a greater problem.

3. High need, poor urban districts (in addition to misallocating all of their resources to the schools serving rich white kids in their district???) are simply wasting massive sums of money on things like cheer leading and ceramics.

This is another absurd and empirically unfounded argument. Again, you ask, is anyone really saying that high need, low performing school districts are actually wasting money on cheerleading and ceramics that could easily be translated into sufficient resources for improving reading and math performance (can we really fire the cheer leading coach and hire 6 more math specialists)? Surely no-one is advancing an argument – SMOKESCREEN – that utterly absurd. But again, these quotes can be found all over the Beltway talk-circuit regarding the best fixes for school funding inequities and inefficiencies (and nifty was to stretch that school dollar).

Here’s the advertisement headline from a recent beltway discussion at the Urban Institute:

Urban Institute Event Headline (based on content from Marguerite Roza)

Imagine a high school that spends $328 per student for math courses and $1,348 per cheerleader for cheerleading activities. Or a school where the average per-student cost of offering ceramics was $1,608; cosmetology, $1,997; and such core subjects as science, $739.

I’ve only recently begun exploring more deeply the resource differences across school districts that fall into different performance and efficiency categories. I’ve been specifically looking at Illinois and Missouri school districts, and estimating statistical models to determine which districts are:

a) resource constrained and low performing (low-low)

b) resource constrained and high performing (low-high)

c) resource rich and high performing (high-high)

d) resource rich and low performing (high-low)

These categories are based on thoroughly cost adjusted analysis. As such, a district identified as having low or constrained “resources” may actually spend more per pupil in nominal dollars than a district identified as having high resource levels. The resource levels are adjusted for various cost pressures including differences in student needs. I should be posting the forthcoming paper on my research page some time in the next month. But here’s a preview.

In both states, most districts fall into categories a) and c), where you would expect. There’s somewhat more “scatter” in Missouri, either because Missouri has some better funded high need districts (less regressive than Illinois) or because my statistical model just isn’t working quite right. I picked these neighboring states because Missouri is less regressive than Illinois and because I had similar data on both. So, the big question here is – if I compare the dominant categories of resource constrained low performing schools to resource rich high performing ones what do we actually see in the organization of their staffing and course delivery?

In Missouri, I tabulate each individual course to which teachers are assigned. In Illinois my tabulation is by the main assignment of each teacher. To begin with, in both states, the high spending high performing schools have more course offerings per pupil and more teachers per pupil (and smaller class sizes). These differences are far greater under the more regressive Illinois policies.

Here are a few fun visuals of what I’m finding so far, expressed in “shares of staff” allocation and relating staffing allocations in low-low districts to those of high-high districts.

The first two graphs compare the main assignments of teachers in high resource high performing Illinois schools (high school assignments only) to those in low resource low performing ones. The diagonal line represents “comparable” allocation to high resource high performing schools. Assignments falling below the line represent “deficits” (relative) in low resource low performing schools.

Across all assignment areas, Figure 4 shows that kids in low resource low performing schools tend to have reduced access to physical education, biology, chemistry and foreign language. Sadly, no indicator for ceramics in these data.

Figure 4

Allocation of Main Teaching Assignments in Illinois Districts


Focusing on less frequent assignment areas – lower budget share & staff allocation areas – Figure 5 shows that in Illinois, kids in low resource low performing schools tend to have reduced access to advanced math and science courses and drivers education, but have greater access to basic courses. That is, these districts are already channeling their resources to the basic, at the detriment of potentially important advanced coursework in math and science, and even basic coursework in biology and chemistry.

Figure 5

Allocation of Main Teaching Assignments in Illinois Districts (less frequent assignments)


Missouri – despite having somewhat higher relative resource levels in higher poverty settings (than Illinois, but still regressive), shows very similar patterns. Figure 6 shows reduced access to physical education for kids in low resource low outcome schools and elevated access to “general” math and language arts courses.

Figure 6

Allocation of Assigned Courses in Missouri Districts


Kids in low resource low outcome schools have reduced access to advanced math courses including calculus and trigonometry, and reduced access to chemistry. They have higher shares of teachers in special education, basic life skills, earth and physical (basic/introductory) science and in JROTC. Again, significant reallocation to “basics” is already occurring and within significant resource constraints.

Figure 7

Allocation of Assigned Courses in Missouri Districts (less frequent courses)


Also, LET IT BE KNOWN THAT HIGH SPENDING HIGH PERFORMING SCHOOLS IN MISSOURI HAVE TWICE AS MANY CERAMICS COURSE OFFERINGS PER PUPIL AS LOW SPENDIGN LOW PERFORMING MISSOURI SCHOOLS!!!!!

4. None of this school funding equity – between district stuff – matters anyway!

Rigorous peer reviewed studies do show that state school finance reforms matter. Shifting the level of funding can improve the quality of teacher workforce and ultimately the level of student outcomes and shifting the distribution of resources can shift the distribution of outcomes.

We conclude that there is arbitrariness in how research in this area appears to have shaped the perceptions and discourse of policymakers and the public. Methodological complexities and design problems plague finance impact studies. Advocacy research that has received considerable attention in the press and elsewhere has taken shortcuts toward desired conclusions, and this is troubling. As demonstrated by our own second look at the states discussed in Hanushek and Lindseth’s book, the methods used for such relatively superficial analyses are easily manipulable and do not necessarily lead to the book’s conclusions. Higher quality research, in contrast, shows that states that implemented significant reforms to the level and/or distribution of funding tend to have significant gains in student outcomes. Moreover, we stress the importance of the specific nature of any given reform: positive outcomes are likely to arise only if the reform is both significant and sustained. Court orders alone do not ensure improved outcomes, nor do short-term responses.

If it doesn’t work, don’t do it! CAP’s ROI

The Center for American Progress released its new Return on Investment (ROI) Index for K-12 public school districts of greater than 250 students this week. I should note in advance that I had the opportunity to provide advice on this project early on, and occasionally thereafter and I do believe that at least some involved had and still have the best intentions in coming up with a useful way to represent the information at hand. I’ll get back to the validity of the information at hand in a moment.

First, I need to point out that the policy implications and proposals, or even general findings presented in the report cannot be supported by the analysis (however well or poorly done). The suggestion that billions of dollars might be saved nationally (little more than a back-of-the-napkin extrapolation) at no loss to performance outcomes, based on the models estimated is a huge, unwarranted stretch and quite simply arrogant, ignorant and irresponsible.

The method used provides no reasonable basis for the claim that all low “rate of return” districts could simply replicate the behaviors of high “rate of return” districts and achieve the same or better outcomes at lower cost. The limitations of these methods, when applied in their best and most rigorous complete possible form, no less in this woefully insufficient and incomplete form simply do not allow for such extrapolation.

Further, given the crudeness of the models and adjustments used in the analysis, it is inappropriate to make too much, if anything of supposed differences in the characteristics of districts with good versus bad rate of return indices. You can do some fun hunting and pecking through the maps, but that’s about it!

For example, one major finding is that districts with good ROI’s spend less on administration. However, much more rigorous studies using better data and more appropriate models exploring precisely the same question have found the opposite. The CAP ROI methods are insufficient to draw any conclusion in this regard.

There is little basis in this analysis that states need to provide fairer funding – except that funding does appear to vary within states. But how funding varies is not explored. I like the idea of improving funding fairness, but a better basis for that argument can be found here: www.schoolfundingfairness.org.

And there is no basis for suggesting that fairer funding would be accomplished by student-based funding. Evidence to the contrary might be found here: http://epaa.asu.edu/ojs/article/view/5. This is report writing 101. Be sure that your policy implications follow logically from your findings, and that your findings are actually justified by your analysis.

It was also concluded that on average, higher poverty districts are simply less efficient. When you estimate a model of this type, where you are trying to account for various factors, such as cost factors, outside the control of local school districts – but you aren’t really sure you’ve accomplished the task – when you see a result like this it is a rather basic step to ask yourself – Did I really control sufficiently for costs related to child poverty? Is this finding real? Or is it an indication of bias in my model? A failure to capture real, important differences in the characteristics of these districts?

That single bias – failure to fully account for poverty related costs – is pervasive throughout the entire CAP ROI analysis. There is a strong relationship between poverty and supposed inefficiency in most states in the analysis. That bias exists in states where the state has provided additional resources to higher poverty districts, making them higher spending on average, and that bias exists even in states where spending per pupil is systematically lower in higher poverty districts. And every map and scatterplot in the analysis must be viewed carefully with an understanding of the pervasive, uncontrolled bias against higher poverty districts. A bias that results largely from failure to fully account for cost variation.

Okay, now that I’ve said that rather bluntly, let’s walk through the three different ROI approaches, and what they are missing.

Basic ROI

In any of the ROI’s you’ve got two sides to the analysis. You’ve got the student outcome measures (which I’ll spend less time on), and you’ve got the per pupil spending measures. Within the per pupil spending measures, you’ve got cost adjustments for the “cost” of meeting student population needs and cost adjustments for addressing regional differences in competitive wages for school personnel. The Basic ROI uses an approach similar to that used by Education Week in Quality Counts as a basis for calculating “cost adjusted spending per pupil.”

Weighted Pupil Count = Enrollment + .4*Free Lunch Count + .4*ELL Count + 1.1*IEP Count

After using the weighted pupil count to generate a student need adjustment, CAP uses the NCES Comparable Wage Index to adjust for regional variation in wages. So, they try to adjust for student needs, using a series of arbitrary weights, and for regional wage variation.

The central problem with this approach is that it relies on setting rather arbitrary weights to account for the cost differences associated with poverty, ELL and special education. And in this case, CAP, like Ed Week, shot low – claiming those low weights to be grounded in research literature, but that claim is a stretch at best and closer to a complete misrepresentation. More below.

Adjusted ROI

For the adjusted ROI, CAP uses a regression equation which compares the actual spending of each district to the predicted spending of each district, given student population characteristics. Here’s their equation:

ln(CWI adjusted ppe)= β0 + β1% free lunch + β2 % ELL+ β3 % Special Ed + ε

Now, this method is a reasonable one for comparing how much districts spend, but has little or nothing to do with adjusting for the costs of achieving comparable educational outcomes – a true definition of cost. That is, one can use a spending regression model to determine if a state, on average spends more on high poverty than on low poverty districts. But this is a spending differential not a cost factor. It’s useful, and has meaning, but not the right meaning for this context. One would need to determine how much more or less needs to be spent in order to achieve comparable outcomes.

So, for example, using this approach it might be determined that within a state, higher poverty districts spend less on average than lower poverty districts. This negative or regressive poverty effect would become the cost adjustment. That is, it would be assumed that higher poverty districts have lower costs than lower poverty ones. NO. They have lower spending, but they still most likely have higher costs of achieving constant educational outcomes. Including outcomes and holding outcomes constant is the key – AND MISSING – step toward using this approach to adjust for costs.

Further, the overly simplistic equation above completely ignores significant factors that do affect cost differences and/or spending differences across districts, such as economies of scale and population sparsity as well as more fine grained differences in teacher wages needed to recruit or retain comparable teachers across districts of differing characteristics within the same labor market.

Predicted Efficiency

Finally, there’s the predicted efficiency regression equation, which attempts to generate a predicted achievement level based on a) cost adjusted per pupil spending, b) free lunch, ELL and special education shares. This one, like the others doesn’t attempt to adjust for economies of scale or sparsity and suffers from numerous potential problems with figuring out how and why each district’s actual performance differs from its predicted performance.

achievement = β0 + β1 ln(CWI adjusted ppe) + β2 % free lunch + β3 % ELL + β4 %Special Ed + ε

In this (dreadfully over-) simplified production function approach, any individual district’s actual outcomes could be much lower than predicted or much higher than predicted for any number of reasons. It would appear from scanning through the findings that this particular indicator is most biased with respect to poverty.

Summary of what’s missing or mis-specified

The table below summarizes the three ROI indices – or at least the “adjusted expenditure” side of those indices – with respect to what we know are the major cost factors that must be accounted for in any reasonable analysis of education spending data in relation to student outcomes. Here, the basic conception of cost, and cost difference is “what are the differences in cost toward achieving comparable outcome objectives?” Cost cannot be estimated without an outcome objective.

First, I would argue that the selected weights in the Basic ROI are simply too low, especially in certain parts of the country.

Second, none of the models address economies of scale. CAP notes this, but in a section of the report most will never read. Instead, we’ll all see the pretty maps that tell us that all of the rural districts in the upper Hudson Valley in NY State or in north Central Pennsylvania are really, really inefficient.

Third, recall that the “adjusted ROI” model really doesn’t control for cost at all, but rather for underlying spending variation, without respect for outcomes.

Table 1


Regarding pupil need weights in particular, there exists at least some literature – the most rigorous and direct literature on the question – which suggests the need for much higher weights than those used by CAP. For example, Duncombe and Yinger note that in two versions of their models:

Overall, this poverty weight ranges from 1.22 to 1.67 (x census poverty rate), the LEP weight ranges from 1.01 to 1.42, and the special education weight varies from 2.05 to 2.64.

Across several models produced in this particular paper, one might come to a rounded weight on Census poverty of about 1.5 or weight on subsidized lunch rates of about 1.0 (100% above average cost, or 2x average, more than double the CAP weight), a weight on limited English proficient students around 1.0 and on special education students over 2.0 (slightly less than double the CAP weight).

Other work by me, along with Lori Taylor and Arnold Vedlitz, done for the National Academies of Science, reviewing numerous studies also comes to higher average weights – using credible methods – for children in poverty.

While one can quibble over the selection of “cost” weights from literature, the bigger deal for me remains that the findings of the various ROIs reflect such a strong bias that any reasonable researcher would be obligated to explore further, and perhaps test out alternative research based weights as a way to reduce the bias. It’s a never ending battle, and when you’ve improved the distribution in one state, you’ve likely messed it up in another (because different patterns of poverty and distributions of ELL children lead to different appropriate weights in different settings – even within a state). But this happens – if it turns out to simply be unreasonable to identify a global method for estimating ROIs across school districts and across states, THEN STOP!!!!! DON’T DO IT!!!!! IT JUST DOESN’T WORK!!!!

Here is an example of how much a corrected cost adjustment might matter, when compared with the Basic ROI. The scatterplot below includes one set of dots (red triangles) which represent adjusted operating expenditures of Illinois school districts using the Basic ROI weights. The other set of dots (blue circles) uses a cost index derived from a more thorough statistical model of the costs of achieving statewide average outcomes for Illinois school districts. For the highest poverty districts, the adjusted spending figures drop by $4,000 to $5,000 per pupil when the more thorough cost adjustment method is used. This is substantial, and important, since the ROI is much more likely to identify these districts as inefficient and might be used by state policy makers to argue that cuts to these districts are appropriate (when they clearly are not).

Figure 1


How you specify models to identify efficient or inefficient districts matters, a lot!

Here’s one example of how this type of analysis can produce deceiving results, simply based on the “shape” of the line fit to the scatter of districts. Below is a scatterplot of the cost adjusted spending per pupil for Illinois school districts (unified K-12 districts) in 2008, and the proficiency rates (natural log) for those districts. In this case, I’m actually using much more fully cost adjusted spending levels, accounting for regional and more local wage variation, accounting for desired outcome levels, for poverty, language proficiency, racial composition and economies of scale. As a result, the graph actually shows a reasonable relationship between cost adjusted operating expenditures per pupil and actual outcomes. Spending – when appropriately adjusted – is related to outcomes.

Figure 2


Even then, it’s a bit hard to figure out what shape “best fit” line/curve should go through this scatter. If I throw a straight line in there, and compare each district against the straight line, those districts below the line at the left hand side of the picture are identified as really inefficient – getting much lower outcome than the trendline predicts. But, if I were to fit a curve instead (I’ve simply drawn this one, for illustrative purposes), I might find that some districts previously identified as below the line are now above the line. Are they inefficient, or efficient? Who really knows, in this type of anlaysis!

My biggest problem with the CAP production function analysis is that they came to a result that is so strongly biased on the basis of poverty and instead of questioning whether the model was simply biased – missing important factors related to poverty – they accepted as truth – as a major finding that higher poverty districts are less efficient. It is indeed possible that this is true, but the CAP analysis does not provide any compelling evidence to this effect.

Research literature on this stuff

Note that there is a relatively large literature on this stuff… on whether or not we can, with any degree of precision, classify the relative efficiency of schools or districts. There are believers and there are skeptics, but even among the believers and the skeptics, all are applying much more rigorous methods and refined models and more fully accounting for various cost factors than the present CAP analysis. Here are some worthwhile readings:

Robert Bifulco & William Duncombe (2000) Evaluating School Performance: Are we ready for prime time? In William Fowler (Ed) Developments in School Finance, 1999 – 2000. Washington, DC: National Center for Education Statistics, Office of Educational Research and Improvement.

Robert Bifulco and Stewart Bretschneider (2001) Estimating School Efficiency: A comparison of methods using simulate data. Economics of Education Review 20

Ruggiero, J. (2007) A comparison of DEA and Stochastic Frontier Model using panel data. International Transactions in Operational Research 14 (2007) 259-266

What, if anything, can we learn from those pretty maps and scatters?

Now, moving beyond all of my geeky technical quibbling is there anything we actually can learn from the cool maps and scatters that CAP presents to us. First, and most important, any exploration of the data has to be undertaken with the understanding that all 3 ROI’s suffer from a severe bias toward labeling high poverty urban districts as inefficient and affluent suburban districts as highly efficient (especially in Kansas!). But, with that in mind, one can find some interesting contrasts.

First, I think it would be useful for CAP to reframe and re-label their color schemes. Here’s my perspective on their scatters and color coding. The assumption with the ROI is that there exists an expected relationship between adjusted spending and student outcomes. That’s the diagonal line. Districts in the lower left and upper right are essentially where they are supposed to be. There is nothing particularly inefficient about being in the lower left, or upper right. The use of orange to represent the lower left makes it seem like the lower left is like the lower right. The lower left hand districts in the scatterplot, in theory, are those that got screwed on funding and have low outcomes. Arguably, the lower left hand quadrant of the scatterplots is where one should go looking for school districts wishing to sue their state over inequitable and inadequate funding. These districts aren’t to blame. They are getting what’s expected of them. They are getting slammed on funding and their kids are suffering the consequences – that is, if there really is any precision (which is a really, really suspect assumption) to these models.

Figure 3


Historically, Pennsylvania has operated one of the least equitable, most regressive state school finance formulas in the nation (www.schoolfundingfairness.org). Philadelphia has been one of the least well funded large poor urban core districts in the nation. Strangely, Pittsburgh has made out much better financially. Here’s what happens when we identify the locations of a few Pennsylvania school districts in the CAP ROI interactive tool. I’ve recreated the locations of 4 districts. The location of Philadelphia actually makes some sense on the basic ROI. Philly is royally screwed. Low funding and low outcomes. The implication of the orange shading seems problematic. But if we ponder the meaning of the lower left quadrant it all makes sense. Now, I’m not sure Pittsburgh is really overfunded and/or inefficient, as implied by being in the lower right quadrant – but at least relative to Philadelphia, it does make sense that Pittsburgh falls to the right of Philadelphia on the scatterplot. Lower Merion, an affluent high spending suburb of Philly seems to be in the right place too. I’m not sure, however, what to make of any of the districts, including affluent suburban Central Bucks, which fall in the upper left – the Superstars.

Figure 4

Because the various ROIs generally under-compensate for poverty related costs, if a district falls in that lower left hand quadrant, we can be pretty sure that the district is relatively underfunded as well as low performing. That is, the district shows up as underfunded even when we don’t fully adjust for costs. This is especially true for those districts that fall furthest into the lower left hand corner. The basic ROI is most useful in this regard, because you know what you’re getting (specific underlying weights). I’ve opened up the comments section so you all can help me identify those notable lower left quadrant districts!

Finally

This type of analysis is an impossible task, especially across all states and dealing with vastly different student outcome data as well as widely varied cost structures. Only precise state by state analysis can yield more useful information of this type. A really important lesson one has to learn when working with data of this type is to realize when the original idea just doesn’t work. I’ve been there a lot myself, even trying this very activity on more than one occasion. There comes a point where you have to drop it and move on. Sometimes you just can’t make it do what you want it to. And sometimes what you want it to do is wrong to begin with. Releasing bad information can be very damaging, especially information of this type in the current political context.

But even more disconcerting, releasing bad data, acknowledging many of the relevant caveats, but then drawing bold and unsubstantiated conclusions that fuel the fire… that endorse slashing funds to high need districts and the children they serve – on a deeply flawed and biased empirical basis – is downright irresponsible.

References

Andrews, M., Duncombe, W., Yinger, J. (2002). Revisiting economies of size in American education: Are we any closer to consensus? Economics of Education Review, 21, 245-262.

Baker, B.D. (2005) The Emerging Shape of Educational Adequacy: From Theoretical Assumptions to Empirical Evidence. Journal of Education Finance 30 (3) 277-305

Baker, B.D., Taylor, L.L., Vedlitz, A. (2008) Adequacy Estimates and the Implications of Common Standards for the Cost of Instruction. National Research Council.

Duncombe, W. and Yinger, J.M. (2008) Measurement of Cost Differentials In H.F. Ladd & E. Fiske (eds) pp. 203-221. Handbook of Research in Education Finance and Policy. New York: Routledge.

Duncombe, W., Yinger, J. (2005) How Much more Does a Disadvantaged Student Cost? Economics of Education Review 24 (5) 513-532

Taylor, L. L., Glander, M. (2006). Documentation for the NCES Comparable Wage Index Data File (EFSC 2006-865). U.S. Department of Education. Washington, DC: National Center for Education Statistics.

NCTQ: We’re sure it will work! Even if research says it doesn’t!

Last spring, I had the pleasure of presenting on teacher labor market research in the same conference session in which a very interesting paper on mutual consent teacher contract changes was also presented (by Bethany Gross). This paper is a product of an organization I’ve poked fun at in the past (Center for Reinventing Public Education) but this one is good stuff, by credible authors.  The methods are relatively tight, but it is a bit tricky to figure out the implications of the findings – discussed blow.  This study fits into the broader topic and policy concern of “how do we get a better balance of teacher quality across poorer and less poor schools in the same district?”

Now, pundits (not these researchers) like those from Center for American Progress, Education Trust, some from CRPE and those from New Teacher Project and National Council on Teacher Quality (NCTQ) all seem to argue that the biggest teacher quality/sorting problems are those that occur across rich and poor kids within school districts – all because of teacher seniority preferences, tenure and contractual issues which all favor the interests of adults over the interests of children – like letting the senior teacher keep his/her job in the cushy school in the district (or letting the senior teacher bump the junior teacher from such a position).

As I’ve shown in many recent posts, in general, rich kids and poor kids, black kids and white kids don’t often attend the same school districts – but for a few very large urban ones and some other sprawling countywide systems. The BIG disparities in resources and teacher characteristics are across not within districts. They are disparities that result across different bargaining units – not within bargaining units. So it’s pretty hard to argue that most disparities in teacher quality across rich and poor kids from one location to another are caused by seniority focused, adult interest, contractual provisions.

But, setting that broader issue aside, what do we actually know about within district disparities in the distribution of teacher characteristics, and whether changing contracts to remove these “offensive” protections can actually help redistribute teacher quality? Well, here’s what Gross and colleagues found:

http://www.nctq.org/docs/Mutual_Concent_8049.pdf

We conduct an interrupted time-series analysis of data from 1998-2005 and find that the shift from a seniority-based hiring system to a “mutual consent” hiring system leads to an initial increase in both teacher turnover and share of inexperienced teachers, especially in the district’s most disadvantaged schools. For the most part, however, these initial shocks are corrected within four years leaving little change in the distribution of inexperienced teachers or levels of turnover across schools of different advantage.

So, initially the policy change actually made things worse and in the end, the policy change made things no different. There may actually be some reasonable explanations for these findings. Perhaps most problematic, teachers who are really beginning to hit their stride in years 5 to 10 or so, might take advantage of their newly discovered mobility to jump more quickly from positions in higher poverty, higher need schools into more desirable positions once reserved for the most senior teachers. This could create a substantial drain of quality – non-novice but not really old – teachers from high need schools.

Notice the URL for this study. It is posted on the NCTQ website. That doesn’t mean they ever read it though. SOMEHOW, THE NEW NCTQ REPORT WHICH ARGUES THAT THESE CHANGES ARE PART OF THE SOLUTION, DOESN’T PAY ANY ATTENTION TO THIS!

Here’s a link to that report: http://www.nctq.org/tr3/docs/nctq_site_based_hiring.pdf

Here’s the list of the 3 “major barriers” to improving the distribution of teacher quality, as identified by NCTQ:

• Centralized hiring. In most districts, the human resources office controls the hiring process, determining whom to recruit and hire and where to place teachers. Principals, at most, are given the opportunity to voice their preferences.

• Inadequate evaluations. Teachers in most districts are not regularly, or sufficiently, evaluated, meaning that evaluations can only play a minor role in personnel decisions, when they should be paramount. It is seniority, not performance, that decides the movement of teachers within the district.

• Contractual obligations. Most teacher contracts stipulate that, if a teacher loses her current assignment—because of a shift in the student population, for example—the district has to find her a new assignment, regardless of whether another school wants to accept the teacher. Compounding the problem is that most state laws limit the reasons districts can dismiss a teacher, and being without a classroom assignment is not one of them. Districts are left with little choice but to either assign teachers to positions or keep them on the payroll, sometimes for years, even if they aren’t teaching.

See that third one – Yep – it’s those  contractual provisions that keep these disparities in place. Remove then and all will be fixed!

Now briefly on their first point – that centralized hiring is the other really big problem. The answer – let school site principals make decisions and teachers decide which principal they really want to work for in a district. That couldn’t backfire?   Well, I used to believe the same – that this could be a reasonable idea. The problem with this idea is that principal quality is so disparately distributed. I have recently worked on several studies of principal labor markets, the distribution of principals by their academic preparation and other factors across schools within districts and the relationship between principal attributes and the teachers they hire. Given what we are learning from these studies, it is in fact very likely to backfire! The weakest principals tend to be in the highest need schools and weak principals tend to attract and potentially even retain weaker teachers.

This line in the NCTQ press release is particularly fun, because it’s based on nothing but “gut” and “emotional appeal” – which is always the best basis for experimenting with the lives of low income and minority children, right?

“Giving principals the authority to hire who works on their staff is critical,” says Kate Walsh, NCTQ’s president. “It is the only fair way to hold schools accountable for results. But if the principal doesn’t have enough control over the quality of her staff, the school—and, of course, the students—will suffer.”

You know what – not addressing the larger resource disparities across and within districts – which lead to the disparities in leadership quality across schools and districts – and then handing greater control over teacher hiring/firing to the least qualified principals in the highest need schools – yeah… that’s when children will suffer. Even if we start by getting good principals where they are needed most, we must provide them the resources to attract and retain the “better” teachers.

To summarize:

1. Decentralizing control of teacher hiring to principals, where principal quality distribution is disparate, to the disadvantage of high need schools, is likely to lead to worse, not better distribution of teacher quality;

2. Altering contractual provisions, such as moving from seniority based to mutual consent placement, appears to disadvantage higher need schools initially and in the end, leads to little or no substantive change in the distribution of teachers across schools.

Yet, let’s go with it. What the heck. Why not – it chips away at those facially offensive protections of stubborn old selfish, lazy unproductive teachers (yes, I do know a few, but that’s not the point). We know it should work, even if we have no evidence to that effect. When organizations like NCTQ present policy recommendations against their own evidence and built on such flimsy logic, why do we even listen?

Additional resources

The NCTQ report above concludes with a rant about ESEA Comparability Regulations. For my thoughts, see: https://schoolfinance101.wordpress.com/2010/05/31/research-schmresearch-caps-misguided-analysis-again/

Regarding financial resources, “autonomy” and the distribution of teachers, see: Baker, B.D. Re-arranging deck chairs in Dallas: Contextual constraints on within district resource allocation in large urban Texas school districts. DeckChairsinDallas.Baker

Regarding the distribution of school leaders, teacher hiring and all that stuff, see:

Fuller, E., Young, M.D., Baker, B.D. Career Paths and the Influence of School Principals on Teachers. (Available on request)

Baker, B.D., Fuller, E. The Declining Academic Quality of School Principals and Why it May Matter. Baker.Fuller.PrincipalQuality.Mo.Wi_Jan7

Punswick, E., Baker, B.D., Belt, C. Principal backgrounds and school leadership stability: Evidence from Missouri. Educational Administration Quarterly Punswick.Baker.Belt.MoPrins09

Principal moves/exits: http://eaq.sagepub.com/content/46/4/523.abstract

Principals and hiring: http://eaq.sagepub.com/content/41/3/449.abstract

(the references in the above articles may provide some additional useful guidance on the role and the current distribution of principals)

Research, Schmresearch – CAP’s misguided analysis… AGAIN!

Center for American Progress has just released a new report titled Comparable, Schmomperable which argues that within-district disparities are the major equity problem of the day. As I have noted previously, I agree that within-district inequities in schooling resources including teacher quality are a concern – A major concern.

However, to ignore and brush aside disparities between districts is absurd.

The schmreport author Reagan Miller argues:

State funding formulas tend to exert an equalizing effect on per pupil revenues between districts, on average, and not by accident. These formulas were sculpted by two generations of litigation and legislation seeking equitable or adequate funding for property-poor school districts.

Sculpted they were? By litigation? With the consistent (they “tend to”) and persistent effect of resolving the vast majority of between-district funding disparities? Interestingly, the Schmreport author Reagan Miller cites these claims to a book by Eric Hanushek and Al Lindseth which had as its singular objective to argue that school funding litigation and school finance reform are invariably ineffective.  A critique of the book’s arguments and review of school finance litigation and its effects can be found here.

Co-author Kevin Welner and I provide a smack-down of the Center for American Progress (and Education Trust) argument that between-district inequities are a thing of the past – solved by years of litigation – here:

  • Baker, B. D., & Welner, K. G. (2010). “Premature celebrations: The persistence of interdistrict funding disparities” Educational Policy Analysis Archives, 18(9). Retrieved [date] from http://epaa.asu.edu/ojs/article/view/718

In the new Schmreport, CAP’s Reagan Miller seems to take the rhetoric to a new level. The  recent Education Trust “Loophole” report took a similar approach, making similar arguments, but I found the language and disturbingly thin research base in the new CAP report particularly troublesome.  Also, the inference that school finance reforms addressing between-district disparities have been ineffective (by way of citing Hanushek and Lindseth as the primary source chronicling state school finance reforms) but that leveraging Title I funding to force school districts to resolve internal disparities is the final frontier of reform seems a bit odd. If between district disparities and state school finance reform don’t matter (as per Hanushek and Lindseth), should we really care about within district disparities? Certainly we should care about both.

After declaring that state funding formulas “tend to” fix those little between-district funding equity problems, the report goes on to point out that New Jersey has achieved a strong positive relationship between state and local revenues and child poverty across districts and does acknowledge that other states have not – using Connecticut as an example. Strangely, while Connecticut’s funding distribution is problematic, it is not one of those strongly “regressive” states. [There are plenty of those. There are also those states which essentially spend nothing! and those where fewer than 80% of school aged children even use the public school system. We call those states RttT winners.] Rather, Connecticut suffers a strange randomness in school funding across districts. The report goes on to point out that even if Connecticut were to preemptively (before the pending case goes to trial) solve between-district funding problems, within-district funding problems could thwart any chance of actually solving the state’s equity problems. At the very least, if you’re going to make a claim about a state, take a few minutes to check that the data on that state you pick is at least somewhat consistent with your claim. This brief, flyover section of the report revealed to me a peculiar disregard for precision or accuracy.

I discuss the sorting of children across districts in Connecticut here, pointing out that between district concerns are by far the dominant issue in that state, merely as a function of the patterns of student segregation across districts (not between them!)

After brushing aside the possibility that between-district disparities remain a major concern, the CAP Schmresearch Schmreport goes on to say:

Scandalous inequity in the distribution of resources within school districts has plagued U.S. education for more than a hundred years.

Indeed… scandalous, but those disparities between school districts in states like Illinois could never reach the height of scandalous? I also found no citation to the hundred year old studies that document these disparities, but now I’m just being picky.

The Schmresearch Schmreport also argues in its introduction that:

“empirical literature documenting the extent of within-district inequity is astonishingly thin.”

Yet, as Kevin Welner and I point out, there is actually quite a large volume of research on within-district disparities in schooling resources, including within-district disparities in teaching quality, and some of it – much of the “research” relied on by CAP to construct their argument is empirically problematic (to be kind). There is also a significant body of good empirical research on the topic, notably absent in the Schmresearch Schmreport. (the report cites a few – very few – good studies on teacher quality distribution)

Without a doubt, there exist some truly problematic – perhaps even scandalous disparities in resources across schools within school districts. It is quite possible that Title I funding could be better leveraged to encourage districts to do a better job at improving equity across schools within districts. But it is completely irresponsible and outright ignorant to suggest that between district disparities have already been largely resolved.

Further, it is completely unnecessary to frame the argument in this way, unless CAP has a political motive to blame districts not states and to argue that no more money is actually needed (forcing districts to re-arrange deck chairs without solving between district disparities can be a ‘revenue neutral’ solution. It’s not much of a real solution though). And yes, states can do a better job of providing data systems that allow more precise tracking of within district inequities. But, I should note that many already do a much better job than Reagan Miller suggests. A wealth of information can be found in statewide personnel data systems which link individual teachers to schools.

This is not an either/or issue. It’s not about solving within-district disparities because between-district disparities are solved – been there, done that. Wrong.  Both are persistent problems, more in some places than others, and it’s worth the time and effort to figure out how to leverage all available policy options to figure out how to fix both within and between-district disparities.

Ed Trust Getting Loopy Again

Education Trust has released another BIG statement about an issue that I would argue is a minor distraction – at best. At worst, this issue becomes a major policy distraction, diverting attention from far more significant equity concerns.

http://www.edtrust.org/sites/edtrust.org/files/publications/files/Hidden%20Funding%20Gaps_0.pdf

Education Trust’s summary bullet points for their new report are as follows:

  • Federal law permits hidden funding gaps to persist between high-poverty schools and more affluent counterparts within the same district.
  • These gaps occur partly because teachers in wealthier schools tend to earn more than their peers in high-poverty schools and because of pressure to “equalize” other resources across schools.
  • By closing loopholes in the comparability provisions of Title I of the Elementary and Secondary Education Act, Congress could promote funding equity within school district budgets.

    • The report is grounded in this premise:

    Many states have made progress in closing the funding gaps between affluent school districts and those serving the highest concentrations of low-income children. But a hidden funding gap between high-poverty and low-poverty schools persists between schools within the same district. District budgeting policies frequently favor schools with the fewest low-income students. This undercuts the aim of Title I and robs poor children of funds intended to help them.

    The layers of problems with this premise and Education Trust’s major conclusions are downright baffling. I am not suggesting that we should not be concerned with inequities that might occur between schools within districts, inappropriately as a function of district budgeting practices or teacher assignment practices. These are a concern. They are just not the major equity concern du jour. And further, while Title I funding might be leveraged better to correct this concern, the role of Title I funding in improving equity overall across states is minimal.

    Issue 1 – Ed Trust’s misguided perception that states have solved between-district inequities, leaving within-district inequities as the primary policy concern

    On this particular point, here is an abstract of a forthcoming peer-reviewed article and analysis which I have written (with Kevin Welner) on this topic:

    Two interlocking claims are being increasingly made around school finance: that states have largely met their obligations to resolve disparities between local public school districts and that the bulk of remaining disparities are those that persist within school districts. These local decisions are described as irrational and unfair school district practices in the allocation of resources between individual district schools. In this article, we accept the basic contention of within-district inequities. But we offer a critique of the empirical basis for the claims that within-district gaps are the dominant form of persistent disparities in school finance, finding that claims to this effect are largely based on one or a handful of deeply flawed analyses. Next, we present an empirical analysis, using national data, of 16-year trends (1990 to 2005) and recent patterns (2005 to 2007) of between-district disparities, finding that state efforts to resolve between-district disparities are generally incomplete and inadequate and that in some states, between-district disparities have actually increased over time.

    Education Trust chooses anecdotal comparisons between New York City schools in order to make their case that within-district funding disparities are the problem du jour.  Without a doubt, within district disparities persist in New York City schools, some unexplainable and many that should be remedied. I’m all for fixing illogical disparities.

    The use of New York City anecdotes to illustrate supposed major national policy concerns, in this case by authors Daria Hall and Natasha Ushomirsky piggy-backs on similar “shock” comparisons used in op-eds by Marguerite Roza – an author cited by the Hall/Ushomirsky brief. We are led to believe that this is an example of one of the greatest equity problems facing schoolchildren in the entire NY metropolitan area – if not the nation as  a whole. After all, the state of NY has done its job to fix between district disparities. Right? It’s the city that’s not pulling its weight?

    Here’s one fun example, from our forthcoming article:

    Following a state high court decision in New York mandating increased funding to New York City schools, Roza and Hill (2005) opined: “So, the real problem is not that New York City spends some $4,000 less per pupil than Westchester County, but that some schools in New York [City] spend $10,000 more per pupil than others in the same city.” That is, the state has fixed its end of the system enough.

    This statement by Roza and Hill is even more problematic when one dissects it more carefully. What they are saying is that the average of per pupil spending in suburban districts is only $4,000 greater than spending per pupil in New York City but that the difference between maximum and minimum spending across schools in New York City is about $10,000 per pupil. Note the rather misleading apples-and-oranges issue. They are comparing the average in one case to the extremes in another.

    In fact, among downstate suburban[1] New York State districts, the range of between-district differences in 2005 was an astounding $50,000 per pupil (between the small, wealthy Bridgehampton district at $69,772 and Franklin Square at $13,979). In that same year, New York City as a district spent $16,616 per pupil, while nine downstate suburban districts spent more than $26,616 (that is, more than $10,000 beyond the average for New York City). Pocantico Hills and Greenburgh, both in Westchester County (the comparison County used by Roza and Hill), spent over $30,000 per pupil in 2005.[2] These numbers dwarf even the purported $10,000 range within New York City (a range that we agree is presumptively problematic); our conclusion based on this cursory analysis is that the bigger problem likely remains the between-district disparity in funding.

    The bottom line regarding this first claim is that much of the “research” to which the new Ed Trust brief points as validating within-district disparities being the greatest problem of our day, and states largely fulfilling their objectives to improve equity, is misguided, shoddy or both. Between district disparities remain a major concern and vary widely across states.

    Issue 2: Ed Trust’s strange misperception that rich and poor kids actually attend the same school districts, such that between district inequity even could be the major equity concern du jour

    Yes, there are cases where lower poverty schools exist in generally higher poverty school districts. But as I have pointed out in other posts, for the most part, low poverty schools exist in some districts and high poverty schools in other districts. The same is true of minority student concentrations. When we start talking about leveraging Title  I aid to FIX equity problems within districts we are largely talking about redistributing resources across schools which have from 60% to 80% or higher low income children, while ignoring all of the surrounding school districts which have 0% to 60% low income kids. Here are just a few examples of the clustering of high poverty schools within districts.

    First, Chicago Public Schools. Here’s a map of the higher and lower poverty and higher and lower minority concentration schools in Chicago Public Schools and surroundings. Yes, there are some lower poverty schools in Chicago and perhaps a few (very few) with fewer minority children. But, on average reshuffling Chicago public schools resources means reshuffling them from poor to really poor schools or black to Hispanic schools, or back.

    Here are the school enrollments – weighted by number enrolled – expressed as a frequency distribution by low income student concentrations:

    You see, the vast majority of CPS kids are in high poverty schools and the majority of kids in schools outside of CPS but in the same metro area, are in low poverty schools. It’s really that simple. Yeah… there’s variation in CPS. But the greater variation is between CPS and surroundings.

    Here’s Newark, NJ:

    And here’s the frequency distribution:

    The assumption implicit in the Education Trust reports, including this brief, that higher and lower poverty schools are distributed (evenly?) across districts, that states have solved the between-district equity problem, and now districts are the ones that need to step up, seems a bit off base. No. Actually, it’s just WRONG.  On average, high poverty schools exist in high poverty districts and low poverty schools in low poverty districts. In some very large districts like NYC or CPS you can find examples of lower poverty schools in the presence of higher poverty ones. But this is not the norm.

    Issue 3: Education Trust’s untested and misguided assumption that Title I funding provides major leverage for improving school funding equity

    I don’t want to totally downplay the role of Title I funding here. It is important and quite significant for some school districts. But, the effects of Title I funding on equity – especially cross-district equity within states, are relatively small. In a forthcoming report, we lay out a methodology we use for estimating the extent to which states drive more funding (cumulative state and local revenues) to higher poverty school districts. Kevin Welner and I use a similar method in the paper cited above. I will provide a link to that paper when it is posted.

    In these graphs, I estimate a model of the relationship between state and local revenue and school district poverty rates, controlling for a variety of other cost factors (see Baker/Welner article). In a second version of the models I include Title I funding. As you can see, if a state is already providing an upward tilt (progressive with respect to poverty) to their funding, Title I modestly bumps up that tilt. But, for states with a downward tilt (regressive with respect to poverty) Title I aid doesn’t even bring it back to flat.

    Quite honestly, if there’s a loophole associated with Title I funding, it’s that states like New York, Illinois and Pennsylvania (and Arizona) have been able to use Title I aid to provide a small slice of support to poor urban and small city districts that the states themselves have chosen to neglect entirely. Now, Pennsylvania has started to make some progress on this front. But, Illinois has actually slid backwards over time, and New York made little progress following the court ruling noted above.

    In closing, I do concur with Education Trust that we should be concerned with inequities that arise for a variety of reasons across schools within large school districts.  But, overemphasizing this point creates a major distraction from the more significant disparities that persist. Actually prefacing the argument with the claim that these disparities are less important or unimportant – as this Education Trust report does – is very problematic and unfounded.

    Suggesting that Title I funding and the “comparabiltiy loophole” are the panacea for the most significant persistent disparities is just…well… LOOPY!

    CAP’s Title I Myth

    I just read a copy of “Spoonful of Sugar” from Center for American Progress in which they again propose fixes to Title I funding, which I have pointed out in the past are based on misguided assumptions and analyses (or lack thereof).

    Please see my previous analysis here:

    https://schoolfinance101.wordpress.com/2009/11/27/title-i-does-not-make-rich-states-richer/

    The authors of this “spoonful” note:

    Still other districts would see their allocations drop because the proposed formula
    removes the current bias toward high-spending states that exert relatively low fiscal
    effort in education funding.

    (page 2)

    The inclusion of an effort factor in this particular version of the CAP analysis appears to improve their proposal somewhat (perhaps… maybe?). But the proposal – or spoonful – still fails to address the poverty measurement problem which is substantial.  For example, the spoonful recommends much larger increases for Mississippi, Arkansas and Alabama than for New York, California or New Jersey (Figure 1). But, this figure shows that after correcting for poverty measurement and regional costs, the latter three states are much less well funded under Title I presently than the former three states.

    My post above explains clearly and with data and with links to prior presentations and Census bureau analyses on this topic that this assumption by CAP that Title I funding favors rich states is simply wrong and based on bad analyses (rooted in mis-measurement of poverty differences across states).   Yes, it is illogical to drive money to states simply on their ability to spend more on their own. That portion of the current Title I formula is problematic (okay… just plain silly) on its face. But, the resulting distribution pattern is far less problematic and – once again – does not, on average favor Rich States over Poor States.

    Apparently, I need to reinforce this point even more (from previous post):

    Let me clarify that the same issue of mis-measurement of poverty plagues urban-rural comparisons within states. Rural poverty is, in relative terms, overstated compared to urban poverty. So too are rural costs (competitive wages) lower than urban costs. So, just as it is true that Title I does not necessarily overfund “rich” states, Title I also does not necessarily overfund urban districts at the expense of rural ones. Unfortunately, I do not yet have available a finer grained adjusted poverty measure which will allow me to easily display the urban/rural issue.

    CAP’s spoonful brief is backed by their analyses here: http://www.americanprogress.org/issues/2010/02/pdf/bitter_pill.pdf

    These analyses include faulty assumptions about rural-urban poverty distribution in an ill-conceived example applied to Missouri – a state on which I have conducted extensive research on school funding in the past.

    Racial Achievement Gaps and Within-District Funding Inequity

    I’ve written on this blog on a number of occasions, comments regarding the relative significance of within versus between district funding inequities. For example, I’ve explained (in response to an absurd claim by pundits from Education Trust) that southern states have not, in fact, substantively resolved between district funding disparities – leaving only district allocation policies to blame for persistent inequities.

    A Center for American Progress report claims:

    One of the most harmful manifestations of this is that local school district funding is allocated in a way that hurts poor and minority students. A study by the Thomas B. Fordham Institute found that educational funding is being allocated on the basis of “staff allocations, program-specific formulae, squeaky-wheel politics, property wealth, and any number of other factors that have little to do with the needs of students.”1

    The outcome of such practices is predictable: A further widening of the dangerous achievement gap that has become endemic in American schools today.

    http://www.americanprogress.org/issues/2008/06/pdf/comparability.pdf

    Notably, some of the above concerns raised in the Fordham report are between, not within-district concerns (property wealth related disparities), but the CAP report focuses on within district disparities as the central issue.

    I have a forthcoming article (which I will forward by request) which explains that the existing literature which claims that within-district disparities far outweigh between-district disparities is problematic at best, drawing premature if not entirely unfounded and overextended conclusions. This is not to deny the problem of within district disparities but rather to point out that between district disparities persist and must be addressed either simultaneously with or as a precursor to resolving within district disparities.

    Many pundits who wish to shift focus entirely onto within district disparities, blaming districts instead of state funding policies, seem to base their arguments on the idea that within-district funding disparities are the reason for persistent racial achievement gaps. Their story goes… that over the decades through the 1990s, states fixed between district funding disparities and achievement gaps improved. Since that time however, improvement to achievement gaps has stagnated if not backslid (true), with pundits arguing that the persistent within district disparities are the cause (unlikely). That is, that individual school districts are now funding their non-poor, white schools well and depriving their poor minority schools. That districts are allowing their better teachers to transfer from the high poverty, black schools to their low poverty white schools. There are certainly cases where this is true (Cincinnati accomplishes this through its weighted student funding formula by weighting gifted children more than poor children).

    As a broader policy concern, the above argument might make sense if, in fact, student populations across schools within districts varied widely but that student populations vary less between districts. That is, it might make sense to argue that between-school within-district funding disparities are causing racial achievement gaps if racial minorities and whites attended the same districts but not the same schools. But that’s not always, or often the case, especially in densely populated states and metropolitan areas which included many small school districts.

    Allow me to use Connecticut – a state with among the largest racial achievement gaps – as an example. Here’s the racial composition (black enrollment share in red bars, Hispanic share in yellow bars) for Hartford area school districts (click to enlarge). Those flat bars in other districts are schools with few or no black or Hispanic children.

    In Connecticut, like New Jersey or like the Chicago metro area, school districts tend to be either minority or white – not a balanced mix sorted across schools. Hartford, in this case, can only re-allocate resources across schools that are all approximately 99% poor, and either majority black (north end) or majority Hispanic (south end) schools (except for the magnet schools which serve relatively smaller portions of the district population).

    New Britain, to the southwest of Hartford can allocate resources across predominantly Hispanic schools or other predominantly Hispanic schools.

    Meanwhile, West Hartford, Simsbury, Avon, Newington, Wethersfield and others can allocate resources across white schools and other white schools (some in West Hartford having modest minority populations)

    Here’s the New Haven area:

    And the Bridgeport area:

    So, at least in Connecticut, it would appear highly unlikely that within- district resource allocation across schools could be fueling their large achievement gaps. That’s because – for the most part – the minorities attend some districts and the whites attend other districts. That’s not to say there aren’t likely some pretty big within district funding disparities in these districts, but in some districts those disparities exist between blacks and Hispanics, or Hispanics and other Hispanics, blacks and other blacks and  in the other districts the disparities are between whites and other whites. For the most part, minority students attend minority districts and white students attend white districts in Connecticut. Patterns are similar in the Chicago metro area and in New Jersey.

    Yes there are exceptions – racially integrated middle class inner-urban-fringe and suburban districts. But these exceptions do not account for the majority of minority or white students by any stretch of the imagination. And yes, in these exception districts, there are often very large achievement gaps even within schools. That is a separate and equally important (though smaller in magnitude) story.

    It is an absurd stretch, however, to blame between-school within-district allocation policies for large achievement gaps in states like Connecticut, where minority students and white students attend different districts, much more so than different schools within the same districts.

    See my previous post on between-district disparities in Connecticut here:

    https://schoolfinance101.wordpress.com/2009/08/19/random-thoughts-on-ct/

    Here are the scatterplots of the school level free/reduced lunch rates and black and Hispanic concentrations for the above urban CT districts – elementary schools. Dots in red are schools within the district in question. Blue dots are schools in all other districts, including the other urban districts. Note that in Hartford and Bridgeport in particular, all elementary schools are high poverty and high minority concentration. New Haven is more diverse, but still less diverse than the statewide (between district) distribution.

    CT School Demographics-Elementary