Here it is – my annual update of America’s most screwed school districts. This time, for stability purposes, I’ve used a 3-year average based on 2009-2011 data (2011 data being released earlier this week).

As I’ve explained in my previous posts on this topic (from last year’s post on screwed districts)…

It’s important to understand that the value of any given level of education funding, in any given location, is relative. That is, it doesn’t simply matter that a district has or spends $10,000 per pupil, or $20,000 per pupil. What matters is how that funding compares to other districts operating in the same labor market, and for that matter, how that money relates to other conditions in the region/labor market. Why? Well, schooling is labor intensive.  And the quality of schooling depends largely on the ability of schools or districts to recruit and retain quality employees. And yes… despite reformy arguments to the contrary – competitive wages for teachers matter!  The largest share of school district annual operating budgets is tied up in the salaries and wages of teachers and other school workers. The ability to recruit and retain teachers in a school district in any given labor market depends on the wage a district can pay to teachers a) relative to other surrounding schools/districts and b) relative to non-teaching alternatives in the same labor market.

In our funding fairness report, we present statewide profiles of disparities in funding with respect to poverty. But, I thought it would be fun (albeit rather depressing) here to try to identify some of the least well-funded districts in the country. Now, keep in mind that there are still around 15,000 districts nationwide.

Here is this year’s empirical definition of “screwed” in school finance terms:

1. State and Local Revenue per Pupil (Census Fiscal Survey, 3-year Average) less than 95% of average for districts in the same labor market*
2. Adjusted Census Poverty Rate for 5 to 17 year olds (Census Small Area Income and Poverty Estimates, 3-year average) greater than 50% above average for districts in the same labor market.

*where “labor market” is defined as it is defined in the NCES Education Comparable Wage Index (essentially by core based statistical area for all districts in metropolitan or micropolitan areas).

Put very simply, districts with higher student needs than surrounding districts in the same labor market don’t just require the same total revenue per pupil to get the job done. They require more. Higher need districts require more money simply to recruit and retain similar quantities (per pupil) of similar quality teachers. That is, they need to be able to pay a wage premium. In addition, higher need districts need to be able to both provide the additional program/service supports necessary for helping kids from disadvantaged backgrounds (including smaller classes in early grades) while still maintaining advanced and enriched course options.

The districts in this table not only don’t have the “same” total state and local revenue per pupil than surrounding districts. They have less and in some cases they have a lot less! In many cases their child poverty rate is more than twice that of the surrounding districts that continue to have more resources.

List includes only those districts with Urban Centric Locale Codes for Cities (11,12,13) or Suburbs (21,22,23).

And here’s a list of the states with the largest shares of children attending “screwed” districts:

Here are the patterns of “screwedness” in states which seem to have relatively large numbers of screwed districts.

In short – school funding disparities are alive and well – and certainly don’t appear to by improving substantively in recent years. More on that at a later point.

As I’ve explained on previous posts, specific to New Jersey, claims of the dying private sector in education are grossly over exaggerated.

These days, such claims are often over exaggerated with the purpose of framing some broad policy interest in supporting private schools. That is, some need for immediate public policy attention to the problem – some reason to consider how to better integrate our private sector schools into the provision of the public good of elementary and secondary education.

It is argued broadly that the loss of our ever important private sector of schooling is a threat to educational excellence – or even national security. That this loss is of particular concern for our middle and lower income populations who have now lost access to private sector schooling.

In short, policymakers must act swiftly to stabilize this “too big to fail” sector of schooling that is critical to the future of low income children in America. This is not a religious issue. It’s a public interest issue. It has no religious boundaries. No specific religious identity. It is entirely neutral of religion. Or is it?

Indeed, I’m painting a caricature of recent arguments regarding private schooling and the public good. But I would argue that this caricature is reasonably representative of Checker Finn’s recent Fordham Institute editorial on “Why Private Schools are Dying Out.”

Checker begins with bold claims of a dying private schooling industry… though noting that elite schools are doing just fine.  His precision of analysis ends about there – elites okay… everyone else dying.  Checker points to “other countries” (an ambiguity that drives me nuts… and is entirely pointless in its absurd imprecision), noting:

Most other modern countries have essentially melded their private education sectors into their systems of public financing

The implication to be drawn (though not entirely clearly stated) from Checker’s rant on the decline of private schooling in America is that we too must find ways to act like “other modern countries” (which obviously trounce us on international tests) and meld our public and private systems with public funding toward the greater public good.

To believe this line of reasoning, one must be entirely ignorant of the social/cultural religious demography of the United States and the history, long past and recent of private schooling in America (some data provided here).  One would also have to ignore any and all publicly available data from which one might actually explore patterns of private school enrollment over time. I just can’t do that. It’s too easy to check.

Evidence from the NCES Private School Universe Survey

Let’s start with the national biennial survey of private schools conducted by the National Center for Education Statistics.  First, here are estimated total private school enrollments by grade level.

As we can see, elementary enrollments do appear to be dropping.

Now, here it is by region. There appears to be some decline in the Northeast and Midwest. But, the south and West are relatively stable.

Now, by affiliation, or in the NCES dataset, an aggregate typology they’ve created. Here what we see is that most sectors of private schooling in the U.S. are, in fact stable. One is not. Parochial Catholic schools have declined appreciably in the past decade, assuming these data to be accurate (with the decline starting well before that). Yes, it’s a big sub-sector within private schooling. Formerly the biggest. But it is one specific sub-sector.

Evidence from Census Data and the American Community Survey

Now, let’s take a look at Census Bureau data from 2000 to 2011, including a breakout of children from lower income families (those below 250% of the income level for poverty). Here, I look at 6 to 16 year olds because, in states where kindergarten is half-day, larger shares attend private schools, and where full day kindergarten increases by policy change, private school shares decline. Here I’m trying to capture the core of the school aged population.

What we see is some drop among the total population, but the lower income population is relatively stable.

Clarifying the Policy Inferences

First and foremost, the graphs here are still far too highly aggregated. This issue requires much more parsing by state, metro area, by affiliation and grade ranges, and so on. National averages of anything in the U.S. tend not to be very useful.

That aside, what we see here is that lower income families are enrolled at roughly a constant rate over time. They are not declining. Not losing substantial access. That said, they have much less access than their higher income peers to begin with. 

Further, most types of private schools have remained stable over time – nationally.  There is an apparent small shock between 2008 and 2010. Catholic Church affiliated schools have indeed declined in enrollment over the long haul, and significantly. This includes especially parochial, largely elementary schools, and recently, church affiliated Diocesan upper schools.  

Overall, high school enrollments are up. Middle school enrollments are stable. Elementary school enrollments are down, and largely due to the decline in Catholic parochial elementary schools.

So, let’s be sure to see the public policy question for what it is. And to see it for what it is, we must start with a reasonable look at the available data.

Private schooling isn’t generally in decline. Church affiliated Catholic schooling is.  There may be a variety of reasons for this. Middle class Catholic families have migrated out of the cities for decades.  Urban catholic school enrollments have become increasingly non-Catholic (in many regions/cities), arguably altering (if not compromising) the Catholic mission of those schools. Non-Catholic low income families in cities have been provided more non-religious choices, through expanded charter schooling (perhaps the primary reason?).  While attempting to uphold their service mission, holding down tuition and providing breaks for low income families, Catholic schools have become increasingly cash strapped. Notably, Catholic schools today operate with a much smaller share of in-kind, Church staff than they used to. Staffing costs have risen dramatically.

So then, if we accept these trends among one subset of private schools as being a substantial national policy concern (which some may), then the question is not broadly how to better integrate private entities of all stripes into the provision of elementary and secondary education as a public good, but rather about whether we should be using public funding to stabilize a subset of private schools of a single religion – in other words, should we use public dollars to bail out Catholic schools in decline?  

When viewed as a taxpayer subsidized bailout of Catholic Church affiliated schooling, these policy proposals are no longer religion neutral – are they? (nor are they particularly good for Catholic education)

New York State repeatedly makes the top of our list of most INEQUITABLE state school finance systems in our National Report Card.

http://www.schoolfundingfairness.org/National_Report_Card_2012.pdf

Further, NY State was acknowledged in a report I prepared last year, with colleague Sean Corcoran of NYU, identifying states that are both generally inequitable and that actually exacerbate those inequities through their state school finance systems.

http://www.americanprogress.org/wp-content/uploads/2012/09/StealthInequities.pdf

As I’ve explained in several previous blog posts, NY State’s primary policy response has been to brush aside the issue – blame school districts for being inefficient – pretend that the state already spends way too much – and ram ill-conceived policies down the throats of local district officials.

Here’s one choice quote from the Governor:

“The problem with education in New York is not money,” Cuomo said. “We have one of the highest spending rates in the nation. Our performance isn’t where our money is.”[1]

Of course, it’s not about money. Couldn’t possibly be.  If Scarsdale has enough, they must all have enough. If New York districts spend, on average, more than New Mexico districts, it has to be more than enough.

But these are meaningless comparisons. Cost pressures in education are primarily local/regional. Education is a labor intensive industry. Salaries must be competitive on the local/regional labor market to recruit and retain quality teachers. And for children to have access to higher education, they must be able to compete with peers in their region. And within any region, children with greater needs and schools serving higher concentrations of children with greater needs require more resources – more resources to recruit and retain even comparable numbers of comparable teachers – and more resources to provide smaller class sizes and more individual attention.

More information on how and why money matters can be found here:

http://www.shankerinstitute.org/images/doesmoneymatter_final.pdf

Here’s one snapshot of spending disparities in downstate/NYC/Long Island (identified by Regional Cost Index Region).  Remember – on average – higher need districts generally require more financial support per pupil to move toward comparable outcomes.  Here, along the horizontal axis, we have district rates of children qualified for free or reduced lunch compared to the average district in their “core based statistical area” (basically, metro area). Along the vertical axis, we have the Approved Operating Expense per Pupil, relative to the average district in the CBSA.  Now, in a “fair” system, the pattern would tilt upward from left to right. Not in NY.

Figure 1

What we see here is that many districts having several times the average low income concentration as their surrounding districts also spend much less than the average of their surrounding districts.

Now, state policymakers have been patting themselves on the back of late for their supposed generous increases to state aid for the coming year. The following graph puts those increases into context. As I’ve explained in previous posts, NY State has failed for years to even come close to funding it’s state general aid formula. The graph below organizes districts into quintiles by student need – from low need districts to high need districts based on the state’s own pupil need index (driven largely by low income population shares). The blue bars show how much less state aid than promised, each group is receiving on average. The red and green bars show the aid increases for the coming year. They barely make a dent in the current underfunding of the state’s own formula (which sets a very low bar to begin with).

Figure 2

So then, what are the consequences of this? Michael Rebell and colleagues at Teachers College put out a series of papers last Fall/Winter discussing essential resources, where the essential resource benchmarks are derived from an earlier court order in Campaign for Fiscal Equity v. State.

Although there is no specific maximum class size number beyond which children cannot learn, the Court of Appeals has indicated that classes of about the sizes listed below are appropriate and that larger class sizes may lead to unsatisfactory results.101 For schools and classes with large concentrations of students below grade level, and for AIS and RTI services, smaller class sizes may be necessary.

Kindergarten-grade 3: 20 students

Grades 4-6: 21-23 students

Middle and High School: 21-23 students (p 13-14)

• First, that high poverty schools which most need smaller class sizes tend to have higher proportions of children in larger classes.
• Second, it’s getting worse, not better as a function of the state’s failure to fund high poverty schools appropriately.

Figure 7. Middle School Class Sizes in NYC

 

Figure 8. Middle School Class Sizes in Wealthy Districts in Region

• class sizes in NYC schools continue to increase over time.
• class sizes in NYC schools are much larger than those in the top wealth/income quintile
• Further, compared to essential resources thresholds, class sizes in NYC are freakin’ huge! Yeah… that’s a technical term for you… freakin’ huge!!!!!

For those hack pundits who’ve latched on to the “uncertainty” or “narrowness” of research on the effectiveness of class size reduction (& bogus characterizations of “cost effectiveness”), there is little if any justification for permitting class sizes in high poverty settings at 30 or higher. Further, class size, and total student load are a relevant working condition influencing teacher recruitment/retention.

In simpler terms, there is certainly little basis for the inequity here. From a simple fairness standpoint, it makes little sense that children in the top 20% districts by wealth and income should have access to such smaller classes than children in New York City and that these disparities should year after year be a byproduct of the state’s dysfunctional, inequitable school finance system and overblown false claims that serve to maintain that status quo!

PDF of original CER response to LA court ruling: Louisiana High Court Violates Parent Rights (in case they revise/retract)

Now I know that the last thing reformy types really want to think about – to bother themselves with – is a basic understanding of law, civics and the structure of American government. All  that stuff is just an annoyance – an impediment to reformy awesomeness.

As such, it comes as no surprise to me that Jeanne Allen of the Center for Education Reform in the wake of today’s  Louisiana Supreme Court decision overturning that state’s private school voucher program, has issued perhaps the most over the top ignorant response I’ve seen in quite some time. Here are a few choice quotes from the CER press release:

“If indeed the Louisiana constitution, as suggested by the majority court opinion, prohibits parents from directing the course of the funds allocated to educate their child, then the Louisiana constitution needs to be reviewed by the nation’s highest court,” said Center for Education Reform President Jeanne Allen.

Allen added: “I urge Governor Jindal to file an appeal to the US Supreme Court, and ask for the justices’ immediate review of the decision. The Louisiana justices actions today violate the civil rights of parents and children who above all are entitled to an education that our Founders repeated time and time again is the key to a free, productive democracy.”

Allen’s response, in her view, is based on her understanding (or lack thereof) of the 2002 U.S. Supreme Court decision in Zelman v. Simmons Harris, which involved an establishment clause challenge to the Cleveland Ohio Voucher program.

So, what does that mean? Well, Ohio had adopted a voucher program that permitted children in Cleveland to apply their taxpayer funds to attend either a public or private school, including religious schools.  Because religious schools dominated the marketplace of alternatives in Cleveland, most voucher users/recipients applied their vouchers to religious schools. Taxpayers sued over this use of their funds – claiming that their tax dollars were being used – against their conscience – to support religious objectives/promoting or advancing religion. So, here’s a quick summary of what the court found/decided in that case:

In a 5-4 opinion delivered by Chief Justice William H. Rehnquist, the Court held that the program does not violate the Establishment Clause. The Court reasoned that, because Ohio’s program is part of Ohio’s general undertaking to provide educational opportunities to children, government aid reaches religious institutions only by way of the deliberate choices of numerous individual recipients and the incidental advancement of a religious mission, or any perceived endorsement, is reasonably attributable to the individual aid recipients not the government. Chief Justice Rehnquist wrote that the “Ohio program is entirely neutral with respect to religion. It provides benefits directly to a wide spectrum of individuals, defined only by financial need and residence in a particular school district. It permits such individuals to exercise genuine choice among options public and private, secular and religious. The program is therefore a program of true private choice.” http://www.oyez.org/cases/2000-2009/2001/2001_00_1751

So then, what are the policy implications? Well, this finding really just means that the establishment clause of the first amendment of the U.S. Constitution DOES NOT PROHIBIT a voucher scheme like that adopted in Ohio.

That, by no stretch of reformy logic (otherwise known as the imagination), means that the U.S. Constitution MANDATES that states must permit such voucher schemes.

It may come as a surprise to some reformers that states have their own constitutions. Indeed, states cannot adopt a constitution which attempts to deprive rights guaranteed under the U.S. Constitution. Civics 101!

Notably, the U.S. Constitution does not include a right to a voucher for private schooling. Rather, the right in question in Zelman was whether the taxpayers’ rights (under the first amendment) were being violated by the voucher scheme.

State Constitutions may establish additional rights which then apply within their boundaries and some of those rights might lead to a declaration of a voucher program being unconstitutional (in state court). Among other things, many state constitutions prohibit the use of public funds for religious entities, including schools (expanding the protection of the establishment clause). These provisions may be interpreted as prohibiting a voucher program which includes religious schooling.

Many states also have provisions that require that the state provide for a “uniform system of PUBLIC schools,” a phrase that has been interpreted in some states as meaning that the state legislature must, in fact, provide education funding – toward a goal of uniformity – through a “system of public schools,” which would mean that doing so through private schooling might not comply.

Some reformers might say – pshaw… that’s just liberal activist courts reading into their constitutions stuff that’s really not there. How can one possibly interpret the phrase “the state shall provide for a uniform system of public schools” as meaning that the state shall actually provide for a system of schools that is both uniform and public? (note that there exist important delineations in legal/governance terms between “public” schools, voucher receiving private schools, and charter schools).

It is indeed permissible for state constitutions to include such requirements. And quite honestly, it is ridiculous to assume that Zelman, which merely declares that vouchers don’t violate the establishment clause of the U.S. Constitution, somehow negates these provisions of state constitutions.

How ridiculous is this logic? Let’s apply it to another well known U.S. Supreme Court case on education. In 1973, the U.S. Supreme Court found (as a step toward their final ruling related to funding inequities) in San Antonio ISD v. Rodriguez that education was not a fundamental right under the U.S. Constitution. It’s just not there. There’s no education article/amendment in the U.S. Constitution.

But state constitutions make reference to the state’s obligation toward providing schools/education, etc. These are education articles/clauses. And in many states, high courts have determined that these education articles in fact provide a fundamental right – under the state constitution – to some form of education (uniform system, thorough and efficient system, sound basic education, etc.). But ooohhhh no… not by Jeanne Allen’s logic. How dare state’s grant a right to education? or obligate their legislatures to provide a system of public schools?

Applying Jeanne Allen’s logic, if the U.S. Constitution does not grant this right, then states should not be permitted to do so either (which doesn’t mean states can’t provide schools, they just can’t guarantee a right to them? which I guess might suit the CER agenda…). In other words, states should absolutely not be permitted to guarantee a fundamental right to an education, because the Rodriquez court said that the U.S. Constitution included no such right.

No rights, no problem. After all, it’s the reformy way.

I was reviewing the past few days of news coverage on NJ teacher evaluations and came across the following quote, which was not-so-amazingly left unchallenged:

Cerf said research shows test scores are “far and away” the best gauge of teacher effectiveness, and to not use test score data would be “very anti-child.”

http://www.nj.com/news/index.ssf/2013/05/state_board_of_education_adjus.html

Here’s a reporters’ guide to follow up questions….

Mr. Cerf… can you show me exactly what research comes to that conclusion? (this should always be the immediate follow up to the ambiguous “research shows” comment)

Exactly how is “far and away” measured in that research?

And what is meant by “best gauge of effectiveness?”

That is, what is the valid measure of effectiveness against which test scores are gauged? (answer… uh… test scores themselves)

So, Mr. Cerf, are you trying to tell me that the Gates MET study proved that test scores are “far and away” the best gauge of teacher effectiveness? (seems to be most common reference point of late)

Can you show me where they said that?

And how did they measure what was the best predictor of effectiveness? In other words… what did they use as the true measure of effectiveness?

So… you’re telling me that the Gates study found – not far and away, mind you – that test score based measures are, well… the best predictor of themselves a year later? Is that right?

That’s what you mean by best gauge? Right? That they are the best predictor of themselves… if we also use other measures to try to predict test scores? Right? Seems a bit circular doesn’t it?

And how well did test-score based measures predict themselves a year later? if we accept as logical that validity test?

Well, that seems like a rather modest relationship from year to year, doesn’t it?

How does that make test scores the best predictor of actual effectiveness if actual effectiveness is broader than test scores themselves?

Okay… moving on… Since we’re leaning on those Gates foundation findings as providing the basis for placing heavy weight on test scores in NJ teacher evaluation… I note here (pointing to NJDOE documents on SGPs) that New Jersey has chosen an approach called Growth Percentiles to measure teacher effectiveness…. Can you show me in the Gates studies where the authors find this approach to be appropriate – or even more specifically “far and away” the best approach for measuring teacher effectiveness?

I don’t see any reference to SGPs or MGPs in the Gates studies… why is that?

Are SGPs and VAMs the same thing? I’ve been told there are substantive differences.

Isn’t one of these, SGPs, not even designed to isolate the effect the teacher has on test score gains?

In which case, how can they possibly be “the best gauge of teacher effectiveness?”

Let’s save the “very anti-child” stuff for another day!

The following is a hypothetical case I am using as the culminating activity in Public School Law this semester.

The Dismissal of Principal X

Principal X is principal in a local public middle school in a state that has recently adopted through legislation, articulated with greater precision in state department of education regulations, a new teacher evaluation scheme. The teacher evaluation laws and regulations now require that:

• Any teacher who receives two sequential evaluations less than “satisfactory” shall have his/her tenure status revoked;
• Teacher evaluations shall consist of 40 to 50% measures of student growth, where the majority shall be based on state provided metrics.
• By regulatory decree of the State Commissioner of Education, any other measures selected by local district officials for inclusion in evaluations must be proven correlated with state approved and provided measures of student achievement growth.

Further, the state now conditions receipt of “any and all increases to state aid for local public school districts” on full compliance with statutes and regulations pertaining to teacher evaluation.

On September 20^th of 2013, Principal X was provided with growth percentile data on her teachers from the prior year. Of the approximately 40 certified staff in her school, 8 received growth percentile data, two of whom achieved unsatisfactory growth percentile estimates for their students, one of whom received a second unsatisfactory rating in a row ‐Teacher Y.

In keeping with the requirement that any and all other measures used in the state approved teacher evaluations be correlated with the growth percentile measures, the principal was compelled to assign this teacher a second unsatisfactory rating, and thus compelled to revoke the tenure status of Teacher Y. She was a 10 year veteran teacher perceived by the principal and many others in the school to be one of the school’s most valuable human resources. In fact, over the past several years, the principal had relied on this teacher to take the difficult students including playing a more significant role than others in inclusion of children with disabilities in her classroom – and the teacher not only willingly, but eagerly complied.

Frustrated with the outcome of the new state teacher evaluation laws, Principal X took her case to the public and to state officials simultaneously. Without specific reference to the case in question – but via stylized example – the principal used the case of Teacher Y to illustrate how strict requirements of job action based largely on limited and problematic measures could lead to damaging decisions – decisions

1| Page

that she argued were neither in the best interest of the teachers nor the children they served, and decisions likely to negatively affect the quality of education statewide.

The principal made the case for returning discretion on issues of teacher evaluation and human resource management to local officials, including school principals. The principal’s letter led to a sympathetic uprising from community members and parents, who were quick to catch on as to which teacher was actually the basis of the principal’s hypothetical. Parents of that teacher’s students were outraged, and expressed their outrage at local board of education meetings. During this time, the local board of education maintained quiet support of the principal.

The principal had also begun to engage other principals statewide establishing a network of principals publicly proclaiming their opposition to newly adopted state teacher evaluation statutes and regulations. A web site was created, a non‐profit organization (political action organization) was formed, and the original letter of opposition to new state policies posted on the site, along with a petition for other school principals to show support for the group’s cause and/or become an official member.

State officials were less supportive and unamused by this principal’s apparent disrespect for their authority, and her “willful disobedience of existing statutes and regulations” expressed by Principal X’s stalling on submitting relevant evaluation information necessary for revocation of Teacher Y’s tenure status. Further, state officials were less than thrilled with the mounting insurrection initiated by the publicly posted letter to state officials outlining problems with the state teacher evaluation laws.

State officials released a letter to the local board of education indicating that their state aid would be frozen for the coming school year if, in fact, their rogue principal continued to stall and refuse compliance with the teacher evaluation laws. Under pressure from the board, Principal X agreed to initiate procedures that would lead to tenure revocation for Teacher Y. Instead of waiting out this process, Teacher Y chose to resign and pursue employment elsewhere.

But with mounting pressure on the local board of education from state department officials to control the growing movement among principals statewide against the teacher evaluation laws, a movement initiated by one of their most respected principals (who had received only glowing evaluations in prior years), the district board chose to dismiss Principal X, citing that the principal’s activities had distracted her from doing the job required, substantively compromised her effectiveness as a principal and significantly interfered with the ability of district officials to efficiently and effectively carry on district operations (including the uncertainty created over the district’s future state aid receipts).

Principal X is now suing the district for wrongful dismissal, arguing that the district’s dismissal is in violation of her first amendment right to express herself to the public on issues of public interest, for which she, as an informed public school employee has relevant information.

2| Page

Required Reading

Key Cases

Pickering v. Board of Education: http://www.oyez.org/cases/1960‐1969/1967/1967_510

Connick v. Myers: http://www.oyez.org/cases/1980‐1989/1982/1982_81_1251

Garcetti v. Ceballos: http://www.oyez.org/cases/2000‐2009/2005/2005_04_473

Blogs

EdJurist: Garcetti & Schools http://www.edjurist.com/garcetti‐and‐schools

EdJurist: Academic Blogging & Garcetti: http://www.edjurist.com/blog/2008/5/9/academic‐freedom­garcetti‐blogging.html

Law Reviews

Oluwole, J. O. (2007). On the Road to Garcetti: Unpick’erring Pickering and Its Progeny. Cap. UL Rev., 36, 967.

3| Page

Behavioral economics is an interesting and potentially useful field of academic inquiry. At its best, real behavioral economics attempts to address some of the concerns I raise here. But many if not most assumptions about human behavior and response to incentives are not representative of behavioral economics at its best.

Specifically,  I’m increasingly concerned with what I see as the simple-minded projection of economic thinking onto everyone and anyone else, leading to ridiculous policy recommendations – that amazingly – get taken seriously – at least by the media and punditocracy.

See, for example, Roland Fryer’s experiment on loss aversion as a strategy for incenting teachers to make sure that their students gain a few extra test score points in limited content areas. Indeed, if we pay you up front, and threaten to take your salary away if you don’t get those test score points out those kids, the data suggest  a greater likelihood of squeezing the kids for a few more points. Whether that tells us anything about the motivation and morals of teachers, or of the economists framing this argument is an entirely different question. This tells us little or nothing of the appropriate policy response. Thankfully, the policy implications of this paper were sufficiently absurd that they gained little traction.

Let’s assume classic economic assumptions about human behavior really hold steadfast and can be grossly simplified to an anything for an extra buck, or not to lose one, position. I would argue that it is perhaps economists themselves that are most stereotypical in this regard –  at least as represented in the thinking the project onto others.  In fact, I would argue that many, born out of a culture that self-selects into economic professions, are simply going out of their way to project their own thinking on others.

Further, many of these economists operate in a world where they can influence/control public policy and they too have an incentive in how they behave in this system. They are not impartial observers by any stretch of the imagination. Their goal is to use their economic research to shape public policy to their own advantage.

Put simply, just because the average morally bankrupt economist might do pretty much anything for an extra buck (or a billion), doesn’t mean the average teacher, doctor, nurse, fireman or police officer would!

This issue has been on my mind for some time, but recently came to a head when I read this completely ridiculous Washington Post article on health care policy – specifically – how to remove the incentive for hospitals and physicians resulting from surgical complications.

I should note, I come from a medical family, so some of my arguments herein are drawn from dinner table conversations (across generations), coupled with my tendency to read health policy research out of personal interest in exploring connections with education policy.

It was implied in the WaPo article… well… actually it was explicitly stated in the article that hospitals and physicians have a big financial incentive for their patients to have serious complications, leading to extended hospital stays and additional procedures.

Now, the average economist might be so morally bankrupt such that if he/she were in an operating room (OR) considering the implications of complications relative to potential earnings that they might intentionally introduce infection or other complication, but thankfully the average economist is not in the OR. Thankfully, they self-selected into economics and not medicine (likely foreseeing greater opportunity to earn more for much less work and upfront investment).

The WaPo article does make the following statement, to head off this argument:

The study does not imply that hospitals intentionally complicate surgeries to bring in more revenue.

But, I would argue that this is actually a rather half-hearted disclaimer (to a half-assed argument) for an article that very much implies just that.

Certainly the economists’ policy response – how to employ crude economic assumptions of human behavior to fix this dreadful perverse incentive – implies that cutting off this financial benefit for malpractice would improve hospital and physician behavior [meanwhile conflating the hospital and physician incentives & roles in the various related processes]. Here is the policy solution recommended by the economists cited in the WaPo article:

If hospitals receive a set amount for every heart surgery they perform, for example, they suddenly have an incentive to reduce complications — they know the extra medical spending will come out of their own budget.

Lost in the economists’ reasoning here are a) the potential longer term financial and career implications to the physician repeatedly entangled in litigation over post-surgical complications, b) and the stress/mental toll on the physician arising from managing complications in tense moments in the OR.

Indeed this is anecdotal, but I’ve not met a physician – surgeon or anesthesiologist – who prefers a day when things go bad in the OR – or would be likely to see dollar signs in those moments of stress. What kind of sick bastard even thinks that way? Well, perhaps the average economist does.

Economists rarely – uh… NEVER face comparable professional stress to managing a patient’s life on the edge – even when they make a massively stupid spreadsheet error stimulating economic turmoil across the globe. Nor do they pay hefty malpractice premiums to shield themselves from such egregious malpractice (despite measurable financial damages). I would assert that the economist never faces the stress of having to care for a classroom of 20 to 40, 5 to 15 year old kids, whose immediate safety and well-being, as well as their long term futures is on the line.  This is in part, why they get away with such ludicrous thinking.

It’s all freakin’ game (Freakin’ used here in a technical freakonomics sense)… a game of playing with big data – several layers removed from reality – from people – from real human consequences.

Perhaps that’s the central issue… even more so than economists’ financial self-interests?

Taken in perspective, it’s a fun game and a pretty cushy lifestyle to have opportunity to ponder policy implications of big data, as long as we don’t start thinking that what we do is so freakin’ important and indispensable and as long as we understand where we sit in this big messy puzzle of human behavior and incentives.

Of course, the other interesting piece here is the leap we often see these days between what the study behind the headlines actually said, and the resulting spin in the media headlines. We also often see the economists themselves engaging in the spin. This was equally true in the famed Chetty, Rockoff, Friedman Fire Teachers First, Ask Questions Later study.

For example, here’s what the original study – in the Journal of the American Medical Association – on reimbursements associated with complications actually said:

Depending on payer mix, many hospitals have the potential for adverse near-term financial consequences for decreasing postsurgical complications.

It takes one hell of a leap of logic to get from this measured finding to the policy recommendation above.  It takes projecting economists thinking –amoral greed – onto all actors involved. It also takes ignoring entirely a multitude of contextual factors and perverse consequences (economist thinking – first, we assume none of that stuff exists). Indeed, many complications relate to preexisting conditions and/or overall health of the incoming patient. Do we really want to incent risk aversion? (avoiding those far more likely to have complications?). Well, if it leads to lower premiums for and taxes paid by economists, then perhaps?

Tangentially (or not?), there is an equally ill-conceived movement afoot to apply to healthcare management the brilliance of what we have supposedly learned from measuring teacher effectiveness with value-added models, as explained in this policy brief from Mathematica. Notably, I tend to think Mathematica does pretty good work on education policy (better than most. See here, but for more critical perspective, see here) Put in its best light, this policy brief is merely Mathematica researchers engaging in another I’ve got a Hammer… where’s the freakin’ nail exercise.

Put in the light of economic thinking about human behavior – which many economists prefer to project on all others, the incentive here is for Mathematica to broaden its market, gaining contracts to develop value-added metrics for health care systems and for state and Federal government – to ultimately be used in reducing payments for healthcare, and reducing the tax burden and healthcare premiums paid by Mathematica researchers – their funders and their peers. It’s a win/win. More contracts and higher income, and lower taxes and health benefits expenses (not costs, but expenses*).

That is, as long as they are never in need of surgery.

====

*Cost  reduction implies that quality of service remains constant, whereas expenditure reduction may lead to service quality reduction.

Two buzz phrases have been somewhat quietly floating around reformyland of late, for at least a year or so. I suspect that many have not even picked up on these buzz phrases/words.  They are somewhat inner circle concepts in reformyland. The first is the notion of the great relinquisher (a seemingly bizarre contradiction indeed… to be great at surrendering… but I believe that’s the point). The second is the idea that we all must learn to be sector agnostics. That is, we all must stand behind the provision of a system of great schools as logical replacement for existing school systems and that this system of great schools might be provided by any sector – public/government, charter, private non-profit, private for profit. After all, it doesn’t matter how we provide them, as long as they are great schools. Who can argue with that?

Linking these two conceptions, the great relinquishers – primarily public officials perceived as otherwise self-interested bureaucrats – must learn to relinquish their self-interested stronghold on publicly financed schooling to alternative providers.  Among inner circle reformers, these ideas are treated as somehow ground breaking, deep intellectual thoughts about re-envisioning schooling. But in reality, they are anything but.

On Relinquishers & Sector Agnosticism

Some abbreviated backdrop on the relinquisher notion.  I converse (constructively) on occasion via e-mail with Neerav Kingsland who promotes this particular notion. For those who don’t know Neerav, he’s a Yale Law grad who completed a Broad Residency, and is currently CEO for New Schools for New Orleans. Thus, as I interpret it, he derives his core arguments largely on his perception of the (highly debatable) successes of post-Katrina New Orleans.  That in mind, and with all due respect to Neerav, I have grave concerns about what he refers to as the movement toward “relinquishment” or creating a culture of “relinquishers” among current public officials regarding the provision of the public good of schooling (differing substantively from public schooling.)

Neerav introduced the concept of Relinquishers in a letter he wrote to urban (not all, just “urban”) superintendents in Education Week:

Before I begin in full, let me say this: Superintendents, over the years I’ve begun to believe that your identities–how each of you perceives your professional charge–are often misguided. In my experience, most of you view yourselves as system reformers–leaders who can make the current educational system much better. For the sake of the letter, let’s call you, well, Reformers. With great diligence, you fight to make our government-operated system better.

But let me suggest another identity–one whose charge is to return power, in a thoughtful manner, back to parents and educators. Let’s call these types of superintendents Relinquishers. With great diligence, these superintendents attempt to transfer power away from a centralized bureaucracy.

Both Reformers and Relinquishers possess noble aims, but only one group, I think, possesses a sound strategy.

Superintendents, in the rest of this letter I hope to convince you to become Relinquishers. Specifically, I will advocate that you return power to parents and educators through the creation of charter school districts, which are the most politically acceptable mechanisms for empowering educators. (my emphasis)

Let’s start by taking the word “relinquish” literally for a moment. A quick synonym search in Microsoft Word yields: Surrender, Abandon, Renounce, Resign

The implication here is that public officials must “surrender” or “abandon” or “renounce” their schools, handing them over largely to private managers of charter schools (note that Neerav Kingsland has suggested that charter operators are the “politically acceptable” choice, leaving for others including Smarick to consider conventional private schooling and voucher models). Yeah… I get that this is an interesting notion – to suggest that there is some nobility is declaring defeat and handing control over to those who might be able to play a positive role. I get that. But I find this use, and this framing rather disturbing.

This is not to suggest that I don’t believe that many local public school districts, large or small, need work (some, a hell of a lot of work) on how they interact with their local communities and how they balance stakeholder interests (responsiveness to parents/students, etc.). That’s an ongoing concern in any public or private sector business, with differing structural/governance issues involved in public governance. This is also not to suggest that public officials should never look to other sectors for appropriately contracted, sufficiently regulated support. But “relinquishment” is an extreme perversion of this notion, especially when we start considering relinquishment of the system as a whole – Surrendering, abandoning, renouncing any and all role for public governance and centralized public policy.

Now for this notion of “sector agnosticism” – In his book The Urban School System of the Future and in several tweets and blog posts, former deputy commissioner of Education of New Jersey, Andrew Smarick promotes the reformy religion of what he refers to as Sector Agnosticism. A brief explanation is provided in a recent education week post:

Smarick: “Second, we need to have a three-sector accountability system that treats similarly district public schools, charter public schools, and private schools; we must focus on school results, not school operator. I call this “sector agnosticism;” in other words, we shouldn’t care who runs a school as long as it is superb.”

In the 1990s, when this idea arguably first gained some momentum (summarized in Paul Hill’s book Reinventing Public Education), I was actually a pretty big fan of the idea – which consisted primarily of finding ways to employ private contractors through performance contracting to improve urban schools. Heck, my own first conference paper ever was on the issue of private management of public schools, at a time when I thought there might be great hope for such strategies. Unfortunately, the self-interest of the (publicly traded, for profit) private manager (who eventually fell into financial collapse) to extract as much revenue as possible from the urban district (Baltimore) coupled with their outright disinterest in, and obstruction of having their outcomes measured, started giving me doubts. How could they possible show an efficiency advantage (doing more with less) if they managed to game their budget allocations to their advantage and then wouldn’t provide evidence of results?

Unfortunately, I wrongly assumed things would get better as the industry evolved. Further, over time, as I completed graduate work studying education finance and policy and became reasonably well versed in school law and education governance (teaching it at the graduate level for over a decade & writing/publishing numerous co-authored articles in law review journals) I became more acutely aware of the potential pitfalls of taking an uniformed leap into sector agnosticism.

Defining superbitude?

First, let’s take Smarick’s sound bite notion that it should matter as long as the school is “superb.” Even with a narrow, test score or graduation & post-secondary matriculation-based measure of “superbitude,” neither charter nor private schools are revealing any decisive edge, holding student characteristics or access to resources constant. Rather, as one might logically expect, these less regulated sectors merely produce greater variation around largely the same mean (if comparing similar students). Across sectors, the drivers of outcome variation continue to be the substantive differences in student populations served and oft correlated variations in access to schooling and non-schooling resources (in public schools, charter schools or private schools).[1]

Why do those KIPP charter middle schools appear to perform so well? What about New York City or Newark charter schools more broadly? And what about years of findings on private schools, or students participating in the New York City private school voucher experiment? It’s not about sectors, but rather about strategies and resources. And if it’s about strategies and resources, then if we can identify what works and the resources needed to legitimately serve all children, we can provide those opportunities within a publicly governed, publicly accountable system of common schools. Indeed, if these measured outcomes were in fact the only issue of concern, we might leverage an appropriately mixed set of schooling providers to get the job done. In fact, the lack of decisive advantage by sector alone is equal justification for agnosticism as it is against it.

But, that’s only if we ignore entirely that there might actually be other tradeoffs involved, beyond whatever test score, graduation, matriculation or employment outcome might be achieved.

Trading Off Legal Rights for Test Scores?

It’s not just about figuring out how to achieve crudely measured “superberific” schooling.  Our children’s schooling exists in a broader social, political and legal context. Kids have legal rights, and under most state constitutions kids a right to access/participate in/gain the benefits of a system of schooling (sometimes, quite explicitly, a system of public schooling). In many states, they not only have a right to access schooling (at times, of some measured degree of quality), but a legal obligation to attend up to a specified age (compulsory schooling laws).

As I’ve discussed on a few previous blog posts, privately governed and/or managed charter schools, more like traditional private than like public schools, may not be (are likely not) subject to the full protection of students’ constitutional or statutory rights (summary table from previous post included below). When attending a private school, it’s clear that kids have no right to continued attendance. They can be expelled, excluded outright for any number of reasons (including admissions testing). They may be compelled to recite school oaths and may be obligated to participate in religious activities and may be restricted in their ability to freely express themselves and subject to disciplinary action including expulsion for failure to comply. Parents may also be obligated to participate in certain activities as a condition of continued enrollment.

While charter advocates love to declare their schools as necessarily “public,” with regard to at least some of these same issues/questions, Charter school legal defense attorneys are quick to argue that they are in fact, private. That, for example, children’s rights under disciplinary codes should be treated as private contracts entered into by parents, just as in private schools – and substantively different from “public” schools – or those formally governed and operated by agents of the state (local elected school boards and public district administrators).

Further, the public-private delineation and murky middle ground of charter schooling raises numerous additional substantive legal questions regarding public employment law and employee rights, taxpayer and citizen rights to open public meetings and public records, and rights, responsibilities, liabilities and protections of “public officials” such as school board members and public employees as opposed to governing boards of private citizens, and employees of private contractors.

Sector agnosticism, as dreadfully simplified by Andrew Smarick requires completely ignoring these substantive tradeoffs.  Trading off constitutional rights to reduce supply of some and increase access to other sectors is not benign, if those sectors could/might possibly yield other advantages.

The Distribution of Lost Rights

Nor do I suspect that the tradeoff of rights will ever be randomly distributed across children by the wealth and income of their families. No-one is asking the superintendent of Scarsdale (great guy, by the way) to Relinquish his schools and adopt a policy of Sector Agnosticism. This is a policy for the children of New Orleans, New York City, Chicago, Philadelphia and Newark.

In the extreme case – a case favored by Smarick and seemingly endorsed (through relinquishment) by Kingsland – a district – or now merely a geographic space – where children have only access to privately governed/managed charter schools may require that any/all that wish to actually exercise their state constitutional right to attend school, have to choose which rights to forgo in the process? Will 100% of parents in that zone be required to enter into contractual agreements (forgoing constitutional & statutory protections) with schools regarding disciplinary policies for their children?

In fact, Kingsland’s logic is that district superintendents should simply succumb or surrender to the forces that wish to forcibly close and takeover their schools, and relinquish those schools or at least the children who would have attended them, to other sectors. Following Smarick’s logic, parents and citizens at large should completely ignore tradeoffs of constitutional protections, or humiliating treatment of children, in lieu of Smarickian measures of “superbification.”

Creating a scenario where only low income minority children in America’s cities must tradeoff their constitutional and statutory protections to gain access to schooling (which they may be compelled to attend) is clearly unacceptable, inequitable treatment.  Before you go there… no… I’m not saying that the responsible policy solution is to make sure that suburban kids and their parents are equally deprived of protections.  What’s not good for some is not good for all.

One logical retort to my arguments here is that if parents want these choices, if they are backed up on waiting lists for existing charters, then we should provide to them. If the demand is there, let the supply meet the demand!? It would be one thing if it was made clear, up front, to potential choosers these hidden tradeoffs, but that’s not the case. If anything, charter advocates are doing their best to conceal that any such tradeoffs exist.

Indeed, appropriate cross-sector regulation might negate some of my concerns raised here, but these issues are too frequently ignored.

Market Manipulation & The Forcible Reduction of the “Public Option”

Worse, in the current policy context, we are not witnessing the emergence of a true, fair and equitable, demand driven and fully open and accessible (driven by open information) system of choice. Policies of relinquishment and sector agnosticism are being pursued in practice as policies of forced relinquishment (read mass closings) of traditional public schooling and sector favoring transfer of assets (public to privately governed charters), coupled with gross misrepresentations of information on sector quality.

In selected cases, we are also witnessing a coordinated effort to provide competitive advantage for non-district alternatives. Where sectors are set up to compete with one another to prove their worth, the likelihood that charter or voucher advocates will lobby for increased resources for district schools is about as likely as the New York Yankee ownership arguing for revenue sharing to help the Kansas City Royals, or Walmart to lobby for tax breaks for Target. Similarly, the likelihood that well endowed charters will share their philanthropy with others less fortunate is slim to none where the emphasis remains on flaunting one’s competitive advantage.

A veneer of demand (as measured by duplicative waiting lists) for private and charter sectors has been induced by forcible reduction of supply of urban schooling, and gross misrepresentations & mismeasures (New Jersey/ New York) of neighborhood schooling quality and manipulation of the playing field.

Closing Thoughts

Before we jump on these reformy bandwagons, and start waiving the white flag of relinquishment and promoting the virtues of sector agnosticism, we need to take a hard look at how this is playing out in our cities. Numerous New Orleans schools were wiped out by a natural disaster, displacing large shares of the lowest income residents to Houston (and elsewhere), many of whom have not been able to return in part because the market based model of New Orleans has chosen not to serve their former, blighted neighborhoods.  This was tragic, and the initial occurrence largely beyond policymakers’ control.  The choice to leave children and their families unserved or underserved was a conscious policy decision (or a least a predictable result of the policy response).

Proposed Chicago (and Philadelphia) school closings would appear comparably poised to induce increased demand for charters, which will likely be used as rationale for expanding charters even further and advancing the cycle toward its ultimate end (as if Katrina by design, and more surgically targeted at schools with low test scores and poor minority children). In most U.S. cities, however charter market shares remain modest, and publicly subsidized private school enrollment even smaller, providing an opportunity to pause and rethink current strategies.

So then, what do we do about all of this? First, reformers and non-reformers alike (and anti-reformers too!) need to step back from these oversimplified talking points and buzz phrases which so illustrate the worst of intellectually lazy, undisciplined, under-informed policy development. I don’t mean to be a hypercritical, ivory tower (actually, public university 1960s era building basement) academic … okay… yeah… that is what I mean to be here. Why? Because it matters! Exploring and understanding these tradeoffs matters. Ignoring them is reckless.

Yesterday, Matt Di          Carlo over at Shankerblog put out his April fools post. The genius of the post is in its subtlety.  Matt put together a few graphs of longitudinal NAEP data showing that Maryland had made greater than average national gains on NAEP and then asserted that these gains must therefore be a function of some policy conditions that exist in Maryland. In the Post-RTTT era, Maryland has been the scorn of “reformers” because it just won’t get on board with large scale vouchers and charter expansion and has resisted follow through on test-score based teacher evaluation. Taking a poke a reformy logic, Matt asserted that perhaps the low charter share and lack of emphasis on test score based teacher evaluation… along with a dose of decent funding might be the cause of Maryland’s miracle!

Of course, these assertions are no more a stretch than commonly touted miracles in Texas in the 1990s, Florida or Washington DC, most of which are derived from making loose connections between NAEP trend data and selective discussion of preferred policies that may have concurrently existed.  The difference is that Matt was poking fun at the idea of making bold, decisive, causal inferences from such data. Such data raise interesting questions.

What I found so fun and at the same time deeply disturbing about Matt’s post is that the assertions he made in satire… were nowhere near as absurd as many of the assertions made in studies/reports, etc. I discussed here on my blog over the years. Here are but a few examples of “stuff” presented as serious/legit policy evidence, that make Matt’s satirical assertions seem completely reasonable.

The Many Variations of Money Doesn’t Matter Graphs:

I start with this one, because there are so many versions of it floating around out there, that come and go over time, and are often used to advance the “money doesn’t matter”… we’ve spent ourselves into bankruptcy and gotten nothing for it… graph. Every good reformer has a laminated copy of one version or another of this graph which they carry in wallet-size.

I blogged about this graph when Bill Gates used it in a HuffPo article.

Gates asserted:

 Over the last four decades, the per-student cost of running our K-12 schools has more than doubled, while our student achievement has remained flat, and other countries have raced ahead. The same pattern holds for higher education. Spending has climbed, but our percentage of college graduates has dropped compared to other countries… For more than 30 years, spending has risen while performance stayed flat. Now we need to raise performance without spending a lot more.

Among other things, the chart includes no international comparison, which becomes the centerpiece of the policy argument. Beyond that, the chart provides no real evidence of a lack of connection between spending and outcomes across districts within U.S. States.  Instead, the chart juxtaposes completely different measures on completely different scales to make it look like one number is rising dramatically while  the others are staying flat. This tells us NOTHING. It’s just embarrassing. Simply from a graphing standpoint, a blogger at Junk Charts noted:

Using double axes earns justified heckles but using two gridlines is a scandal!  A scatter plot is the default for this type of data. (See next section for why this particular set of data is not informative anyway.)

Not much else to say about that one. Again, had I used an example this absurd to represent reformy research and thinking, I’ d have likely faced stern criticism for mis-characterizing the rigor of reformy research!

This alternate version comes to us from none other than Andrew Coulson of Cato Institute. Coulson has a stellar record of this kind of stuff. So, what would you do to the Gates graph above if you really wanted to make your case that spending has risen dramatically and we’ve gotten no outcome improvement? First, use total rather than per pupil spending (and call it “cost”) and then stretch the scale on the vertical axis for the spending data to make it look even steeper. And then express the achievement data in percent change terms because NAEP scale scores are in the 215 to 220 range for 4th grade reading, for example, but are scaled such that even small point gains may be important/relevant but won’t even show as a blip if expressed as a percent over the base year.

Chris Cerf’s Poverty Doesn’t Matter Graph!

Now, it’s one thing when and under-informed tech CEO goes all TED-style on us with big screens, gadgets, bells and whistles and info-graphics that just don’t mean crap anyway. But, it’s yet another when a State Commissioner of Education presents something not only equally ridiculous… but arguably far more ridiculous, disingenuous, unethical and downright WRONG.

This is a graph for the ages, and it comes from a presentation by the New Jersey Commissioner of Education given at the NJASA Commissioner’s Convocation in Jackson, NJ on Feb 29. State of NJ Schools presentation 2-29-2012

The title conveys the intended point of the graph – that if you look hard enough across New Jersey – you can find not only some, but MANY higher poverty schools that perform better than lower poverty schools.

This is a bizarre graph to say the least. It’s set up as a scatter plot of proficiency rates with respect to free/reduced lunch rates, but then it only includes those schools/dots that fall in these otherwise unlikely positions. At least put the others there faintly in the background, so we can see where these fit into the overall pattern. The suggestion here is that there is not pattern.

Note: this graph may not even be the worst one in the presentation. You decide!

The apparent inference here? Either poverty itself really isn’t that important a factor in determining student success rates on state assessments, or, alternatively, free and reduced lunch simply isn’t a very good measure of poverty even if poverty is a good predictor. Either way, something’s clearly amiss if we have so many higher poverty schools outperforming lower poverty ones. In fact, the only dots included in the graph are high poverty districts outperforming lower poverty ones. There can’t be much of a pattern between these two variables at all, can there? If anything, the trendline must be sloped uphill? (that is, higher poverty leads to higher outcomes!)

Note that the graph doesn’t even tell us which or how many dots/schools are in each group and/or what percent of all schools these represent. Are they the norm? or the outliers?

Well, here’s what the pattern really looks like with all schools included:

Hmmm… looks a little different when you put it that way. Yeah, it’s a scatter, not a perfectly straight line of dots. And yes, there are some dots to the right hand side that land above the 65 line and some dots to the left that land below it.

Note: New Jersey’s Chris Cerf is not alone among state commissioners in promoting completely bogus analysis posing as empirical validation. In fact, New York’s John King presented a completely fabricated graph provided to him by a consultant to the state and has used that graph to frame his state’s policy initiatives.

Rishawn Biddle’s Graph of, well, something? What?

Not to be outdone, Rishawn Biddle who on occasion fashions himself a “researcher” on education policy issues, provides a graph that comes close to the degrees of intentional deception presented by Commissioner Cerf above.  I blogged about this graph here!

In response to arguments I had made on my blog regarding the role of substantive and sustained school finance reforms in improving school quality, Biddle argued:

Despite the arguments (and the pretty charts) of such defenders as Rutgers’ Bruce Baker, there is no evidence that spending more on American public education will lead to better results for children.

My claims are substantiated in this peer reviewed article and this separate more comprehensive report:

• Baker, B. D., & Welner, K. G. (2011). School Finance and Courts: Does Reform Matter, and How Can We Tell?. Teachers College Record, 113(11), 2374-2414.
• Baker, B. D. (2012). Revisiting the Age-Old Question: Does Money Matter in Education?. Albert Shanker Institute. http://www.shankerinstitute.org/images/doesmoneymatter_final.pdf

And what does Biddle provide as counter evidence to this – apparent lack of evidence I summarize above (I’ve sent the article link to Biddle on more than one occasion, but he apparently doesn’t read this kind of academic stuff)?

Biddle counters with a link to this graph – a true gem (I’ve added some annotation, not in his original)!

Yes, Biddle’s counter to the body of research he has not and likely will never read, is to use this graph of “promoting power” by student race group for Jersey City, NJ in 2004 and 2009. Note that the infusion of additional funds in NJ occurred mainly from 1998 to 2003, leveling off thereafter. But that’s a tangential point (not really).  So, Biddle’s absolute verification that more money doesn’t matter is to simply assert without verification that Jersey City got a whole lot more money and then to use this graph to argue that nothing improved!

First of all, that analysis wouldn’t pass muster in as a master’s degree level assignment (I teach a class on this stuff at that level), no less major research conclusions. From a graphing standpoint, I often criticize my students’ work for what I refer to as gratuitous use of 3d – especially where the use of 3d bars actually obscures the comparisons by making it hard to see where they align on the axis.

But, the really funny if not warped part of this graph is that there appear to be significant gains for black males between 2004 and 2009, but those gains are obscured by hiding the 2009 black male score behind the 2004 black female score.

Note that the graph also contains no information regarding the actual shares of the student population that fall into each group? Not very useful. Pretty damn amateur. Certainly fails to make any particular point, and certainly doesn’t refute the various citations above – all of which employ more rigorous analytic methods, apply to more than a single district, and most of which appear in rigorous peer reviewed journals.

Reason Foundation’s Today’s Policies Affected Yesterday’s Outcomes Study!

Finally, in my years as a reviewer for the National Education Policy Center’s Think Tank Review Project I’ve reviewed a lot of sketchy stuff. Some of it stands out, and has even won Bunkum awards from NEPC.

For example, a recent report from ConnCAN repeatedly footnoted a claim as being substantiated to earlier reports…only to result in a dead end where the claim was never substantiated… and in fact, when checking the data turned out to be patently false!  So, this one isn’t even a subtle data interpretation issue. It’s just a lie.

Then there was a report by the organization Third Way, which gathered numerous sources of incompatible data, across incompatible time frames (along with many other bizarre claims) in order to make the argument that America’s middle class schools are failing miserably.

Either of these reports make Matt’s assertions in his post on the Maryland Miracle look totally reasonable!

But for me, the winner among all of the think tank reports I’ve read comes from the Reason Foundation in their 2009 Weighted Student Funding Yearbook! Here’s the abstract of my review:

The new Weighted Student Formula Yearbook 2009 from the Reason Foundation provides a simple framework for touting the successes of states and urban school districts that grant greater fiscal autonomy to schools. The report defines the Weighted Student Formula (WSF) reform extremely broadly, presenting a variety of reforms under the WSF umbrella. Accordingly, when the report concludes that WSF is successful and should be widely replicated, it is difficult to sort through the claims and recommendations. Moreover, the approach and recommendations lack critical inquiry, thought, or empirical analysis. Perhaps most disturbing is the fact that in a third of the specific districts presented in the report, the evidence of success provided predates the implementation of the reforms, and the Reason press release makes the outright claim that past improvements are somehow a function of yet-to-be-implemented reforms. While the report does provide some reasonable recommendations, they are overshadowed by others. Overall, the policy guidance provided by the Reason report is reckless and irresponsible.

Yes… you read it correctly…. If you go through the smashing successes claimed by Reason in this report, in 1/3 of the cases, the reforms in question were implemented after the window of test scores discussed! Hence, the Bunkum time machine award!

Matt’s satirical example didn’t go anywhere near this far.

In Closing….

In my view, there are at least two lessons from Matt’s post, for either side of the reformy aisle.

First, as I so often point out in my classes on applied data analysis, we need to always take  time to carefully evaluate what our data – whatever data and whatever measures – can and cannot tell us. The latter is key here. Descriptive data can be very useful… as long as we understand what they can and cannot tell us. For that matter, various types of inferential statistical analyses (regression models) can also be useful (and in policy research are often primarily descriptive), but often don’t tell us what we think or would like them to tell us. I’ll likely write more about this topic in the future.

Second, we all should take time to carefully scrutinize the link between empirical evidence and policy assertions (and many should take time to take some legit graduate level research methods and statistics and measurement courses on these topics if they wish to continue to opine so boldly about policy inferences!). Perhaps most importantly we should actually take more time and put more effort into scrutinizing those reports and claims that appear most agreeable to our own predisposed beliefs/opinions.  Everyone has predisposed beliefs (especially those who pretend not to). I would argue that experienced researchers likely have stronger beliefs and opinions… and we should… precisely as a result of years of experience researching specific topics.

Oh… and a third lesson… Don’t make completely BS, false/fabricated/absurd graphs like those above. That’s just ridiculous. Are you kidding me? Hiding 3d bars? (Rishawn?) Deleting most of the cases that define the trend? (Cerf?) That’s just ridiculous! Infuriating! Sickening!

Okay, it’s not entirely surprising to find mind-boggling ignorance conveyed in the editorial pages of the New York Post. Today’s example comes to us in an Op-Ed written in response to a report released by the Alliance for Quality Education.

Usually, I’d just let it pass. It’s the Post after all. But, for two important reasons I just had to address this one.  First, the editorial was written by a member of the Governor’s Education Reform Commission.  Second, the editorial made use of our School Funding Fairness report to make its most absurd claim. And here is that claim:

Despite all of AQE’s complaints, there is no need to change the way we allocate this money, since the state already directs almost 70 percent of education funding to high-need districts. In fact, School Funding Fairness’s National Report Card gave New York a grade of “A” in its Effort category, putting us among the top five states in that category.

http://www.nypost.com/p/news/opinion/opedcolumnists/ny_schools_money_not_the_problem_qHXdkNNBLssY1Swqj8LfQK

Apparently the authors of this quote have a little difficulty reading and perhaps some problems interpreting relatively simple numbers and letter grades.

Let’s take a look at the graded indicators in our school funding fairness report:

• Effort – This measures differences in state spending for education relative to state fiscal capacity. “Effort” is defined as the ratio of state spending to state per capita gross domestic product (GDP).
• Funding Distribution – This measures the distribution of funding across local districts within a state, relative to student poverty. The measure shows whether a state provides more or less funding to schools based on their poverty concentration, using simulations ranging from 0% to 30% child poverty.

 http://www.schoolfundingfairness.org/National_Report_Card_2012.pdf

The authors use the state’s high grade on “effort” as basis for asserting that New York State has no allocation problem. Hmmm… perhaps the distribution indicator would be a better indicator for whether the state has an allocation problem. After all, even a quick synonym check in MS Word lists distribution as the first synonym for allocation (followed by provision, apportionment, sharing).

New York Received a D on this measure!

And a D is actually rather generous for New York’s distribution/allocation issues. Figure 1 provides the profile for mid-Atlantic states from our funding fairness report.

Figure 1

In other words, in New York State, higher poverty districts have systematically less per pupil state and local revenue than do lower poverty ones. And despite all of the other completely ridiculous a-contextual and otherwise wrong and misleading claims in the Op-ed, high need New York State school districts face significant financial disadvantages relative to their competitive surroundings.

Despite the Op-Ed authors claims, their Governor has done little or nothing to help these districts, and much to harm them (including misguided reforms).

Sean Corcoran of NYU and I dug deeper into New York State’s distribution issues and sources of inequity in a recent report for the Center for American Progress.

First, with updated analysis following the funding fairness methods, we identify New York State as among the least equitable states in the Nation. Here are the numbers:

And here’s a nice colored map (which may resonate with the editorial authors who’s grasp of numbers appears severely limited):

http://www.americanprogress.org/wp-content/uploads/2012/09/StealthInequities.pdf

We explain that a hypothetical, rational, equitable school funding system should look something like this:

But that New York’s system actually looks like this – when not correcting for costs/needs (it’s much, much worse when you do!).

We also then identify and explain the various sources of inequity in New York State’s school finance system, including the allocation of Tax Relief aid disproportionately to wealthier districts, and other adjustments to foundation aid that favor wealthier districts to the detriment of poorer ones. Here are some figures, and descriptions from our report:

Figure 13 puts New York’s School Tax Relief program aid allocations into context. Federal aid to schools is largely designed to improve equity by targeting resources to higher-need, especially higher-poverty, districts. The School Tax Relief pro­gram aid to New York schools tends on average to be slightly less than federal aid. But to the extent that federal aid creates any improvement to the distribu­tion of resources across New York school districts, the state’s School Tax Relief program aid wipes out that improvement entirely. Aid under the program, which is indicated by the darkest blue in Figure 13, is allocated in nearly perfectly inverse proportion to federal aid, such that when the two are stacked on the other, the cumulative effect is that districts receive about the same regardless of their wealth.

 New York’s foundation-aid formula includes a series of “if/then” steps to deter­mine whether a district should receive state aid based on its initial calculation of local fair share or based on an alternative calculation, one of which is the provision of minimum aid of $500 per pupil. Figure 17 shows the pattern of state and local sharing that would occur if foundation aid were based solely on the income-wealth index estimated by the state. Under that index, the lowest-wealth districts would receive about $12,000 to $14,000 in aid per pupil, and districts with an income-wealth index greater than 1 would receive no aid. After including the various alter­native calculations, districts with an income-wealth index above about 2.5 would receive the minimum of $500 per pupil, while districts with index rates from 1 to 2.5 would receive a sliding scale toward the minimum rather than either the mini­mum or $0. The adjusted version is shown in red. Note however, that neither was fully funded in recent years. (Recent reality is achieved by taking the red squares and shifting them downward but preserving the minimum aid.)

If fully funded, the cost of retaining the minimum aid provision tops $1.2 billion, and the cost of preserving the diagonal, sliding-scale adjustment between the income-wealth index of 1 and 2.5 is $2.47 billion (if we exclude the disproportion­ate effects of New York City). That’s real money—money that could be perhaps targeted toward higher-need districts to reduce the overall regressive nature of New York’s finance system.

The cumulative effects of these adjustments and the School Tax Relief program on the distribution of resources across New York school districts is shown above. The left hand portion of Figure 18 shows local revenue with formula aid prior to the adjustments in Figure 17. Even this isn’t a very pretty picture because state aid remains insufficient to provide even nominal funding equity from lower- to higher-poverty districts. But the right hand side of Figure 18 shows the effect of the adjustments in Figure 17, with the icing of school tax relief aid on top.

Prior to foundation-sharing adjustments and the School Tax Relief program aid, the per-pupil difference in state and local revenue per pupil between the lowest- and highest-poverty quintile is about $1,100. After the adjustments, the per-pupil difference is more than $2,300. New York makes adjustments to its aid formula and throws on tax relief funding in a pattern that more than doubles the nominal inequity between the state’s lowest- and highest-poverty districts.

In other words – YES – NEW YORK DOES NEED TO CHANGE THE WAY IT ALLOCATES MONEY. Any suggestion to the contrary displays a mind-boggling degree of ignorance!