Should Think Tanks be Licensed to Think? (and when should a license be revoked?)

We all have our flaws. As social science researchers, writers, reviewers, etc. dealing with highly political and politicized topics, we all let our personal biases creep inappropriately into our work at times. In the world of education policy writing and research, there are many occasions where individuals and organizations are provided opportunities to selectively review and present data and summaries of existing literature in order to make one point or another or argue one side of an issue or another. There will always be reasonable differences of opinion on complex social science issues, both with regard to the methods that should be used to provide definitive answers, and the definiteness of the answers themselves.

All of that aside, there are, I believe some really basic rules of thumb that should be followed and some offenses that should be considered so egregious that they warrant public rebuke and removal from the table around which public discourse occurs on key education policy topics.

My latest example of an offense so egregious comes from a review I completed today on a report from Lisa Snell of the Reason Foundation – the Weighted Student Formula Yearbook 2009. Reason Foundation is a Think Tank, so-to-speak, or at least fashions itself as such. There are many “Think Tanks” across the political spectrum and these Think Tanks frequently chime in with well-publicized findings regarding education policy and finance issues. So, this is not new. That said, this particular report is simply so awful, I can’t shake it from my head.

To summarize, the report touts urban school reforms which involve decentralized management and school choice, two issues on which I personally do not have strong opinions one way or the other, from an ideological perspective (rather, my concern is that if one pursues a particular reform, it should be done using appropriate evidence, and should be done well – or as well as can be – and not half-baked). The report inappropriately casts the varied reforms presented as Weighted Student Formulas  – a specific reform which involves using a student need focused funding formula to drive financial resources to schools – something which, if done well, could be quite a good idea.

So… I digress… what is so awful then about the Reason report? Why do I think their license to think should be revoked and their seat at the education policy debate table eliminated? In an effort to make their case that the reforms implemented in 14 cities and 1 state have all been highly successful, the authors go so far as to cite evidence of improved student outcomes – in one third of their 15 cases – that occurred in years prior to the implementation of the reforms. That is, the reforms somehow, amazingly, through space-time travel perhaps, had retroactive effects on student outcomes. I’m dumbfounded!

Here’s a section of my review:

=====

Most problematic is the fact that in five of the 15 cases discussed (one third of the cases), outcome successes mentioned actually occurred prior to the implementation of WSF or SBB/SBM (see Appendix A). For example, the report commends NYC for winning the 2007 Broad prize, the year before its Fair Student Funding policy was implemented. The report might arguably attribute this success to mayoral takeover, which began in 2002. But this is the WSF Yearbook, and even the expansive definition of WSF used in the report did not encompass mayoral takeovers.

Similarly, the report commends Hartford for raising test scores in 2008, the year before implementing WSF, and it commends Denver for making strong improvements between 2005 and 2008, whereas WSF was implemented in 2008-09. These successes lead one to question why these districts would want to implement WSF and risk undoing their prior achievements.

The most egregious claim of retroactive causation appears in the press release for the report:

The results from districts using student-based funding are promising. Prior to 2008, less than half of Hartford, Connecticut’s education money made it to the classroom. Now, over 70 percent makes it there. As a result, the district’s schools posted the largest gains, over three times the average increase, on the state’s Mastery Tests in 2007-08 (emphasis added).[i]

Yet, the report itself states that Hartford only began implementing WSF in 2008-09, and only expected to achieve the 70 percent target of available resources allocated to schools and classrooms by 2009-2010 (p. 61). It is difficult to conceive of any defense for Reason’s claims.

=====

Link to complete review: http://www.epicpolicy.org/files/TTR-Baker-Reason-WSF.pdf

I’ve personally come to refer to this offense as the Ouchi Retroactive Causation effect, since the first time I encountered such an absurd argument was in the work of William Ouchi who made much the same analytic gaffe in reference to the success of reforms in Houston. Scott Thomas and I, in our report on Weighted Funding for Hawaii noted: “… most obviously problematic is the authors’ touting of Houston’s success in improving performance through its WSF and decentralized governance, based on performance data from 1999 to 2001, when the formula was implemented (first year of phase-in) in 2001-02” (Baker and Thomas, 2006, p. 9).

http://www.hawaiireporter.com/file.aspx?Guid=1e930a29-3017-40fc-b58c-cf1513149ce7

http://www.hcps.k12.hi.us/STATE/COMM/DOEPRESS.NSF/a1d7af052e94dd120a2561f7000a037c/36dfa2492b206d2c0a2571be000450fd/$FILE/WSF-Baker.Thomas-2-07.28.06.pdf

For a more thorough critique of the work of Ouchi on this topic, see my article with Doug Elmer in Educational Policy (vol. 23 no. 1, 2009)

Look… we all make errors of logic and errors of analysis. So too do relatively safe and conscientious drivers get into an occasional auto accident. However, if a driver drives 15 times in one week, and drives drunk 5 times, that driver should lose his/her license. Claiming retroactive causation 5 times in 15 cases in a report which then proposes “best practices” for schools is comparably irresponsible. Arguably, the first or second offense is bad enough.

If nothing else, the Reason Foundation and the Eli and Edythe Broad Foundation which provided support for the Weighted Student Formula Yearbook should evaluate carefully whether such haphazard and arguably reckless thinking really advances the public policy agenda in any productive way. Whatever good elements there may be in the attempts of these districts to improve educational opportunities for the children they serve are severely undermined by the Weighted Student Funding Yearbook.


[i] http://www.reason.org/news/show/1007460.html

Special Education Funding & Wage Differentials under New Jersey’s SFRA

As the hour of oral arguments in Abbott v. Burke approaches, I suspect there will be some debate as to whether specific elements of the state’s new school finance formula SFRA are, in fact, appropriate. Two components of SFRA that concerned me long before I participated at trial on behalf of plaintiffs were the adjustment created to accomodate  wage variation across New Jersey school districts, and the flat funding – census based approach – used for special education. Quite simply, both are bad policy and are particularly damaging to the state’s highest need children who are clustered in certain school districts – primarily Abbott districts. I will not further editorialize on this point here. But I will post links below to two academic papers on these specific topics which I have presented in recent years.

The first paper was presented at the Annual Meeting of the American Educational Research Association in the Spring of 2008 and involves analysis of wage adjustments in several state school finance formulas, concluding that statistically, New Jersey’s new GCA is among the worst (a finding I did not expect at the outset of the study).

Link to Wage Paper

The second paper, for which an abstract was submitted in Summer 2008, was presented at the 2009 Annual Meeting of the American Educational Research Association, and explores the assumption that children with disabilities are evenly distributed. The paper further explores the equity consequences of distributing funding on this assumption if false.

Link to Special Education Paper

I welcome legitimate critiques of the statistical analyses and policy conclusions in either paper.

On the “Costs” of Private Schooling

Most often, private school costs are addressed in popular media and think tank reports in the context of policy debates over tuition tax credit and voucher proposals. The typical argument is that public school students may be tuitioned out of the public system at lower cost than the operating cost of the public system to private schools which provide greater benefit. In the best of cases, authors of such reports provide reasonable publicly reported estimates of public school expenditures – because such data are readily available – and then choose estimates of private school “costs” from a handful of sources or simply propose a ballpark estimate.
In one recent example, Podgursky, Brodsky and Hauke (2008) explain how Missouri might use a tuition tax credit program to provide opportunities for children in failing urban public schools to attend private schools. The authors explain that public school spending in the urban core districts is on the order of $10,000 to $12,000 per pupil and that tuition tax credits could be used to raise funds to provide vouchers to urban public school students at a level of $5,000, yielding a 50% savings rate for each child who opts out of the public schools and takes the voucher. To support their argument that the $5,000 voucher is sufficient subsidy for private schooling, Podgursky and colleagues note: “we assume that average private school tuition is $4,000 per year, in line with estimates of the national average” (Podgursky et al., Jan 2008) The authors footnote this assertion as follows:

  • For example, the U.S. Department of Education’s National Center for Education Statistics reports in Table 59 of the 2005 Digest of Education Statistics that the average national cost of elementary/secondary private school tuition is $4,689 per year. In addition, a well-cited 2003 Cato Institute study found that the average elementary tuition cost was less than $3,500 per year by looking at private schools in several major U.S. metropolitan areas. See Salisbury, David F. “What Does a Voucher Buy? A Closer Look at the Cost of Private Schools,” The Cato Institute, Policy Analysis, No. 486, August 2003

That is, the authors posit that the average private school tuition in Missouri major metropolitan areas 2008 is somewhat less than the average impartially (NCES) reported average national tuition in 2004, and similar to the average tuition reported in a Cato Institute survey of 2002-03 tuition levels in select cities nationally.
Indeed there are no easily accessible estimates of actual private school costs in St. Louis or Kansas City, Missouri. That said, the choice to recognize a voucher level of $4,000 to $5,000 as sufficient subsidy, based on the cited information is problematic at best. The use in a 2008 policy brief in Missouri of the $3,500 (Cato, 2002-03) and $4,000 (NCES, 2005) figures assumes these figures to be both (a) timeless, not subject to inflation and (b) and spaceless, insensitive to regional price variation. These are two deeply flawed assumptions.
Note that the only reference to a national average tuition of $3,500 in the Cato report comes from the report’s abstract, which cites NCES data from 1999-00 rather than the report’s own survey findings from 2002-03: “Government figures indicate that the average private elementary school tuition in the United States is less than $3,500 and the average private secondary school tuition is $6,052.” (p. 1) The $3,500 average national tuition figure has taken on almost mythical status in political and media circles and think tank reports, as evidenced by results of a Google search of the terms “private school cost” and “$3,500.” Notably, the higher secondary tuition level is rarely mentioned.
At the time of the Podgursky brief, even if the $3,500 Cato (2003) figure had been representative of Missouri urban private school tuition, that figure was already 9 years old. Beyond inflation, the National Center for Education Statistics Education Comparable Wage Index reflects that both the Kansas City and St. Louis labor markets have higher than national average labor costs, which may affect private school tuition. Indeed, the labor markets identified in the Cato report may be even higher wage labor markets, but no attempt is made to correct for regional variation or identify tuition levels in Missouri urban private schools. In simple terms, dollar values are neither spaceless nor timeless.
Cato’s own summary of city median tuition levels for primary/elementary schools and secondary schools shows considerable variation and shows that the 1999-00 NCES national average figure of $3,500 is insufficient in 2002-03 at the secondary level to meet full tuition in any city, and is only sufficient at the elementary level to meet tuition in 2 of 6 cities.

Table 1
Findings from Cato Tuition Survey
City    Median 2002-03
Tuition
New Orleans    $3,895
Houston    $4,325 primary
$6,150 secondary
Denver    $3,528 primary
$5,995 secondary
Charleston, SC    $3,153 primary
$4,056 secondary
Washington DC    $4,500 primary
$16,075 secondary
Philadelphia    $2,504 primary
$4,310 secondary
Cato, p. 5

Perhaps the best available national resource for understanding private school tuition rates and how they very by school type and location, is the Private School Survey component of the National Center for Education Statistics Schools and Staffing Survey. SASS is the source from which private school tuition averages are generated for the NCES Digest of Education Statistics. SASS is based on a sample of private schools, not all private schools, but that sample may be weighted such that calculations of mean tuition rates and teacher salaries, or other variables are representative of national or regional populations of all private schools. That said, these data do not get the level of public exposure they perhaps deserve, given the thirst for private school financial comparisons. SASS tuition data do play a role in a number of empirically rigorous analyses of the private school enrollment behavior. For example, Epple, Figlio and Romano (2004) use the SASS tuition data to evaluate income related stratification of students in private schools in metropolitan areas into “elite” (highest tuition) private schools.
Central to the analysis herein is the point that tuition levels do not indicate operating cost levels. I am far from the first author to make this acknowledgement. For example, in a cost-benefit analysis of education reform strategies, Yeh (2007) explains:

  • The real social cost of educating large numbers of students in private schools (who are currently educated in public schools) is difficult to estimate for several reasons: Private school tuition figures exclude costs that are offset by corporate and noncorporate subsidies (U.S. General Accounting Office, 2001), as well as the cost of services that would be required by many students (and, by law, are currently provided by public schools, but not private schools), including transportation, free and reduced-price meals, special education, vocational education, and services for students with disabilities and limited English proficiency (Belfield, 2006; Levin, 1998; Levin & Driver, 1997).

Similarly, in an analysis of the effects of private and charter school competition on the teaching profession, Hoxby (2002) notes:

  • For instance, in some metropolitan areas, up to 15 percent of the elementary student population is enrolled in private schools where tuition is about two-thirds of the schools’ per-pupil expenditure. (Typical amounts for schools with religious affiliation would be tuition of about $1,600 and expenditure of about $2,300 dollars). (Hoxby, 2002, p. 861)

Interestingly, however, Hoxby provides no citation for the ballpark figures she chooses. Yet, others stand stubbornly in denial that there exists any problem in using tuition data to represent operating costs for private schooling:

  • One can also get some additional insight on the comparative costs of private and public schools by looking at the quoted tuition charged by private schools. For obvious reasons, quoted private school tuitions necessarily have a somewhat loose connection with costs. They are usually supplemented by endowments, contributions, fundraising events, in-kind contributions by parents, and below-cost wages for religious teachers and other staff. Yet, clearly these do not account for much of the observed difference between private tuition and public school costs. (Wenders, 2005, p. 223)

While unfounded and in conflict with the previous two statements, Wenders (2005) assertion is neither easily supported nor refuted with existing literature.
Why does it matter whether tuition is representative of cost or whether the size of margin between tuition and costs is 1% or 20%? Assume that a group of church subsidized private schools charges $3,500 per child in tuition, but those schools operate at a cost of $7,000 per pupil. When serving the children of the church community, parents sending their children to the school pay their tuition per child and likely also offer a tithing, perhaps equal to or greater than the difference between tuition and cost. Assuming that on average, the parent tithing is less than the margin of difference, other parishioners’ tithing and other contributions to the church and school make up the difference.
Now, assume we wish to send, on public voucher, 100 additional children to the private school, from the public system, at a voucher level that matches the full tuition level, $3,500. But, if operating costs remain at $7,000 per pupil, someone must contribute an additional $350,000 to the school to cover costs of the additional 100 students. If the students tuitioned to the school through the voucher system are both poor and non-parishioners it is unlikely that they will provide the additional resources themselves. The larger the desired voucher system – more extensive participation rates – the larger the additional philanthropy requirement becomes. This is potentially even more constraining under tuition tax credit programs which rely on philanthropy to generate the initial tuition vouchers.
To their credit, the Cato (2003) report is one of the only sources available that attempts to compile information on private school tuition rates for a multitude of schools  in specific metropolitan areas, including New Orleans; Houston; Denver; Charleston, S.C.; Washington, D.C.; and Philadelphia. Cato surveyed several hundred, primarily Catholic and other religious private schools, gathering 2002-03 tuition data, and concluded that a voucher level of $5,000 would give students access to most private schools in the cities surveyed and that “Since average per pupil spending for public schools is now $8,830, most states could offer a voucher amount even greater than $5,000 and still realize substantial savings.”  (Cato, p. 1)
The Cato report, however, suffers the central problem of asserting that private schools can take on additional students at these tuition levels and subsidize the difference via philanthropy. A select review of financial statements (IRS 990) for 2006 spending per pupil, compared with tuition levels listed in the Cato report appendix reveals the following:

  • Riverside Academy in New Orleans reported tuition of $2,385 to $2,790 to Cato. Riverside Academy spent in 2006, $3,857,985 on 528 students, or $7,307 per pupil.
  • Northland Christian reported tuition of $8,300 and spends $8,467. Here, tuition is closer to spending, but relatively high on the Cato list for Houston. (Houston)
  • Galloway school reported tuition of $5960, but spends $8,431. (Houston)
  • Westbury Christian reported $4,450 but spends about $7,059. (Houston)
  • Friends Select reported $14,255 to $16,070 but spends $20,161. (Philadelphia)
  • City Center Academy (a Presbyterian school) reported tuition of 3,800 but spends over $10,000, serving only about 75 students.

Private school vouchers are commonly recommended or applied at levels ranging from under $3,000 to just over $6,000, rarely higher. Podgursky (2008) and Aud (2007) each discuss Arizona tuition tax credit programs offering $4,200 to $5,000 vouchers, Florida’s A+ voucher at $4,063, Cleveland’s well-known voucher program offering $2,686 vouchers in 2004-05 and the Milwaukee voucher program offering vouchers of $6,351 per pupil in 2005-06. Invariably these voucher levels are argued by pundits to be sufficient on the basis that it is well documented and well understood that private school tuition is approximately $3,500 to $4,000 per child.
A recent policy brief on saving Catholic schooling in urban American highlights problems with undersubsidized vouchers. Hamilton, Finn and Petrilli (2008) note that:

  • In Milwaukee, the city with the nation’s largest publicly funded school voucher program, enrollment is still declining in many inner-city Catholic schools. In Washington, D.C., despite federally funded vouchers for the tuition of poor, mostly non-Catholic inner city children, the Church is turning seven schools into public charters – which will be well funded, but non-religious. (p. 6)

Recall that the voucher level in Milwaukee is actually much higher than other existing publicly financed voucher policies ($6,351). Yet even at this level the voucher is insufficient for propping up the urban catholic schools.
Belfield, Levin and Schwartz (2006) elaborate on the sufficiency of voucher levels in Milwaukee. The authors note that by 2001, 40% of voucher receiving schools had more than 80% of their students on vouchers, and that the supply of providers had increased since implementation of the voucher program. But, the authors note that explanations for the increase in supply are not found by exploring the relationship between voucher revenue and costs because “Many schools report costs above the value of the voucher, and costs only weakly converge to the voucher amount” (p. 1). Again, this scenario occurs under one of the higher funded voucher models.
The primary strategy proposed by Hamilton, Finn and Petrilli (2008) for saving urban catholic schools – short of converting them to charter schools – is to scale up dramatically philanthropy among church parishioners and to refocus catholic schooling on catholic children of parishioners. Where subsidies fail to cover costs, philanthropy must fill the gap. Costs may not be reduced sufficiently to match the voucher subsidy. The authors point to recent efforts in Wichita, KS as a model:

  • Catholic schooling would be free to all parishioners. To make the financials work, the bishop asked all Church members to tithe a significant portion of their salaries, which largely went into the school operations fund. Parishioners responded enthusiastically. Today, all Wichita Catholics can send their children to parochial school. Tuition is no barrier. (p. 9)

The authors point out, that among other issues in raising financial support, church members are reluctant to take on the additional public service mission of funding the non-catholic poor.

  • Catholic parishioners have been willing to help to a point, but our survey shows that about six in ten Catholics now view “working with economically disadvantaged students” as the domain of public schools. (p. 8)

Stepping back somewhat from traditional voucher arguments that public taxpayer dollars should be used to partially subsidize the catholic school education of non-catholic poor urban children, Hamilton, Finn and Petrilli (2008) suggest instead that urban catholic schooling might better survive under a heavily subsidized parish tax system (strong-armed and substantial tithing) coupled with emphasis on serving burgeoning Hispanic, catholic low income populations (rather than non-catholic urban blacks, historically served in urban voucher systems like Milwaukee).

  • The Church should heed Wichita’s example and embark on a serious campaign to make Catholic education affordable – even free – for all Catholics. Such an effort will be particularly significant for America’s recent Hispanic immigrants, many of whom live near urban Catholic schools with a rich history of educating children new to our shores. This means asking parishioners to dig deep. It also means being aggressive about revitalizing rundown, ill-managed parish schools with an eye to making the system as a whole as efficient and effective as possible.

The bottom line is that when vouchers cover only tuition rates or portions of tuition rates, someone must dig deep to ensure that service providers can be sustained.  Alternatively, to reduce the requirement of additional philanthropy, policymakers must have more accurate information regarding the actual costs of providing private schooling rather than average tuition rates.

(this draft excerpt is taken from forthcoming work in which I evaluate IRS tax filings of over 1,500 private schools nationally in order to provide a more thorough understanding of private school costs and how those costs vary by school type and location)

What does $5,000 get you in Georgia?

This proposal caught my eye the other day… but it took me a few days to round up the relevant data and figures.

http://www.beaconcast.com/articles/20090307

The proposal is to give every kid in Georgia a $5,000 dollar voucher to take to the public or private school of his or her choice. At the time of this blog post, I am currently working on a massive research project compiling the financial statements of private schools (IRS 990) and combining those data with enrollment data for those schools in an attempt to figure out what private schools actually spend per pupil. NOT WHAT THEIR TUITION RATE IS, BUT WHAT THEY SPEND! I’m looking at total expenditures per pupil in a given tax year (2006) and comparing with total expenditures per pupil in public schools in the same labor market (mainly metro areas).

I’m all for any plan that could substantially disrupt the educational disparities that result from racial and socioeconomic segregation in neighborhoods, housing stock, etc. that ultimately dictate who goes to what school, who chooses to teach where and as a result, strongly influences the distribution of schooling quality in ways that contribute substantially to persistence of black-white achievement gaps. Watch this exceptional lecture for a more thorough explanation:

http://www.uark.edu/ua/der/Lecture_Series/Hanushek/Hanushek.html

The first simple critique I have of the current Georgia Senate Bill (SB) 90 proposal is that it fails to provide at least some transportation support/subsidy… which means that choice will be severely limited for lower income families. That’s a serious problem which undermines most if not all good that might come of such a proposal.

But aside from that little problem, let’s take a look at some basic stats on private schooling enrollments, types and per pupil spending in Georgia… to see how this $5,000 voucher stacks up.

Figure 4 (sorry about not including 1 though 3, limited space in the blog) shows that the largest two sectors of private schooling in Georgia are Christian (not Catholic) schools (affiliated with AACS and/or ACIS) and Independent Schools. Actually, the strong presence of independent schools could potentially lessen concerns that most if not all choices made by students and families would result in channeling funds to religious schools. That is, if funding was sufficient for independent schools.

georgia-private-schools

Figure 5 displays the student weighted frequency distribution for independent school per pupil spending in Georgia in 2006 based on IRS filings of 43 independent day schools serving nearly 26,000 students. The minimum expenditure was $9,579, nearly twice the proposed voucher level.

From my most recent data source, there are over 850,00 students in public schools in the Atlanta metro area alone. So, if even 1% of those students wanted to attend even the lowest spending private independent school (8,500), someone would need to cough up and additional $38,921,500 to cover the gap in annual costs (ignoring the capital expense of taking on an additional 8,500 students). If 1% of students in the Atlanta metro area attended the average (statewide mean, Atlanta mean even higher) independent school, at a gap of $10,061, someone would have to cough up an additional $85,518,500. So much for the big savings from vouchers here. If state government is saving, then it’s only because someone else is forced to pick up the tab.

Private Independent School per Pupil Expenditures in Georgia
Private Independent School per Pupil Expenditures in Georgia

Okay… so lets assume instead that 1% of Atlanta area children shift over to Christian schools. Surely they must be a lot cheaper than the public schools. Everyone… I mean everyone knows and fully accepts the conventional wisdom that private schools spend a whole lot less than public schools… and of course… get much better results… right?

Figure 6 shows the distribution of per pupil spending for Christian schools from IRS filings of 36 Christian schools statewide in Georgia, serving 13,777 students. My IRS filing rate is lower here than for independent schools for two reasons. Many formally religiously affiliated schools don’t file IRS 990. Second, I eliminated from my data, schools with expenditure budgets smaller than $500,000 (mostly really tiny schools, some start-up, some dying out). Figure 6 shows that there are some Christian schools that show spending levels below 5,000 (about 8 schools). But still, the enrollment weighted average is nearly $8,000, meaning that a 1% shift in Atlanta metro public school students to “average” private Christian schools would still require $25,500,000 to come from somewhere.

Perhaps some rethinking of Senate Bill (SB) 90 is in order.

Christian School per Pupil Expenditures in Georgia
Christian School per Pupil Expenditures in Georgia

Raising Arizona School Finance? Can’t be done!

So… I come across this blog which at first appears on target… indicating that typical rankings of state school finance are problematic. But then the blog goes awry…

http://www.paysonroundup.com/news/2009/mar/06/pupil_spending_statistic_meaningless_misleading/

The point of the blog is that Arizona school funding isn’t as bad as it looks. Not 49th nationally, and that funding alone is not all important. So… I went back to my model based estimates of “total state and local revenue per pupil” (which includes capital…), and my state effort indices (total state and local school spending as a share of gross state product and an actual student outcome measure – NAEP math and reading scores to see how Arizona really does.

My state and local revenue comparisons are based on a statistical model which controls for differences in the percent of children attending small remote schools, rates of poverty, regional variation in competitive wages, children with disabilities, etc. It controls for the generally accepted set of cost factors influencing the costs of producing student outcomes. With the model, I generate predicted values for a district of the same characteristics across all states (to the extent that such a district exists). Nonetheless, it’s the best way I can currently think of to get rid of all of that messiness that leads to misleading comparisons in Education Week and the Education Trust.

So… How does Arizona do?

How Does Arizona Really Do?
How Does Arizona Really Do?

So… Arizona is not 49th nationally… it’s 46th. Hooray! And, It ranks 43rd in effort, and 43rd in NAEP reading. The best AZ does is 37th in NAEP math.

In addition to these statistics, on which Arizona performs quite poorly, let me share a few other major concerns I have with Arizona school finance. Arizona is among the few states, which by the design of the state aid allocation formula actually allocates systematically fewer resources to higher poverty districts. This occurs because the state formula – unlike nearly all others – has no adjustment for poverty concentration. And, working against poverty, the state formula has an adjustment to drive more money to districts that already have more experienced teachers (Teacher Experience Index). This is disgraceful… and puts Arizona near the top of my list for “worst of the worst” in school finance.

Quick note to the author of the blog to which I initially responded here. The fact that Arizona ranks 21st (above 25th) on total education spending is meaningless when Arizona ranks 16th in the number of  children to be educated with that spending, and 13th in child poverty (Small Area Income and Poverty Estimates of the U.S. Census, 2006).

A technical note – having worked back and forth between state reported data systems and NCES fiscal data, I usually find a reasonable degree of correlation between the two. Not in Arizona. Arizona data – either the state version or the NCES version are highly irregular and it is difficult to discern which data are most accurate. I suspect that the state data are… but they are not generally available on the web (as they are in most states). AZ is particularly bad at public disclosure of school finance data.

Take your 65 cents and …well… you know

I was pleased to read this morning that the Texas Legislature is now finally reconsidering the error of adopting into law the 65% requirement pitched by pundits a few years back as a revenue neutral “solution” for increasing education spending.

http://www.star-telegram.com/804/story/1242118.html

Of course, it was in Texas that the actual agenda of those pundits was revealed in a memo exposed in the Austin American Statesman.

Doug Elmer of the University of Kansas and I wrote about the 65 cent solution in the January 2009 issue of Educational Policy (The Politics of Off-the-Shelf School Finance Reform).

http://epx.sagepub.com/cgi/reprint/23/1/66.pdf

What makes the 65 cent solution so interesting, in retrospect, is that it was a “reform” that in some states become legislation, but was never based on any evidence whatsoever that allocating 65 cents of every “education dollar” improves outcomes. In fact, Doug Elmer and I discuss the best empirical research on this topic which suggests otherwise.

Further, setting aside those good empirical studies of the relationship between instructional budget shares and student outcomes, the basic argument for the 65 cent solution was utterly absurd. The 65 cent solution was based on the argument that public school systems are inefficient and wasteful and should be told how to use their money. So… the 65 cent target was selected as roughly (slightly higher than) the average percent of current expenditures allocated to instruction (based on NCES expenditure data). So… the idea is that school districts are inneficient and wasteful in general, therefore we should ask them all to spend roughly like the average public school district? Good thinkin’… eh?

Public Schooling in Louisiana and Mississippi

Here are some quick summary points regarding education funding and student outcomes in Louisiana and Mississippi, two states getting some national attention these days…

1. In Louisiana, according to data on 5 to 17 year olds from the American Community Surveys of 2005 to 2007, approximately 17% of children avoid the public schooling system to attend private schools. That’s second highest in the nation. Mississippi is closer to, but still above the middle among states.

2. In addition to ranking second highest in non-public school attendance, Louisiana also ranks second lowest in “effort” in financing public schools, where effort is measured as total state and local spending on public education as a percentage of gross state product. Mississipi, because it is a generally poorer state with lower GSP, is nearer the average on effort.

3. Both states rank near the bottom annually on the reading and math portions of the National Assessment of Educational Progress.

4. Both states rank near the bottom annually on state and local per pupil spending on public schools even after adjusting for regional variation in competitive wages and for various other characteristics of public schools (size, poverty rate, etc.).

The two figures below are illustrative of the position of Louisiana and Mississippi on questions of education funding, effort and outcomes.

Regional Wage and Cost Adjusted State and Local Revenues and NAEP Reading
Regional Wage and Cost Adjusted State and Local Revenues and NAEP Reading

This first chart shows that there exists a modest relationship between state and local spending on education across states and NAEP reading scores. And two states that spend little and achieve little are Louisiana and Mississippi.

Effort and NAEP Reading
Effort and NAEP Reading

This second graph shows the relationship between Effort (% of GDP spent on public schools including state and local resources) and NAEP reading scores. As noted above, Mississippi puts up relatively average effort but in spite of this effort simply cannot muster the resources to achieve desirable outcomes with it’s very high poverty student population.

Lousiana has much less excuse than Mississippi. It’s effort is low. It’s spending is low, and indeed it’s outcomes are low, for those who remain left behind in public schools in Lousiana.

Weighting for Success in Philadelphia?

Once again, efforts to reform Philadelphia public schools are in the news, and one item that remains at least a significant part of that reform package is a proposal to use a Weighted Student Funding formula to improve equity in resource allocation across schools within the Philadelphia public school system. Here’s the link to a recent story:

http://www.philly.com/inquirer/education/20090223_District_to_reveal_plan_for_revamping_schools.html

Let me be absolutely clear that I am not opposed to Weighted Student Funding per se. What I am opposed to, and have been very vocal about, is the overselling of Weighted Student Funding as a panacea for both within district equity concerns and for decentralized management of schools and school systems.

Let me start here by clarifying that Weighted Student Funding and decentralized management are two separate issues that are often purposefully entangled when presented by pundits. Indeed, having a well defined school based allocation formula can enable decentralization of decision making to building principals. I offer (reiterate) three potential concerns here regarding weighted student funding coupled with decentralized management, specifically regarding Philadelphia.

1. It remains very difficult if not entirely infeasible for large urban school districts to successfully tilt their internal playing field (across schools within district) when those large urban districts remain at a competitive disadvantage regarding financial resources compared to surrounding districts competing for teachers on the same labor market.

In no major city in the nation is this concern more true than in Philadelphia (with Chicago running second). In relative terms (urban core per pupil spending relative to surrounding districts), Philadelphia has consistently been the least well funded urban core district in the nation for quite some time… falling in some years as low as 76% of surrounding district spending (using the NCES labor market definition).

In my recent work on Texas and Ohio cities, none of which are as poorly positioned as Philadelphia, I found that the relative funding of the urban core compared to surrounding districts poses a significant constraint on the urban core district’s ability to reshuffle funding. For example, the lowest poverty, lowest minority concentration schools in the urban core typically are higher poverty, higher minority concentration than suburban schools only streets away. Even before tilting the playing field to move funding out of the lower poverty urban district schools and to the higher poverty ones, the lower poverty schools within the urban districts are already at a competitive disadvantage relative to even lower poverty neighboring schools in adjacent suburbs. This is not to suggest, by any means, that the urban core districts should make no attempt at leveling their playing field, but that the urban district may be unable to significantly tilt their playing field to assist the higher poverty schools. Here is a schematic diagram I often use to describe this problem:

Difficulties in Tilting the Within District Playing Field
Difficulties in Tilting the Within District Playing Field

Here is an example of school level budgets per pupil (elementary schools) in a section of Dallas, immediately adjacent to marginally lower poverty but higher spending schools in Mesquite ISD.

Spending per Pupil in Dallas Schools and Neighboring District
Spending per Pupil in Dallas Schools and Neighboring District

Now, credit should be given to ye ol’ Commonwealth of PA for recently adopting their first, statewide, reasonably organized basic education funding formula (special education remains substantially screwed up), albeit slowly phased in and already under the gun due to state budget concerns. However, the figure below shows that even after first year state support shifts, Philadelphia city school district remains at a significant financial disadvantage compared to districts in Montgomery, Bucks, Chester and Delaware counties (neighbors in the same labor market).

Philadelphia is much higher poverty than neighbors, and still has fewer financial resources!
Philadelphia is much higher poverty than neighbors, and still has fewer financial resources!

2. There is little compelling evidence that large urban districts using Weighted Student Funding are achieving any greater equity in resource allocation across schools when compared with other large urban districts in the same state which do not use WSF.

This was a major finding of my recent and ongoing work. Look, the reality is that any resource allocation formula from state to school districts or from school districts to schools, is subject to political tug-of-war.Whether we’re talking about allocating staffing positions from one school to another, or dollars generated by weights, constituents involved in the process will attempt to figure out which levers on the system can be used to benefit them and then the games will begin. This is how, for example, Cincinnati ends up adopting within its weighted funding formula a larger adjustment for gifted children than for children in poverty. This is how, for example, the state of Kansas had adopted a larger weight for children in “new facilities” (in affluent suburban districts) than for children in poverty or children with limited English language proficiency (prior to recent court rulings). These are the politics of weighted funding systems – and these politics differ little from any other politics which involving shifting finite resources toward some and away from others.

One should not be fooled into believing that Weighted Funding eliminates such games. It merely creates new ones. (see my Art of Inequitable School Funding post)

3. After presenting one critical review of weighted student funding and decentralized governance I was challenged by a skeptic of my work to provide any reasonable argument against the decentralized governance component – on the basis that it is somehow a well understood and broadly accepted fact among scholars that decentralized governance of large urban school districts is necessarily good – always and forever.

In light of the first two points above, consider the following. First, decentralized decision making is and can only be as good as the decentralized decision makers – in this case, the principals of schools to be granted greater control. Now, this issue is largely about equity right? Well, as it turns out, in many urban settings, the principals with the weakest academic qualifications (those who’ve repeatedly failed certification exams, attended academically weaker undergraduate and graduate preparation programs) are leading the schools with the highest poverty and minority concentrations. Some of my earlier work (in Educational Administration Quarterly with Bruce Cooper) indicates a propensity among principals to hire teachers with academic backgrounds similar to their own – a propensity which can work for the positive, or negative.

With leadership quality distributed in this manner across schools within large urban districts, would decentralizing control lead to greater equity? I suspect not. Further, if district resources are relatively constrained as in Philadelphia it remains unlikely that leadership quality may be effectively redistributed (by paying high enough salaries to principals of the toughest schools)  and even less likely that those principals can be provided sufficient resources to recruit and retain the teachers they need.

===

So… all of that heavy stuff in mind, I urge caution in making too much of this next round of Philadelphia school reforms and other similar attempts elsewhere. Without sufficient targetted resources to Philadelphia public schools, the district may continue to spin its wheels for some time. I hope not.

Link to recent article: http://www.epaa.info/ojs/index.php/epaa/article/view/158/30

The Fine Art of Inequitable School Finance Policy

One area of school finance about which I’ve written  quite a bit is what I refer to as the “tricks of the trade” in state aid allocation formulas. Most people who study or observe state school funding policies, including popular media outlets like Education Week tend to make the automatic assumption that increased state effort toward financing local schools necessarily leads to more equitable distribution of resources. After all, the initial goals of state intervention and aid allocation were to (a) encourage towns to create public schooling systems for their children and (b) to equalize towns’ ability to pay for schools, because substantial disparities in taxable property wealth across towns made it more difficult for some than others to pay for quality schools.

However, as state aid formulas have become increasingly complex over time, so too have the various ways in which state legislatures may craft allocations of aid in ways that actually send fewer dollars to those who need them more. Here is my short list of some of the more common Tricks of the Trade that shift state aid away from poorer, higher minority concentration school districts and quite often toward neighboring affluent suburban districts. Many of these policies lead to sharp racial differences in funding across school districts. And at least some of these policies are built on historical racial disparities within public education systems and in housing and residential segregation.

1. Aid allocated based on Average Daily Attendance rather than based on enrollment or membership. This one is relatively straight forward. Higher poverty and higher minority concentration school districts tend  to have lower average daily attendance rates. As such, providing per pupil financing on the basis of average daily attendance systematically reduces aid to high poverty, high minority districts. Preston Green and I show this effect in an article titled “Urban Legends, Desegregation and School Finance: Did Kansas City Really Prove that Money Doesn’t Matter.”

===

Missouri is among a handful of states that continues to provide aid to local public school districts on the basis of their average daily attendance (ADA) rather than by enrolled pupil count or membership. From 2000 to 2004, poverty rates and black student population share alone explain 59% of variations in attendance rates across Missouri school districts enrolling over 2,000 students. Both black population share and poverty rate are strongly associated with lower attendance rates, leading to systematically lower funding per eligible or enrolled pupil in districts with higher shares of either population. Table 9 shows that, in 1999, while districts on average (excluding KCMSD) lost 5.6% of state aid due to differences between enrollment and ADA, KCMSD lost nearly 13%. That margin has decreased after KCMSD had improved its attendance rates. Nonetheless, KCMSD continues to receive a lower share of state aid due to ADA based funding, than other districts with lower poverty rates and smaller black populations.

====

2. Incorrectly estimated or wrongly conceived geographic cost (wage variation) adjustments. Geographic cost adjustments seem to be the new, hip and cool way to manipulate state aid formulas in order to drive more funding to affluent suburban districts, or in the case of Wyoming, very affluent tourist locations (Jackson Hole). Because estimating a well conceived and appropriate factor to compensate for variations in employment costs can be a complicated endeavor, Legislatures in some states have latched on to “plain common sense” solutions to twist state aid in their favor. I write about the worst of these in a recent article in the journal of education finance titled Doing more harm than good. The two most offensive examples of which I am presently aware are the Wyoming legislature’s choice to retain an index that provides 40% additional resources to Jackson Wyoming simply because the average housing value in Jackson (for tourist properties of the rich) is much higher than the rest of the state. Equally offensive is the Kansas policy that allows additional revenues to be raised in the 17 districts with housing values over 25% above state average. Most problematic about this Kansas policy is that those receiving the adjustment are districts in the Kansas City metropolitan area where the majority of residential housing was developed with racial restrictions in deeds – restrictions that were introduced as late as 1962 (years after prohibited by the U.S. Supreme Court) and were only symbolically stricken from deeds in 2006 (after some Kansas legislators resisted when the issue was raised the previous year). For a fun article on this topic, see: http://www.pitch.com/2005-04-14/news/funny-math/

More on this topic at a later point.

3. Allocating or adjusting aid based on existing patterns of teacher degree levels or teacher experience. An alternative to creating a complex wage index for adjusting aid is simply to include a factor that reinforces directly the existing patterns of disparities in teacher qualifications across wealthier and poorer school districts. Preston Green and I, in an article in the American Journal of Education titled “Tricks of the Trade” explain how Alabama, which at one time funded its highly segregated public schooling system on teacher units, assigning white teacher units twice the dollar value of black teachers, managed to accomplish roughly the same pattern of disparity across black and white schools by retaining their teacher unit funding model but allocating more money to schools having teachers with advanced degrees (the whiter, less poor schools). The Arizona alternative is to provide more funding to districts where the average teacher experience level is above the state average (whiter, less poor schools) – the Teacher Experience Index.

Like funding on attendance (why should we pay for kids who aren’t there?), funding on teacher experience and degree level can be argued to pass that “plain common sense” test, at least in some states. “Let’s fund ’em for the teachers they dun-did-got… not them ones they wish they dun-did-got or for cryin’ out loud, the ones they actually need.”

4. Allocating substantially higher levels of aid per high school student than per elementary school student in a district (student grade level weighting within K-12 unified school districts).

This one is a little trickier and more subtle, and thankfully the average effect of this “trick” seems to be somewhat smaller than the others on this list. But, I have found in some recent work that when a state puts a big enough differential in place on the cost of high school students relative to elementary school students, poorer, higher minority unified K-12 school districts end up receiving systematically less funding than more affluent districts. There appears to be a relationship between the percentage of children who are in elementary versus high school grades and district poverty rates. Higher poverty districts have more children in elementary grades and fewer in upper grades, and there may be any number of logical explanations for this pattern. First, dropouts may affect the pattern. Second, as children grow up beyond elementary grades, their parents may become more professionaly stable and have the opportunity to move up and out. Also, those same parents may have had the option while the child was in the elementary grades, but when faced with the prospect of poor urban middle or secondary schools they finally exercised that option. Or, they perhaps exercised the private schooling option (urban Catholic high school).

5. Census-based financing of special education programs. I used to strongly favor the “flat” or census based special education funding approach because of the increased flexibility and reduced incentive to identify children with marginal disabilities. For those not familiar, census based funding for special education involves setting a statewide average (or some other arbitrary) rate of assumed children with disabilities that exist in all school districts and then funding on that arbitrarily set assumed rate. Okay… those of you working in the field of special education can see the problem with this already… but that hasn’t slowed the policy momentum in this direction, has it?

After evaluating a handful of state special education finance programs, I’ve changed my mind… quite strongly. Hey… we don’t provide the same amout of poverty based funding across districts – because the concentration of children in poverty varies by neighborhood and town. We don’t provide the same level of funding for bilingual education programs – because the concentrations of children with limited English speaking skills varies by neighborhood and town. And, as I’ve come to learn, the concentrations of children with various forms of disabilities vary by neighborhood and town… and in some cases in logical and explanable ways. Large town and small city hubs that are otherwise distant from major urban centers and surrounded by rural areas often have much higher rates of children with disabilities than their surrounding rural districts. These patterns show up in both school enrollment data and in U.S. Census data based on resident responses (not school identification practices). It seems logical that parents having children identified with a disability might choose to relocate to the nearest population hub where a wider array of social and medical services are available. Whatever the reasons, disability population concentrations, like poverty, vary widely from one location to the next across states. And quite often, disability concentrations vary in relation to poverty and even minority student concentrations. In two states where I have conducted recent analyses, higher poverty school districts are systematically disadvantaged by this flat approach to special education funding.

I will continue to update this list over time.

Cheers!

Quality (Counts) Control…

Okay… so the media is on it. Do a Google News Search on “Education Week” and “Quality Counts” and you’ll get over 100 headlines from major and small local news outlets across the country, like:

Maryland’s Education System Ranks #1 Overall

and

Survey says Md. schools are No. 1 Delmarva Now
Md. Public Schools Ranked Nation’s Best WJZ
Gov. O’ Malley Proud of Maryland’s Education Ranking #1 Overall Bay Net

Congratulations Maryland!

Texas on the other hand…

Texas public schools score higher than national average in

And others, simply happy with improvement

Florida education system improves education quality

Unfortunately, as I discussed yesterday, at least the finance portion of these ratings is relatively meaningless – failing to capture critically important features of state school finance systems. (See my previous post)

This failure is particularly evident in the paragraph below from the Education Week summary of their newest findings. Yes, there are huge disparities in funding across Alaska schools – because tiny remote school districts are incredibly expensive to operate, relative to scale efficient districts in the state’s larger cities and towns. And yes, there are relatively large differences in spending across New Jersey districts. But New Jersey, more than any other state in the country (along with others like Minnesota), systematically allocates greater resources to poor urban districts. Much of the difference in resources across New Jersey school districts can be predicted as a function of student need variation across districts. And so it should be.

=====

http://www.edweek.org/ew/articles/2009/01/08/17sos.h28.html

Broadly speaking, equity indicators still show wide disparities across districts in many states. For example, our Restricted Range indicator measures the difference in per-pupil expenditures between school districts at the 95th and 5th percentiles of spending within individual states, adjusted for regional cost differences and student needs. Smaller gaps denote more equitable spending across the districts in a state. Our analysis found a $12,307 gap between those high- and low-spending districts in Alaska in the 2005-06 school year, the largest difference in the nation. New Jersey had the second-largest gap—$10,838—and West Virginia displayed the smallest gap at $1,895.