Another “You Cannot Be Serious!”

Saw this today:

http://www.washingtonpost.com/wp-dyn/content/article/2010/01/29/AR2010012903405.html

Huffman opines:

I’m picking on New Jersey not because it has the worst plan (it doesn’t) but because it so perfectly embodies the old way of applying for federal education funding — lots of promises and ideas; little chance of change on the ground.

By contrast, Louisiana submitted a clear, concise, actionable plan to reform a large swath of its public schools.

The beauty of Louisiana’s reform model lies in its simplicity. The state has taken critical baseline steps, it proposes expanding projects that have shown promising results, and it has ensured that participating school districts will actually do the things that are in the application.

Louisiana already built and uses a data system that ties students’ test scores to the teachers who taught them and to the universities and programs that trained the teachers. In its application, Louisiana proposes expanding the use of data and using test-score results to count for 50 percent of teacher evaluations and to help drive decisions of hiring, retaining, and promoting teachers and principals.

Thankfully, because I have little time this morning, I’ve already addressed this issue in at least two posts.

I discuss Louisiana specifically here:

https://schoolfinance101.wordpress.com/2009/12/18/disg-race-to-the-top/

And the issue of whether state data systems alone can save a state that has generally abandoned its public education system here:

https://schoolfinance101.wordpress.com/2009/12/15/why-do-states-with-best-data-systems/

One might make the simple argument that New Jersey’s old way of doing things, including sufficient financial support for schools and wider participation in the public education system – actually works – at least when compared to many other states – and certainly when compared to Louisiana. That said, Louisiana is in far greater need of stimulating improvement- but until Louisiana actually makes a substantial state commitment to its public K-12 and higher education system  that’s not likely to happen.

You cannot be serious bonus clip on link above!

NCTQ Teacher Policy Ratings: Where’s the quality?

First, to the media – the National Council on Teacher Quality Ratings are NOT ratings of actual differences in Teacher Quality across states. They are ratings of supposed steps which can be taken in state policy in order to improve teacher quality. Here, the blame goes on the media spin, not on NCTQ.

NCTQ does make some reasonable attempts to explain the research basis for their policy elements.  However, NCTQ fails miserably at understanding the importance of context within which policies are applied. For example, under AREA 2, NCTQ cites the importance of increasing numbers of teachers from more competitive colleges, and cites expanding the teacher pool as a way to accomplish this, through policies such as alternative certification. My own work a few years back on charter school hiring in states with more and less relaxed teacher certification requirements provides some support for this notion. But, my research also shows that in some cases, expanding the pool weakens, on average, the academic credentials of teachers. Some states and some regions of the country simply don’t have more competitive colleges and universities.

As many of these rating/grading systems which strongly favor deregulatory policies (and the power of state data systems) do, the NCTQ policy ratings favor those states that in fact have the weakest overall public education systems including the academically weakest teachers – of all things. NCTQ only handed out Cs and Ds for grades (and a few Fs). A quick tally based on my prior analyses of Schools and Staffing Survey Data finds that 6 of the 8 states that got a C (the high grade) fall in the bottom half of states in the percentage of teachers who attended highly or most competitive colleges (a factor acknowledged by NCTQ as important, and as a factor that would supposedly improve as a function of expanding the teacher pool). Louisiana, Alabama and Arkansas are all in the bottom 10. Most of these states also fall in the bottom half of states, and 3 in the bottom 10 states for the change in percent of teachers (03-04 to 07-08) who attended highly or most competitive colleges. None of the states that received the high grade were even in the top 20 in change in % of teachers from highly or most competitive colleges.

You know – it’s possible that teacher salaries might also be a factor here (there’s some pretty good research on this-see link), and a limiting condition might actually be the available funding for schools which is sadly lacking in many of these states. So too might the supply of high quality public colleges and universities for preparing teachers. States like Louisiana have been taking the axe to their public higher education systems of late. Deregulatory strategies cannot trump these conditions, and in fact, may worsen teacher quality and ultimately school quality under these conditions.

Increased regulatory strategies like improved data for teacher evaluation systems (also advocated by NCTQ, and quite reasonably so) are simply window dressing for states that are choosing to avoid the more difficult and more expensive problems facing their public education systems.

On numerous occasions on this blog, I’ve discussed the systemic failures of the public education systems in states like Louisiana – their failure to serve even 80% of school-aged children – or their failure to provide reasonable overall funding or target any funding to higher need districts (across most of these states).

So, if the Teacher Quality Policy ratings have little to do with actual teacher academic preparation in a state, or overall quality of the state’s education system, then what do they tell us? Apparently not much!

Education Week Does it Again: Please STOP!

Education Week has again posted the problematic QUALITY COUNTS indicator system including grades for school finance across the states. And again, Education Week has paid little attention to producing high quality indicators for measuring …quality? Why doesn’t that surprise me? But, they’ve made my life easier because I can simply refer you to my critique of last year’s Quality Counts School Finance Indicators:

https://schoolfinance101.wordpress.com/2009/01/08/education-week-quality-lacks/

Here are a few quick summary points on issues that occur year to year:

  • Ed Week uses “range” measures and “coefficient of variation” measures in its equity analysis – measures which capture overall variations and high to low variations in current expenditures across school districts. The way that Education Week calculates these measures actually penalizes states which target funds to higher need school districts, including higher poverty school districts or very small remote districts. That is, if a state actually makes efforts to accommodate cost differences across districts, they get a lower equity grade from Education Week. THAT’S JUST WRONG! Education Week uses some “cost adjustments” including a regional wage index and a nominal (and completely arbitrary) poverty adjustment. But, states like New Jersey actually provide more poverty-based support than the Ed Week adjustment, resulting in a reduction in the Ed Week equity measures. Ed Week makes no adjustment for costs associated with economies of scale or population density – major factors affecting spending variation across school districts within states.
  • Ed Week continues to use peculiar (though traditional) school finance measures like the McLoone Index to evaluate the share of children within the state who are in districts near the median spending level. This was originally conceived as a within state relative adequacy measure. But, without appropriate consideration for needs or costs, a state can score well on the Ed Week McLoone index by simply having all of its low income children clustered together in one or a handful of districts that spend at the edge of the lower half of the distribution.

Education Week staffers – Please Stop! Quality Counts is very unhelpful because of the extent to which it misinforms. There may be, and in fact are, some good and useful indicators in the report, but there are at least equal numbers of indicators that are entirely misleading. One cancels out the other.

These indicators can have a serious negative policy impact because of the way in which and extent to which they misinform. Drawing from a forthcoming technical report (referring to both Ed Week and Ed Trust indicators):

To illustrate the potential negative impact of these two reports, in 2003 in the context of state school finance litigation in Kansas, attorneys defending the State submitted in defense of the school funding formula, both the Education Trust finding that higher poverty districts had higher revenue per pupil and the Education Week finding that Kansas showed a good McLoone index. The state’s attorneys and local news outlets did not understand why Kansas received good ratings on these indices nor did they care as long as those indices were from highly publicized, publicly recognized sources. Plaintiffs pointed out that Education Trust finding was not a function of systematic poverty related support, but rather a function of small rural school support which left out the poorer urban and large town districts and that the “good” McLoone index was a function of having nearly half of the state’s children and nearly all of the state’s poor minority children attending six districts with below average revenues. These points were difficult to make in the face of media accolades for state’s supposed achievements regarding school funding equity and adequacy. The district court and eventually Supreme Court of Kansas declared the state school finance system unconstitutional, but not without at least a few vocal critics chastising the judges who would give the legislature a failing grade for a school finance system that had received a grade of “B” from a leading national media outlet.

Update – Do School Finance Reforms Matter?

Here’s an excerpt from a forthcoming article on whether school finance reforms have made any difference for students. The article is partly in response to claims by Eric Hanushek and Alfred Lindseth that school finance reforms have resulted in massive increases in funding to public schools which have not helped and may have in fact harmed children. My forthcoming work on this topic is co-authored with Kevin Welner of U. of Colorado.

=====

In terms of quality and scope, the most useful single study of judicially induced state finance reform was published by Card and Payne in 2002. They found that court declarations of unconstitutionality in the 1980s increased the relative funding provided to low-income districts. And they found that these school finance reforms had, in turn, significant equity effects on academic outcomes:

Using micro samples of SAT scores from this same period, we then test whether changes in spending inequality affect the gap in achievement between different family background groups. We find evidence that equalization of spending leads to a narrowing of test score outcomes across family background groups. (p. 49)

To evaluate distributional changes in school finance, Card and Payne estimated the partial correlations between current expenditures per pupil and median family income, conditional on other factors influencing demand for public schooling across districts within states and over time. Card and Payne then measured the differences in the change in income-associated spending distribution between states where school funding systems had been overturned, upheld, or where no court decision had been rendered. Importantly, they also evaluated whether structural changes to funding formulas (that is, the actual reforms) were associated with changes to the income-spending relationship, conditional on the presence of court rulings.

To make the final link between income-spending relationships and outcome gaps, Card and Payne evaluated changes in gaps in SAT scores among individual SAT test-takers categorized by family background characteristics.[1] Put in terms of our Figure 1, Card and Payne (2002) appear to have taken the greatest care in a multi-year, cross-state study, to establish appropriate linkages between litigation, reforms by type, changes in the distribution of funding, and related changes in the distribution of outcomes.

Notwithstanding the generally acknowledged importance of this study,[2] Hanushek and Lindseth (2009) never mention it in their book, including the chapter in which they conclude that school finance reforms have no positive effects.

This omission – as well as the other omissions noted below – is telling of a larger point. The development of, and reliance upon, a research base should depend on relatively objective criteria. Readers depend on authors of literature reviews to come forward with the best and most applicable research bearing on the issues under consideration. While Hanushek and Lindseth might argue that this particular omission is because Card and Payne (2002) are speaking to equity (not adequacy) litigation, we have already described how the line between equity and adequacy is not so simple. Moreover, the research that Hanushek and Lindseth do choose to include goes far beyond that directly focused on adequacy – including the Cato study of a Kansas City desegregation order discussed below.

Another key study not mentioned by Hanushek and Lindseth (2009) concerned the effects of reforms implemented under the Kansas court’s pre-ruling in 1992 (Deke, 2003). The reforms leveled up funding in low-property-wealth school districts, and Deke found as follows:

Using panel models that, if biased, are likely biased downward, I have a conservative estimate of the impact of a 20% increase in spending on the probability of going on to postsecondary education. The regression results show that such a spending increase raises that probability by approximately 5% (p. 275).

The Kansas reforms addressed by Deke (2003) came as a result of a judicial pre-order, advising the legislature that if the pending suit made it to trial, the judge would declare the school finance system unconstitutional (Baker and Green, 2006).

Hanushek and Lindseth (2009) also omitted from their discussion two additional studies, both peer-reviewed, that explore the effects of Michigan’s school finance reforms, known as “Proposal A,” implemented in the mid-1990s. Michigan’s reforms were implemented without ruling or high level of litigation threat, but the reforms were nonetheless comparable in many ways to reforms implemented following judicial rulings[3] (see Leuven et al., 2007; and Papke, 2001). In the first study, Papke (2001) finds:

Focusing on pass rates for fourth-grade and seventh grade math tests (the most complete and consistent data available for Michigan), I find that increases in spending have nontrivial, statistically significant effects on math test pass rates, and the effects are largest for schools with initially poor performance. (Papke, 2001, p. 821.)

Leuven and colleagues (2007) find no positive effects of two specific increases in funding targeted to schools with elevated at-risk populations, a convenient conclusion for Hanushek and Lindseth to have included.

A third Michigan study (available online since 2003 as a working paper from Princeton University, and now accepted for publication in Education Finance and Policy, a peer-reviewed journal) directly estimates the relationship between implemented reforms and subsequent outcomes (Roy, 2003). Roy, whose work was not cited by Hanushek and Lindseth, finds:

Proposal A was quite successful in reducing inter-district spending disparities. There were also significant gains in achievement in the poorest districts, as measured by success in state tests. However, as yet these improvements do not show up in nationwide tests like NAEP and ACT. (Roy, 2003, p. 1.)

Most recently, a study by Choudhary (2009) “estimate[s] the causal effect of increased spending on 4th and 7th grade math scores for two test measures—a scale score and a percent satisfactory measure” (p. 1). She “find[s] positive effects of increased spending on 4th grade test scores. A 60% percent increase in spending increases the percent satisfactory score by one standard deviation” (p. 1).

Perhaps because there was no judicial order involved in Michigan, researchers were able to avoid the tendency to focus on or classify the judicial order. Moreover, single-state studies generally avoid such problems because there is little statistical purpose in classifying litigation. Importantly, each of these studies focuses instead on measures of the changing distribution and level of spending (characteristics of the reforms themselves) and resulting changes in the distribution and level of outcomes. Each takes a different approach, but attempts to appropriately align their measures of spending change and outcome change, adhering to principles laid out in our Figure 1.

Other high-quality but non-peer reviewed empirical estimates of the effects of specific school finance reforms linked to court orders have been published for Vermont and Massachusetts. For example, Downes (2004), in an evaluation of Vermont school finance reforms that were ordered in 1997 and implemented in 1998, found as follows:

All of the evidence cited in this paper supports the conclusion that Act 60 has dramatically reduced dispersion in education spending and has done this by weakening the link between spending and property wealth. Further, the regressions presented in this paper offer some evidence that student performance has become more equal in the post–Act 60 period. And no results support the conclusion that Act 60 has contributed to increased dispersion in performance. (p. 312)

Hanushek and Lindseth (2009) never acknowledge this positive finding (although they do briefly cite the Downes evaluation, for a different point). Again, one might attribute this omission to the argument that the Vermont reforms were equity reforms, not adequacy reforms. However, similar to the 1992 Kansas reforms, the overall effect of the Vermont Act 60 reforms was to level up low-wealth districts and increase state school spending dramatically, thus addressing both adequacy and equity.

For Massachusetts, two independent sets of authors (in addition to Hanushek and Lindseth) have found positive reform effects. Most recently — after the Hanushek and Lindseth book was written — Downes, Zabel and Ansel (2009) found:

The achievement gap notwithstanding, this research provides new evidence that the state’s investment has had a clear and significant impact. Specifically, some of the research findings show how education reform has been successful in raising the achievement of students in the previously low-spending districts. Quite simply, this comprehensive analysis documents that without Ed Reform the achievement gap would be larger than it is today. (p. 5)

Previously, Guryan (2003) found:

Using state aid formulas as instruments, I find that increases in per-pupil spending led to significant increases in math, reading, science, and social studies test scores for 4th- and 8th-grade students. The magnitudes imply a $1,000 increase in per-pupil spending leads to about a third to a half of a standard-deviation increase in average test scores. It is noted that the state aid driving the estimates is targeted to under-funded school districts, which may have atypical returns to additional expenditures. (p. 1)

Although Hanushek and Lindseth concede that Massachusetts reforms appear successful,[4] they failed to cite Guryan’s NBER working paper, the inclusion of which would have (like most other omitted studies) weakened their overall conclusions about the non-impact of these reforms.

Turning to New Jersey, two recent (though not yet peer-reviewed) studies find positive effects of that state’s finance reforms. Alexandra Resch (2008), in a study published as a dissertation for the economics department at the University of Michigan, found evidence suggesting that New Jersey Abbott districts “directed the added resources largely to instructional personnel” (p. 1) such as additional teachers and support staff. She also concluded that this increase in funding and spending improved the achievement of students in the affected school districts. Looking at the statewide 11th grade assessment (“the only test that spans the policy change”), she found “that the policy improves test scores for minority students in the affected districts by one-fifth to one-quarter of a standard deviation” (p. 1).

The second recent study was originally presented at a 2007 conference at Columbia University, and a revised, peer-reviewed version was recently published by the Campaign for Educational Equity at Teachers College, Columbia University (Goertz and Weiss, 2009). This paper offered descriptive evidence that reveals some positive test results of recent New Jersey school finance reforms:

State Assessments: In 1999 the gap between the Abbott districts and all other districts in the state was over 30 points. By 2007 the gap was down to 19 points, a reduction of 11 points or 0.39 standard deviation units. The gap between the Abbott districts and the high-wealth districts fell from 35 to 22 points. Meanwhile performance in the low-, middle-, and high-wealth districts essentially remained parallel during this eight-year period (Figure 3, p. 23).

NAEP: The NAEP results confirm the changes we saw using state assessment data. NAEP scores in fourth-grade reading and mathematics in central cities rose 21 and 22 points, respectively between the mid-1990s and 2007, a rate that was faster than the urban fringe in both subjects and the state as a whole in reading (p. 26).

The Goertz and Weiss paper (which was, as designed and intended by the paper’s authors, the statistically least rigorous analysis of the ones presented here) does receive mention from Hanushek and Lindseth multiple times, but only in an effort to discredit and minimize its findings.

Card, D. and Payne, A. A. (2002). School Finance Reform, the Distribution of School Spending, and the Distribution of Student Test Scores. Journal of Public Economics, 83(1), 49-82.

Choudhary, L. (2009). Education Inputs, Student Performance and School Finance Reform in Michigan. Economics of Education Review, 28(1), 90-98.

Deke, J. (2003). A study of the impact of public school spending on postsecondary educational attainment using statewide school district refinancing in Kansas, Economics of Education Review, 22(3), 275-284.

Downes, T. A. (2004). School Finance Reform and School Quality: Lessons from Vermont. In Yinger, J. (ed), Helping Children Left Behind: State Aid and the Pursuit of Educational Equity. Cambridge, MA: MIT Press.

Downes, T. A., Zabel, J., Ansel, D. (2009). Incomplete Grade: Massachusetts Education Reform at 15. Boston, MA. MassINC.

Goertz, M., and Weiss, M. (2009). Assessing Success in School Finance Litigation: The Case of New Jersey. New York City: The Campaign for Educational Equity, Teachers College, Columbia University.

Guryan, J. (2003). Does Money Matter? Estimates from Education Finance Reform in Massachusetts. Working Paper No. 8269. Cambridge, MA: National Bureau of Economic Research.

Leuven, E., Lindahl, M., Oosterbeek, H., and Webbink, D. (2007). The Effect of Extra Funding for Disadvantaged Pupils on Achievement. The Review of Economics and Statistics, 89(4), 721-736.

Resch, A. M. (2008). Three Essays on Resources in Education (dissertation). Ann Arbor: University of Michigan, Department of Economics. Retrieved October 28, 2009, from http://deepblue.lib.umich.edu/bitstream/2027.42/61592/1/aresch_1.pdf

Roy, J. (2003). Impact of School Finance Reform on Resource Equalization and Academic Performance: Evidence from Michigan. Princeton University, Education Research Section Working Paper No. 8. Retrieved October 23, 2009 from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=630121 (Forthcoming in Education Finance and Policy.)


[1] Card and Payne provide substantial detail on their methodological attempts to negate the usual role of selection bias in SAT test-taking patterns. They also explain that their preference was to measure more directly the effects of income-related changes in current spending per pupil on income-related changes in SAT performance, but that the income measures in their SAT database were unreliable and could not be corroborated by other sources. As such, Card and Payne used combinations of parent education levels to proxy for income and socio-economic differences between SAT test takers.

[2] As one indication of its prominence among researchers, as of the writing of this article, Google Scholar identified 153 citations to this article.

[3] There is little reason to assume that the presence of judicial order would necessarily make otherwise similar reforms less (or more) effective, though constraints surrounding judicial remedies may.

[4] Hanushek and Lindseth attribute the success of the Massachusetts reforms not to spending, but to the fact that the “remedial steps passed by the legislature also included a vigorous regime of academic standards, a high-stakes graduation test, and strict accountability measures of a kind that have run into resistance in other states, particularly from teachers unions” (p. 169). That is, it was not the funding that mattered in Massachusetts, but rather it was the accountability reforms that accompanied the funding.

Smarter School Leaders

Smarter School Leaders: Enough to reverse the trend?

http://www.nytimes.com/2009/12/05/opinion/05herbert.html?_r=1

This recent New York Times article highlights a new doctoral program for educational leaders that is a joint venture of Harvard Graduate School of Education, Kennedy School of Government and Harvard Business School. An interesting approach indeed and one that will hopefully generate some top quality leaders for public schools and school districts. But, there are about 100,000 public schools out there, spread across 16,000 or so school districts and public charter schools. In the best of cases, each of these schools and districts would get the best and brightest possible leader. My guess, however, is that this new Harvard program will barely make a dent in our national needs.

Perhaps the new Harvard program can serve as a model for making a bigger and better dent.  Now, when I say that, I should clarify that I’m not taking the pop-policy position that this program is a model simply because it involves a business school and public policy school and the education school, but rather because it involves a GOOD business school, HIGH QUALITY public policy school and TOP NOTCH education school. There are as many, if not more intellectually vacuous b-school programs as comparably vacuous ed-school programs. You see, it’s not about b-school versus ed-school. It’s about high quality schools with highly self-selective pools of degree-seekers and top notch faculty deciding to play a more significant role in public school leadership. However, it’s going to be an uphill battle!

A few years back, Michelle Young, Terry Orr and I explored changing patterns of degree production in educational administration. With other colleagues, I explored the characteristics of faculty in educational administration programs, their pipeline and their qualifications. More recently, I’ve been exploring the effects of the changing principal preparation pipeline on schools in states like Missouri. AND IT’S NOT A PRETTY PICTURE!

Michelle, Terry and I found in our degree production study, that:

“The largest number and greatest increase were among master’s degrees. In 2003, there were 15,720 master’s degrees conferred in educational leadership, a 90 percent increase since 1993.”

And:

“Even more striking are the increases in master’s degree granting programs at Comprehensive II and Liberal Arts II institutions. Such program increases reflect a dramatic growth in the availability of programs in local and regional institutions.”

And further, that:

“The percentage of all master’s degrees produced by higher status institutions, the Research I through Doctoral II institutions dropped from 42 percent in 1993 to 36 percent in 2003.”

That is, master’s degree production in particular has mushroomed over the past decade-and-a-half and many of the new masters degrees produced are from institutions that previously had minimal involvement in educational administration and are generally considered lower status institutions.

The figure below shows the top Educational Administration masters’ granting institutions in 1990 and then again for the period from 2002 to 2005, based on data from my study with Michelle Young and Terry Orr. The data are from the National Center for Education Statistics, Integrated Postsecondary Education Data System – Degree Completion files. In 1990, Harvard made the list. But by the later period (and perhaps even worse by now), the list had changed – a lot. The list now includes mass-producers of graduate degrees like Nova Southeastern University and William Woods (Missouri) pumping out about 500 masters degrees per year in educational administration (and related degree codes). Other standout newcomers include Lindenwood University (also Missouri), National Louis University (Illinois) and St. Peters College (New Jersey).

From 2002 to 2005, Harvard continued production at its 1990 levels, like many major research universities. But by 2002 to 2005, Harvard had dropped to 68th in production, right behind Mid-America Nazarene University in Kansas (their radio jingle still sticks in my head from my Kansas years… MNU, not Harvard… who I doubt has a radio jingle).

If trends in Masters’ Degree production weren’t bad enough, similar if not more disturbing trends have occurred in the production of doctoral degrees in educational administration. In 1990, Harvard reported about 40 doctoral degrees in Educational Administration and Nova Southeastern about 100. Bad enough already. By 2005, Harvard was no-longer listing or reporting doctoral degrees granted under program codes for Educational Administration, and the biggest producers nationally were: Nova Southeastern (368), Argosy University – Sarasota (196), St. Louis University (62).  Even if these programs were/are credible, managing the quality control on 200 to 400 doctoral candidates per year seems problematic at best. Simply finding, enrolling and retaining 200 to 400 high quality candidates willing to pursue this type of degree seems a bit of a stretch! How many applied? How many, if any were rejected?

The damage done by these institutions and the diversified production of educational leaders is astounding in some states. In 1999, only a few principals of Missouri public schools held graduate degrees from the state’s emerging degree-mills. By 2006, 185 held their Masters’ degrees from Lindenwood University and 205 from William Woods out of a data set having just over 2,000 completely matched records over time. Nearly 400 of 2,000 or nearly 20% of Missouri principals held degrees from institutions which are arguably hardly qualified to grant them.

Principals who attended these graduate programs are substantially more likely to have attended the least competitive undergraduate colleges. For William Woods University, 80% of Masters Degree recipients who became Missouri principals attended undergraduate colleges in the bottom 3 (of 6) categories of competitiveness (based on Barrons’ Guide ratings) compared to 68% of principals statewide.

And further, the shares of teachers who also attended the least competitive colleges hired into schools headed by these principals have grown dramatically – from 65% to 75% from bottom two categories of Barrons’ ratings in 7 years – and faster than for other schools statewide.

This shift would be inconsequential were it not for strong and consistent evidence from a multitude of studies that the academic caliber of the teacher workforce is highly relevant to student success. While many sources highlight this issue (see for example, Baker & Cooper, 2005), Loeb and colleagues provide a particularly striking in the work in New York City. They report that:

“ . . . almost half of the teachers in the most effective quintile (based on student outcomes) graduated from a college ranked competitive or higher by Barron’s, compared to only ten percent of the teachers in the least effective quintile.”(p. 23)

This is a serious issue and one state policy makers seem unwilling to address. National accrediting agencies are comparably unwilling and/or incapable of addressing this educational leadership brain drain.

A graduate program in educational leadership or any field is only as good as the quality of its students and faculty, but criteria for program accreditation pay little attention to either the academic quality of students or qualifications of faculty.

Altering the quality of school leadership requires greater involvement of leading public and private universities, pursuing endeavors like the new Harvard program. But equally important, altering the quality of school leadership requires that state policymakers step up and shut down institutions that by the quality of their average student and qualifications of their faculty have no business preparing school leaders.

While this argument might easily be construed as academic elitism, it is important to acknowledge that this argument relates to the preparation of leaders for academic institutions –namely public schools. It is difficult to conceive of a rational argument for ignoring the relevance of academic credentials for individuals wishing to lead academic institutions.

Relevant research readings:

Baker, B., & Cooper, B. (2005). Do principals with stronger academic backgrounds hire better teachers? Policy implications for improving high-poverty schools. Educational Administration Quarterly, 41(3), 413-448.

Baker, B.D, Orr, M.T., Young, M.D. (2007) Academic Drift, Institutional Production and Professional Distribution of Graduate Degrees in Educational Administration. Educational Administration Quarterly 43 (3)  279-318

Baker, B.D., Wolf-Wendel, L.E., Twombly, S.B. (2007) Exploring the Faculty Pipeline in Educational Administration: Evidence from the Survey of Earned Doctorates 1990 to 2000. Educational Administration Quarterly 43 (2) 189-220

Checking the Tab

As follow up to yesterday’s post on the completely fabricated and back-of-the-napkin numbers presented in The Tab,  here’s a quick simulated allocation of the $11,000 foundation + $3,000 poverty weight (applied to free or reduced lunch) + $400 per ELL/LEP child.

The Tab pretty much conceals any real changes or patterns of changes by lumping them into a summary table by groups of districts without any documentation as to how the summary stats were estimated (page 27). Above is what the district by district changes would look like. Looks pretty much like a back-of-the-napkin attempt at roughly break-even analysis. Remember, this is a proposal for the future compared against actual spending from 2007-08 – two years back now!

Specifically, the proposal would appear to reduce funding in Hartford and New Haven by greater amounts than it would increase funding in districts like New Britain and Waterbury and only similarly to the increase for Bridgeport. That is, it levels down high poverty districts as much as it levels some up – a fact concealed by the claims of a net increase of $620 per pupil in the short term. Mind you, The Tab certainly provides no evidence that districts like Hartford and New Haven are massively over-funded, as their own policy solutions would imply. Oh wait… The Tab really doesn’t rely on evidence at all. Silly me.

Just checkin the numbers – the made up numbers.

Why is it OK for Think Tanks to just make stuff up?

Something that has perplexed me for some time in my field of school finance, is why it seems to be okay for policy advocates and “Think Tanks” to just make stuff up. For example, to just make up what level of funding would be appropriate for accomplishing any particular set of goals? or to just make up a figure for how much more a child with specific educational needs requires under state school finance policy. Just “making stuff up” seems particularly problematic for “Think Tanks,” which as far as I can tell should be producing information backed by at least some degree of … Thinking? Perhaps based on some of the more reasonable thinking of the field?

This topic comes to mind today because ConnCan has just released a report (http://www.conncan.org/matriarch/documents/TheTab.pdf)    on how to fix Connecticut school funding which provides classic examples of just makin’ stuff up (page 25). The report begins with a few random charts and graphs showing the differences in funding between wealthy and poor Connecticut school districts and their state and local shares of funding. These analyses, while reasonably descriptive are relatively meaningless because they are not anchored to any well conceived or articulated explanation of “what should be.” Such a conception might be located here or even here (Chapters 13, 14 & 15 are particularly on target)!

The height of making stuff up in the report is the recommended policy solution to the problem which is never clearly articulated. There are problems in CT, but The Tab, certainly doesn’t identify them!

The supposed ideal policy solution involves a pupil-based funding formula where each pupil should receive at least $11,000 per pupil (made up), and each child in poverty (no definition provided – just a few random ideas in a footnote) should receive an additional $3,000 per pupil (also made up) and each child with limited English language proficiency should receive an additional $400 per pupil (yep… totally made up). There is minimal attempt in the report (http://www.conncan.org/matriarch/documents/TheTab.pdf) to explain why these figures are reasonable. They’re simply made up.

The authors do provide some back-of-the-napkin explanations for the numbers they made up – based on those numbers being larger than the amounts typically allocated (not necessarily true). They write off the possibility that better numbers might be derived by way of a general footnote reference to a chapter in the Handbook of Research on Education Finance and Policy by Bill Duncombe and John Yinger which actually explains methods for deriving such estimates.

The authors of The Tab conclude: “Combined with federal funding that flows on the basis of poverty and (in some cases) the English Language Learner weight of an additional $400, the $3,000 poverty weight would enable districts and schools to devote considerable resources to meeting the needs of disadvantaged students.” I’m glad they are so confident in their “made up” numbers! I, however, am less so!

It would be one thing if there was no conceptual or methodological basis for figuring out which children require more resources or how much more they might actually need. Then, I guess, you might have to make stuff up. Even then, it might be reasonable to make at least some thoughtful attempt to explain why you made up the numbers you… well… made up. But alas, such thinking seems beyond the grasp of at least some “think tanks.” Guess what? There actually are some pretty good articles out there which attempt to distill additional costs associated with specific poverty measures… like this one, by Bill Duncombe and John Yinger:

How much more does a disadvantaged student cost?

It’s not like the title of this article somehow conceals its contents, does it? Nor is the journal in which it was published (Economics of Education Review) somehow tangential to the point at hand. This paper, prepared for the National Research Council provides some additional insights into additional costs associated with poverty and methods for estimating those costs.

Rather than even attempt to argue that these figures are somehow founded in something, the authors of The Tab seem to push the point that it really doesn’t matter what these numbers are as long as the state allocates pupil-based funding.  That’s the fix! That’s what matters… not how much funding or whether the right kids get the right amounts. In fact, the reverse is true. The potential effectiveness, equity and adequacy of any decentralized weighted funding system is highly contingent upon driving appropriate levels of funding and funding differentials across schools and districts!

I’ve critiqued the notion of pupil-based funding as a panacea, here:

Review of Fund the Child: Bringing Equity, Autonomy and Portability to Ohio School Finance

Review of Shortchanging Disadvantaged Students: An Analysis of Intra-district Spending Patterns in Ohio

Review of Weighted Student Formula Yearbook 2009

Oh, and also here: http://epaa.asu.edu/epaa/v17n3/

Among other things, in each of these critiques of think-tank reports I question why it seems okay to just make up “weights” and cost figures when applying distribution formulas – either for within or between district distribution.

Just thinking… but not making stuff up!

Illinois Salary Gaps – Do they matter?

I picked up this article on Twitter yesterday, which seemed at first to make a veiled version of the classic “money doesn’t matter” argument, or at least that’s how the tweets and headlines were spun. The article is somewhat more thoughtful, discussing many reasons why teacher salaries vary and how those variations are largely tied to differences in taxable property wealth across Illinois school districts, but the article misses a real opportunity to shed light on some striking disparities across the state and across districts within the Chicago metro area.

http://www.chicagotribune.com/news/education/chi-teacher-salary-09-nov09,0,1639857.story

So, how might we better understand salary variation across Illinois school districts and children, and whether that variation is problematic or not? First, we know that teachers matter! Second, we know from work by Hanushek and Rivkin that the uneven distribution of teaching quality by racial composition of students can explain substantial portions of the growth in achievement gap – black-white gap – between 3rd and 8th grade. http://faculty.smu.edu/millimet/classes/eco7321/papers/hanushek%20rivkin%2002.pdf To quote:

Unequal distributions of inexperienced teachers and of racial concentrations in schools can explain all of the increased achievement gap between grades 3 and 8. (p. 1)

Further, we know from these same authors in an earlier study that:

Table 7 suggests that a school with 10 percent more black students would require about 10 percent higher salaries in order to neutralize the increased probability of leaving. (p. 38 of PDF, not numbered)

https://www.utdallas.edu/research/tsp-erc/pdf/jrnl_hanushek_2004_public_schools_lose.pdf.pdf

Recap – Two major factors determining how well kids do in school are the characteristics of the other kids in the same class and the quality of their teacher. Unfortunately, the characteristics of kids in a given class affects who typically ends up teaching that class. Classrooms with greater shares of minority children end up with less well educated, less experienced teachers. This, in combination with peer effects, produces substantial disparities in outcomes which grow over time. Salary differentials might help to offset these disparities.

As such, it would likely be quite problematic if the teacher salaries in Illinois – WITHIN ANY GIVEN LABOR MARKET – were systematically lower in districts with higher concentrations of black and/or black and hispanic children. By cursory analysis it is rather difficult to disentangle the adverse affect of salary alone. That is, you can’t just take average salaries and try to relate them to average test scores, and then conclude that salaries don’t matter, but student characteristics do. The reality is that the two simultaneously matter and interact in important ways.

In many states, salaries and overall funding are actually comparable between districts with higher minority concentrations and other districts and in some states salaries and overall funding are actually higher (though not necessarily enough higher) in higher minority concentration districts (see: http://eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=EJ718694&ERICExtSearch_SearchType_0=no&accno=EJ718694.)

OH, BUT NOT IN ILLINOIS!

In my most recent analysis of individual teacher salary data for 2004 to 2008 in Illinois, I find that for a full time teacher, at constant contractual months, same degree level and same experience, and compared to districts in the same labor market, the teacher in a school that is majority black and Hispanic children, is paid about $2,000 less per year. Further, a teacher with a masters degree makes about $8,500 more per year. And, teachers in majority minority schools in Illinois are only about 60% to 70% as likely to hold a masters degree as teachers in predominantly white schools in the same labor market in Illinois.

We also know that the dropout rate is over 7% higher in majority minority districts compared with other districts in the same labor market. The mean ACT is over 4 points lower and the mean proficiency rates on state assessments are about 20% lower in majority minority districts compared to predominantly white districts in the same labor market.

These are striking disparities. And in Illinois, unlike many other states, state policymakers have applied no financial leverage to attempt to resolve these disparities.  No harm no foul? doubtful!

Leaders and Laggards Lags!

A quick note on Center for American Progress Leaders and Laggards report.

On pages 23 & 24, this report attempts to grade state school funding systems and their level of “innovation.” But, the report pays no attention to a) whether these states actually perform well on any measures of outcomes,  b) whether these states actually fund their schools well overall, or c) whether these states actually target any of that funding to where it’s needed most.

Quite simply, this report is complete garbage – at least the finance section! One cannot possibly rate “innovation” of a state school funding system without any regard for whether that system is sufficiently and equitably funded. You can’t stimulate innovation without an investment in Research and Development or the product itself! It really is that simple.

The best Finance grades in the report are given to such education funding laggards as:

Yet, high performing states that actually fund their systems well and target resources where needed most get lousy grades (Massachusetts & New Jersey).  This  stuff is just plain silly!

====

To lighten the mood a bit, here’s Willy Wonka summarizing the Arizona school finance formula: http://www.youtube.com/watch?v=M5QGkOGZubQ

Hawaii’s Funding Mess: My thoughts on why

It is indeed sad to see the state of public schooling in Hawaii. Teachers are furloughed and students are losing valuable classroom time. The state has chosen to use ARRA stimulus funds to fill budget gaps – which has been done by many states – but Hawaii has chosen to cut more than fill.

Arguably, Hawaii’s current education funding problems can be traced back to 2003 and a hard-nosed attempt at revenue-neutral education reforms – Fad-based reforms! Not fact-based ones. Off-the-Shelf School Finance solutions, as Doug Elmer and I describe in a recent article. (http://epx.sagepub.com/cgi/content/abstract/23/1/66)

Some historical context is provided here:

http://archives.starbulletin.com/2003/11/25/news/story2.html

Among other things, Hawaii’s leaders were misled in 2003 to believe that Hawaii already spent far more than necessary on its schools and that decentralized governance alone would solve their problems, driving more money to classrooms without ever having to add a dollar of new revenue.

The report by Bruce Cooper and William Ouchi concluded:

  • If Hawai’i were to reach classroom spending of 65 cents out of each education dollar, it would mean an additional $46,250 to spend on each classroom per year. This diversion of money to non-core uses is typical only of very large school districts.[1]
  • The results of our study bear on the consideration by the state of moving to a new system of management, Weighted Student Formula (WSF).

But this was an argument based on shoddy analysis and poorly documented summaries of state spending (actually, state and local total revenue) – comparisons which the authors of the original report even failed to understand. Yet, their message stuck with Hawaii policymakers.  No more money for schools. Just structural (read superficial) reform.

Oddly enough the original Cooper/Ouchi report which chastised Hawaii’s Board of Education for spending way to much to begin with and driving less than 65% to the classroom, never actually provided legitimate analyses supporting the secondary conclusions of that report – promote decentralized governance and implement a weighted student formula with the money you already have! Doug Elmer and I discuss these issues in this article: http://epx.sagepub.com/cgi/content/abstract/23/1/66

This whole series of events provided the governor and legislature in Hawaii the platform to continue starving the state’s education system while placing blame on the State Board of Education for not acting on their reforms, which in their view, would have solved everything. http://www.kpua.net/news.php?id=9232

Hawaii’s education system problems run much deeper than any superficial, off-the-shelf management guru strategy can solve.

Hawaii is among the few states where fewer than 80% of 6 to 16 year old children attend the public school system (78.8% according to American Community Survey 2005 to 2007). Yes, less than 80% of children in the age groups where most kids attend public schools are in Hawaii’s public schools. And yes, they are the lower income kids compared to their peers in Hawaii private schools.

That said, Hawaii’s educational effort (share of Gross state product spent on public schools) is relatively average to above average among states. Further, cross state comparisons of Hawaii’s educational spending provide mixed messages:  Hawaii’s current spending  – depending on how it’s measured and/or how it’s adjusted for regional cost variation is relatively average to above average (looking at total state and local revenue) or below average (looking at current expenditures per pupil) , adjusted for regional costs.

During the recent economic downturn, Hawaii’s total state revenue decline has been near the middle (upper middle) of the pack nationally – total state revenue losses from peak to June 2009 (p. 20 and 21), according to this Rockefeller Institute Report (best site for this stuff):

http://www.rockinst.org/pdf/government_finance/state_revenue_report/2009-10-15-SRR_77.pdf

This very recent WSJ article (http://online.wsj.com/article/SB125635093976805443.html) shows how Hawaii’s education funding cuts compare to those in states like California, Florida, Georgia and New Mexico – all of which have experienced much greater declines in total state revenue than Hawaii as of earlier this year – according to the Rockefeller Institute analyses linked above.

Even though Hawaii’s total state revenue is not declining as fast as these other states, Hawaii’s cuts to public schools have been comparable or even greater.

A few years back, Scott Thomas (now at Claremont Graduate School) and I were asked to provide analyses for and guidance to the Hawaii Department of Education regarding implementation of the decentralized weighted student funding plan which had been adopted as part of the comprehensive reforms of 2004.  To a large extent, our attempts at modeling financial redistribution options across Hawaii’s schools under revenue neutral assumptions proved to be an exercise in re-arranging deck chairs on the titanic.  Our two reports can be found here:

Part I – includes executive summary and conceptual framing of analyses, along with comparisons to other state formulas

http://sites.google.com/site/schoolfinancepolicy/consulting-reports/Hawaii.Part1%262.2006.pdf?attredirects=0&d=1

Part II & III – includes specific analyses of teacher labor markets, distribution of teachers by qualifications across richer and poorer neighborhoods, locations & islands, and concludes with simulations of redistribution options

http://sites.google.com/site/schoolfinancepolicy/consulting-reports/Hawaii.Part3.2006.pdf?attredirects=0&d=1

On page 34 of the second report, Scott Thomas and I explain:

=======  Begin Excerpt

A recent New York Daily News (7/2/06) editorial opined:

“Rather than simply pumping more gas into this broken down car, it’s time to design a much smarter and more effective way to get from Point A to Point B. A reform idea called ‘weighted student funding’ does just that, making intelligent use of the resources we already devote to education. How? Unlike the current system—which funds school districts through an incredibly complicated calculus—weighted student funding ties the money to the student.” (Cooper)

Increasingly, pundits supporting this view of WSF use the analogy of students carrying with them a need-based backpack of funding. Hawai‘i’s BOE and Committee on Weights now recognizes that in a system already constrained by limited resources, targeting sufficient need-based weighting simply costs more, not less or the same amount of money. As noted in our original report, we do not envy the members of committee charged with redistributing limited resources. If, as our estimates suggest, some schools need 40% more than others on the basis of poverty alone (we believe this to be a low estimate), and if this is to be done with no new money added to the system, then others must necessarily give up 40% of their funding.

In other words, assume Johnny and Malaya both need backpacks and currently they both have $10, sufficient to buy an ordinary backpack at Target or Wal-Mart. But, Malaya, by virtue of combined economic disadvantage and limited English proficiency, needs a $20 backpack. Johnny may need only an $8 backpack—the cheapest available (but with less padded shoulder straps than Johnny is used to). Unfortunately, if we redistribute the necessary resources to Malaya, then Johnny is out of luck altogether. If we leave Johnny with enough for the $8 backpack, then Malaya is out of luck. It’s a lose/lose proposition. For both Johnny and Malaya to get the backpack (read education) they need through a WSF, we will likely have to find more money. We ourselves might view this issue differently if it was plainly obvious that Hawai‘i’s schools are flush with funds and simply squandering those funds on unnecessary, frivolous endeavors. We lack any evidence to support this conclusion.

======= End Excerpt

While I’ve not followed Hawaii closely for the past few years, it would appear that this ship has now begun to sink – widening the gap between the fewer than 80% of children left in public schools (on the ship) in Hawaii and the 20% from first class who had access to life rafts.

I find it most disturbing that much of this mess may have been avoidable had it not been for purely political interests and self-absorbed snake-oil salesmen ready and willing to serve those interests with the simple message that money can’t fix schools.  Off-the-shelf reforms like WSF can!

The reality is that substantive education reform often costs money – sometimes a lot of money and sometimes a lot more than the amount already being spent. Automatically assuming that there’s enough money in a system just because it looks like a big number is not enough. More detailed analysis is required. You can’t starve a system into reform, especially if the reforms cost money. Unfounded assumptions and arguments that there’s plenty of money and that money doesn’t matter and may never matter are not only absurd but are potentially very harmful. It would appear that Hawaii is now becoming a stark example of that harm.

You can rebuild the engine and transmission as many times and in as many ways as you want, but if you don’t eventually put gas in the car, it won’t run!

(my apologies for combining sinking ship metaphors, backpacks and cars that don’t run in a single blog post)