Tag: Cartel

Cartel Recap

This is an old post which I have moved forward in time on my blog because of the national release of this absurd film.

For series of posts on this topic, see: https://schoolfinance101.wordpress.com/category/the-cartel-movie-schlockumentary/

Okay… so a few people meandering through my posts over time have sought some synthesis of my gripes about Bob Bowdon’s Cartel Movie. First of all, here’s a link to a pretty good review of the film which I just found yesterday: http://www.nj.com/entertainment/tv/index.ssf/2009/10/the_cartel_movie_review_docume.html

  • The divisive, emotional and complicated debate now raging over powerful public teachers unions and “school choice” — a catchphrase that encompasses support for vouchers, charter schools and a variety of other reforms — could use a comprehensive sorting-out by a diligent observer. Bob Bowdon’s smarmy diatribe isn’t it.
  • In taking to task the sorry state of our public schools, former New Jersey TV personality Bob Bowdon employs the three R’s of bad filmmaking: righteousness, revilement and redundancy.

And these glowing reviews accept as a given, Bowdon’s “statistical” argument validating the crisis of schooling in New Jersey.

Here’s my own synopsis of the arguments behind the film – the Crisis that necessitates the Solution.

The Crisis (Bowdon’s Crisis)

There’s a crisis in education in America and more specifically in New Jersey. Quite simply, every country in the world is handing us a beating as a nation and as a state, despite the massive amount of money we are throwing down the rat-hole of our public education system.

Bowdon’s evidence of a crisis:

Bowdon complains of our lagging national performance by making comparisons of international assessments such as PISA to other countries (critique of the relevance of PISA here). Here, Bowdon twists the argument to specifically blame states like New Jersey, which are not only a part of this substandard American education system, but are emblematic of it, by spending obscene amounts of money for these failures. Okay… so here’s the basic logic:

  • Our national average test scores are bad compared to other countries
  • New Jersey spends a lot on schools, and is part of this terrible national system
  • Therefore, spending is bad, our schools are terrible nationally, and New Jersey is even worse

But, as I discuss here: https://schoolfinance101.wordpress.com/2009/06/17/vacuous_bowdon/

New Jersey actually performs very well even on international comparisons, in a legitimate, rigorous statistical analysis by the American Institutes for Research (http://www.air.org/files/International_Benchmarks1.pdf) And, our national average is only as low as it is because of our many very low spending states that have chosen to throw their public education systems under the bus. Can’t blame New Jersey’s high spending for Louisiana and Mississippi’s low performance Bob. (some useful comparisons on this more recent post: https://schoolfinance101.wordpress.com/2010/02/23/common-standards-and-the-capacity-to-achieve-them/)

In an effort to further the argument that New Jersey schools in particular are an abomination, Bowdon points out how New Jersey is by a long shot (okay, I’m exaggerating his point here), first in the nation (if not the world) in spending on schools. Yet, if you correct NJ graduation rates to count only those kids who pass the NJ state tests, we’re only 24th on graduation rate. Yep, mediocre at best for all that money. Down the rat-hole. Clearly, the kids who graduate high school in all those other states, like Tennessee for example, must be able to pass the NJ state tests. Oh wait, they don’t take the NJ tests, do they? Another really dumb comparison Bob (a comparison originally generated by E3, but in the context of a broader critique of graduation rates).

This new report (as well as an older version) shows that the NJ tests aren’t really the least rigorous tests out there: http://nces.ed.gov/nationsreportcard/pubs/studies/2010456.asp. Not great. But not the worst either. Yes, if we’re going to have tests, we should expect kids to pass them. No excuses there. But the graduation rate comparison is still completely bogus. I address this topic in greater detail here: https://schoolfinance101.wordpress.com/2009/10/16/more-cartel-garbage-bowdon-still-an-idiot/. Oh, and by the way, as I point out in that same post, NJ is in good company on per pupil spending, rarely actually topping the list.

The icing on the cake is the analysis Bowdon originally presented as part of his “Facts and Figures” to support his “crisis” case . This still stands as the absolute dumbest analysis I have seen or read pretty much anywhere in my years working in education policy research (okay, this one comes close: https://schoolfinance101.wordpress.com/2009/05/13/should-think-tanks-be-licenced-to-think-and-when-should-a-license-be-revoked/) . Here, Bob Bowdon explains his brilliant revelation that states which spend more on their schools have lower SAT scores – so spending more lowers SAT scores… or at least those states that do spend more simply waste it so badly that SAT scores go down… for some reason. I tackle this outright stupidity in my first post on the topic: https://schoolfinance101.wordpress.com/2009/05/30/idiot-of-week-award-the-cartel-check-this-out/ (While Bowdon has removed much of this completely ridiculous content from the movie site, the logic of his current site content remains the same, and these absurd comments/arguments represent the level of Bowdon’s thinking at the time the movie was initially released. I saved copies of the original SAT graphs. They make great teaching examples of deeply flawed reasoning!)

The solution to the crisis that may not exist:

Okay, so if Bowdon can’t concoct his crisis, there’s really no need for a solution to it. You know, it might not actually be that hard to do a reasonable run through some real numbers to point out some serious problems, inequities and inadequacies in our education system as a nation and in New Jersey schools. They are certainly far from perfect. But, Bowdon can’t seem to string together even one set of legitimate, well argued facts to make such a case. So, I could stop here. By Bowdon’s absurd evidence, no crisis actually exists, therefore, no need for solution. But of course, Bob has one:

The only possible two solutions – Charter schools and Vouchers to private schools – with emphasis on the former. Everyone knows that money doesn’t solve education problems, Charters and Vouchers do (only if they’re well funded, though). Now, let me qualify here that I am a fan of charter schools having been a founding member of the special interest group on charter school research of the American Education Research Association and having written research articles which find favorable results for charter schools regarding academic quality of teachers (http://epx.sagepub.com/cgi/content/abstract/20/5/752) . I’m also a fan of private schools, having taught in one of NYC’s most elite independent day schools and having written on private school finances (http://www.epicpolicy.org/files/PB-Baker-PvtFinance.pdf) . But sadly, my actual knowledge of Charters and Private schools makes it harder, not easier to accept Bowdon’s poorly conceived arguments.

On Charters: Bowdon points to a few specific charter schools that are doing very well compared to other schools. Great. Some schools do better than others. I’m good with that. But, Bowdon seems to argue that because these few schools are good, all charters are good – certainly better than any traditional public school. Therefore, it is an outrage that the state of NJ won’t simply throw the doors wide open to more charters to accommodate the tear filled rooms of parents awaiting their chance at the opportunity to send their kids to one of the many outstanding charter schools. Here’s the glitch in this logic. I explain here (link below) that the average performance of Charter schools is statistically no different from the average performance of poor traditional public schools in NJ. Yes, some are better and many, many are much worse. The chances that a student in a charter is not in a low performing school are only marginally (very marginally) better than for students in the poorest (comparably poor) traditional public schools. While some charter school research shows strong positive results, the balance of that research shows a break-even, on average (see my post: https://schoolfinance101.wordpress.com/2009/10/20/a-few-quick-nj-charter-school-facts-figures/) and NJ charters are no different.

For updated and more extensive analysis of NJ charter schools, see: https://schoolfinance101.wordpress.com/category/new-jersey-charter-schools/

Convincing inner city families that Charter schools will save their children simply because they are charter schools and therefore they must be better than traditional public schools is disingenuous at best. I have no problem whatsoever arguing that parents should have the option to choose a “better” school and should be provided reasonable information to aid them in choosing a legitimately better option for their children. Information is the ultimate equalizer here. Contributing to and/or concocting misinformation – creating a “market for lemons” by distorting information – when the stakes are this high – merely to advance a political agenda and build reputation as a supposed “documentary” film producer is morally repugnant.

Finally, on the private school voucher side of the argument: Like I said, I’m a big fan of private schools and I’ve seen what money can buy in the best of private schools. By the way, I report here on the actual per pupil spending of private schools by the affiliation of those schools (https://schoolfinance101.wordpress.com/2009/08/18/private-school-spending/). When it comes to private schools, like Charter schools or traditional public schools, you get what you pay for, and the average per pupil spending (not tuition, but actual spending) in private independent day schools in New Jersey hovered around $25,000 to over $30,000 in 2007. Urban Catholic school per pupil spending is on par with Charter spending, and only conservative religious schools spend much less. Note that Catholic schools, like Charter schools are struggling these days to operate at such low expense (around $12k per pupil). Providing vouchers at levels similar to charter funding would ensure that the only choices available to parents would be financially struggling Catholic schools or conservative religious schools. There would be no religious neutrality in the options available. Private independent schools would remain well out of reach. Double the voucher level and you might get somewhere, but demand for slots would likely far outpace supply (see for a fun paper on price elasticity and private school attendance: http://www.nber.org/~dynarski/w15461.pdf). Under-subsidized vouchers are a cruel hoax, like distorting information on the true variance in charter school quality.

There are other potential forms of choice here, which are noticeably absent in Bowdon’s arguments (unless I’m missing something). Hey, look at my graph of school performance by DFG in my charter school post:  https://schoolfinance101.com/wp-content/uploads/2009/11/updated-charter-rel-performance.jpg Wouldn’t it be nice to provide open enrollment choice options for kids from the urban core to attend the high performing affluent suburban schools? Why should we only let them choose from the relatively average, under-resourced charter or religious private schools? Seems a little unfair, don’t you think? Seems a little disingenuous to argue that choice will solve our problems as long as we only let the poor minority children in the urban core attend start-up charter schools in church basements and other makeshift rental properties (since the slots in the elite, high performing charters are taken) and low tuition, low spending exclusively religious private schools. You wouldn’t want to include all of those higher performing traditional public schools a few NJ transit stops away.

Summary

So, here it is in a nutshell:

  1. Even if there is a crisis, Bowdon provides no legitimate evidence of one, and in fact, provides laughable claims that make it hard to take him seriously at all;
  2. Since there is no validated crisis, there is no need for a solution, but Bowdon offers one anyway;
  • Instead of attending NJ’s dreadful traditional public schools, students should flock to NJ’s outstanding charter schools, which, it turns out, have average performance the same as the poorest NJ traditional public schools, or
  • NJ children should be provided vouchers at levels that will allow them only to select from cash strapped urban religious private schools.

Seems reasonable enough. Ill-conceived? Intellectually vacuous? Schlockumentary? I must stop myself.

As a professor of school finance who lives every day immersed in national and state databases on school funding and student outcomes and who has advised many national organizations on the development of indicator systems for comparing schools/districts and states, Bowdon’s presentation of “shocking statistics” is quite honestly the most offensive, absurd and amateur presentation I have ever witnessed – regardless of political angle.

Cheers.

Replicating Robert Treat Academy

With little doubt, Robert Treat Academy in Newark is one of those charter schools that is doing well by common outcome measures and likely by even more important measures than state tests. What we know about are the tests. And even if one controls for a variety of factors about student populations, Treat’s test scores are pretty darn good.

Here’s a figure from a model I re-ran the other day (based on older work), using a variety of school, student population and community factors to control for expected differences in student outcomes. Schools above the line are those that outperformed expectations and those below the line fell below expectations. Charters are in red, and again, there are roughly equal numbers of traditional publics above and below the red line and charters above and below the red horizontal line. Treat is one of those above the line.

Treat Beat

So the argument goes, Treat is producing these test scores with much less money, and therefore we should be able to do the same, with similarly less money across poor urban settings by emulating the Treat model.

I addressed in a previous post how charter schools receive less through the state aid formula than traditional public districts. Again, this should shift somewhat over time, but charters will remain relatively disadvantaged. Using Robert Treat’s IRS 990 for 2007 expenditures (instead of their NJDOE reporting of their expenditure of public charter funding only), Treat shows expenditures per pupil in 2007 around $12,600. I’m still not sure I’ve captured the full expenditure here, because Treat’s IRS 990s show unusually low levels of private contribution for a successful charter school.

That aside, is the Treat miracle replicable across Newark? Or, is Treat different in substantive ways that can’t be spread throughout the system. Here are a few numbers that raise concern.

First, as I noted on a previous post, Robert Treat’s student body is only 3.8% special education in a district with an average of 18.1%.  This is from the special education classification data from NJDOE. In the enrollment files, Treat reports 0%. At 100% additional average expenditure per special education pupil, matching district demographics would raise Treat’s expected spending to $14,868 (1.18 x 12,600 in 2007).

Second, while Robert Treat does show about 62.4% students qualifying for free (130% poverty level) and reduced (185% poverty level) lunch, the free lunch share is about 42.9%. That is, Treat’s free or reduced share is boosted by the share of children who are more well off among the less well off. Note that the model I used above used Free & Reduced shares, not Free alone or the ratio between them.

By contrast, Newark Public Schools in total has 82% free or reduced and 71% free lunch alone.

Treat also reports less than 1% limited English proficient students while Newark City schools report 8.7%.

It’s one thing for me to try to control for these differences in estimating who does and does not “beat” odds, but yet another to take a model that has been successful under certain circumstances and apply it widely under very different circumstances, at the same cost.

It’s all well and good to cite other studies from other cities  and states that show that charter schools on average aren’t “cream-skimming,” (where most of those comparisons are based either on student’s initial performance or on free + reduced shares) but the reality in this case is that Treat Academy is producing its current level of outcomes at its current price tag with a substantively different student population – most notably the absence of children with disabilities. Again, they’re doing well, and even in models I’ve run controlling for some of these things, they still stand out and should be applauded for their efforts and results.

But, given the demography of the entire student population of Newark in particular, replicating this model may prove difficult. Adding more schools that serve fewer of the poorest children and few or no children with disabilities may be significantly problematic for those schools which then serve the larger shares of both.

Charter Averages Worse than Originally Estimated

Note: The information below is not a comprehensive research study on the relative effectiveness of New Jersey Charter Schools. Rather, it is a quick summary of average proficiency rates for charters compared to other New Jersey schools by socio-economic strata. Unfortunately, New Jersey Charter schools were not part of two major recent multi-state analyses of charter school effectiveness, which can be found here, along with reviews & critiques of those studies. http://www.epicpolicy.org/think-tank/reviews These studies also found mixed results, with charters in some states slightly outperforming their public school counterparts, in other states performing comparably and in others performing less well. I have put together this post merely to stimulate conversation on how NJ charter schools are doing and perhaps encourage additional more thorough research.

In my original post on NJ Charter School performance, Charter schools appeared to be performing somewhere between performance levels of DFG A and DFG B traditional publics. Here’s one of the graphs to that effect.

% Proficient for All Tested Students

Note that the charter line – R – falls between the DFG A (poorest traditional publics) and DFG B lines. But, this analysis includes all tested students. While I expected that children with disabilties were underrepresented in Charter schools, I had no idea just how under represented until I took a look, here: https://schoolfinance101.com/wp-content/uploads/2009/11/charter-special-ed.jpg

For example, Robert Treat Academy has 3.8% and North Star Academy 7.8% children with disabilities in a district that has 18.1% in 2007. These are higher than many, which actually serve 0%.

So, correcting for this problem by looking only at General Education students, the graph above becomes the graph below:

updated charter rel performance

In this graph, the Charter line maps almost precisely with that of the DFG A line. That is, the slightly higher performance in the first graph is almost entirely a function of the fact that NJ Charters simply don’t serve children with disabilities and don’t have them in their test taking pool. My apologies for this apparently glaring omission.

The biggest change to my analysis however is in the relative probability that a student attends a tested grade level where less than 40% of students are proficient or higher. Making the above correction, leads to the finding that a child in a charter school is 35% more likely than a student in a DFG A traditional school to be in a tested grade level where fewer than 40% of general education students scored proficient or higher.

Here’s the logistic regression, weighted for number of test takers in grade level and on test (general education only), based on the 2008 report card data:

Logistic Regression of Low Performance Grade Level (<40% prof. or adv.)

DFG A is the baseline comparison group. An odds ratio of greater than 1.0 indicates a greater likelihood of being in a grade level with fewer than 40% proficient or advanced than in a traditional DFG A school. Only charters have a greater likelihood – and much greater – 36% greater. Likelihoods vary dramatically for the different tests and subject areas. Apparently, 6th grade tests have cut scores aligned such that many more students do poorly on them. I don’t think that it’s just that 6th graders get dumb for  a year. Newer tests take some tweaking. Note the dip in previous graphs. Note also that in affluent communities (GH through J), there is statistically no chance of being in a low performing grade level.

Here’s a link to the School Reports 2008 Data:

http://education.state.nj.us/rc/rc08/database/nj_rc08.xls

Please – take your own stab at this. I’ve been running these quickly. My Stata data are here.

New Update: Here’s my last shot at it for now. I’ve got the odds for charters down to about 25% greater chance than DFG A schools of being in a grade level where fewer than 40% were proficient or higher. Unfortunately, poverty rates among test takers were only calculable at the district level (and for charters) not school and charters with these data (must use the enrollment data for whole school for that). Also, NJDOE continues the habit of not identifying specific locations of charters in their coding system by county. I have a bridge file somewhere, constructed by zip code, but for charters through 2006. May revisit. Anyway, here’s the logistic regression:

updated logit

Ah the perils of goofing around with data too quickly/on the fly. Fun though.

NJ School Funding Suburban Taxpayer Scam?

I hate wasting so much time countering completely absurd claims, like those that spill out on the E3 Cartel commercials. This is a short reply this time. At the end of one of the commercials, the spokesperson slips in the claim that not only are we wasting a ton of money on our low graduation rates in poor urban schools (I discuss this claim here: https://schoolfinance101.wordpress.com/2009/10/31/cartel-recap/), but this whole inefficient mess is a “suburban taxpayer scam.” Yep, suburbanites (like myself) are being dreadfully over-taxed and our hard earned money is being thrown down the rat-hole. We don’t get any of it back.

A simple question to answer here is whether the property tax effort in suburban communities (however we are supposed to define suburban?)  is that much greater than in “urban” communities. An appropriate way to measure this is by calculating the percent of income paid in property taxes.

Here’s a quick snapshot of tax effort in Essex County by income level and in Monmouth county by income level. These data are taken from http://www.nj.com/news/bythenumbers/, and the data are generally from 2005. Most “Abbott” funding to school districts had scaled up between 1998 and 2005.

Essex Tax Effort

Hmmm… no systematic pattern here. Yep, some pretty big differences, but no systematic pattern between poorer and wealthier communities.

Monmouth Tax Effort

As it turns out, tax effort in Monmouth declines systematically as homeowner income increases. Perhaps this is the “urban tax scam” not suburban one?

Yes, the property tax bill in an affluent suburban community is larger – because it is the tax bill on a more expensive home!  (should I really have to say that?) Yes, low property value, low income communities receive higher rates of state subsidy through the state aid formula for schools. That’s generally how aid equalization formulas work. And yes, New Jersey’s aid is targeted to higher need districts, above and beyond typical equalization (but only since 1998-2003).

Let’s get this straight. If the idea of the funding formula was to send back to communities and school districts exactly the amount submitted to state coffers from residents of those communities – then why the heck would we be collecting it to begin with? This would be a particularly foolish exercise since it costs money to process the tax revenues and send them back. That’s how taxes work – whether collected at the municipal level, providing benefit to the people across the street whose house may be valued (taxable value) less than yours, and tax bill may be proportionately less, or across the state. For those who don’t quite understand this, I recommend the Schoolhouse Rock tune about the Taxman. Pretty good stuff!

In a previous post, I also explain how local media in NJ has distorted comparisons of New Jersey property taxes with other states – https://schoolfinance101.wordpress.com/2009/10/03/should-nj-really-try-to-be-like-de-md-mo-ga-wa/

The Intellectually Vacuous Bob Bowdon’s “Cartel”

See updated post on this topic: https://schoolfinance101.wordpress.com/2010/04/16/cartel-recap/

===== Old post

Had a busy week, so I haven’t posted, but saw a new report yesterday which relates nicely back to the shallow logic of Bob Bowdon’s intellectually vacuous Cartel movie.

The Cartel movie is based on the premise that (a) public schools nationally are failing, (b) public schools in the US spend a ton of money to achieve little, (c) New Jersey is the perfect example of a state which spends a ton of money and fails. All of this, of course, occurs because of a self-interested, self-indulgent cartel of teachers unions and greedy bureaucrats (here’s how their salaries stack up to those “real world” “private sector” workers in NJ). I’ll avoid this latter piece for now, and take a closer look at the logic of points “a” through “c.”

Bowdon cherry picks some national average results from the PISA international assessment of 15 year old students to show that the US compared poorly on math in 2003 and worse in 2006. Of course, any national averages in the U.S. combine the performance of children in states that have largely thrown their public schooling system under the bus  – like Louisiana and Mississippi among others – with those that have done quite well like Massachusetts and New Jersey (indeed it is somewhat unfair to compare directly LA and MS to MA and NJ).

As I have shown in recent posts, there does exist at least some relationship between state aggregate spending (controlling for a variety of factors) and national assessment performance – albeit a relationship heavily entangled with socioeconomic conditions and adult population education levels in states.

Further, as I have also explained previously, an extensive body of research on the effects of school finance reforms including infusion of new resources into poor schools, shows significant positive effects.

A new study out this month from the American Institutes for Research seeks to make more appropriate statistical comparisons of student math performance on another international assessment – TIMSS (Trends in International Math and Science Study). The authors construct a statistical cross-walk between NAEP state assessment scores and TIMSS scores which can be used for international comparisons.  From this analysis, the authors are able to evaluate where individual states stack up against countries participating in TIMSS. This is important because of the variance in state level performance and differences in state policies, fiscal effort and students served.

For starters, on international comparisons, the US on average scored just below the mean for OECD (organization for economic cooperation and development) countries at the 4th and 8th grade level (we do lag from 4th to 8th, an issue of concern). At both 4th and 8th grade on math, the US average is well above the international mean for all TIMSS participants. Now, we may wish to do better – and should. AIR assigns grades to the score ranges for each country and points out that we don’t perform at the levels we should. But this is far from the absurd, apocalyptic (and simply irresponsibly misguided) view presented by Bowdon.

But wait, Bowdon’s premise is that states like New Jersey are the perfect example of inefficiency – spending so much yet producing these terrible national averages. Certainly, New Jersey can’t be blamed for the national average – which carries with it the baggage of states like Louisiana and Mississippi.

How does New Jersey compare to the OECD average? New Jersey ranks 3rd among states on 4th grade math with 25 states beating the OECD average performance. Not bad for Jersey, along with Massachusetts and Minnesota! Louisiana, Alabama, New Mexico, California and Mississippi carry up the bottom end of the rankings, falling below the OECD mean, but above the overall international mean. That is, even Mississippi and Louisiana beat the international mean.

New Jersey drops a little on 8th grade math (consistent with other NAEP based analyses of NJ), but still does well, coming in 6th among the 27 states which perform above the OECD mean. Again, even Louisiana and Mississippi exceed the international mean, but well below the OECD mean.

I am by no means arguing for complacency  – saying – hey – that’s good enough. Rather, my point here is to re-emphasize that the US has a wide variety of education systems in place across states – some which spend a great deal and in fact perform very well, even in international comparisons. New Jersey is among them. We also in this country have some states that have seriously neglected their education systems, spent little, and shifted large shares of (primarily upper class) children in private schooling (schools that spend more, not less than the public schools in those states) where their performance goes unmeasured in these international and even state by state comparisons (in fact, these may be the children who do well in those states, but we don’t know). WHAT THESE STATES HAVE DONE IS A NATIONAL CONCERN!

It is foolish stretch of logic to blame New Jersey’s high spending (and the Cartel that demanded it) for the poor national average performance on select international comparisons. Yes, New Jersey spends on education, and in fact, New Jersey does quite well with that spending compared to other states and on international comparisons.

Certainly, spending alone is not the solution. But little is added to the debate by producing bombastic, misguided, poorly conceived and irresponsible slick-production rhetoric posing as documentary.